vix.ing · top · new · best · stats · spec

The disruption index suffers from citation inflation: Re-analysis of temporal CD trend and relationship with team size reveal discrepancies

2024/06/21 by Alexander M. Petersen, Felber Arroyave, Petersen, Alexander M. +3 · 3 citations
Decision Sciences · Business, Management and Accounting · #scientometrics and bibliometrics research #Intellectual Capital and Performance Analysis #Innovation and Knowledge Management

paper · pdf · doi:10.1016/j.joi.2024.101605

Abstract

Measuring the rate of innovation in academia and industry is fundamental to monitoring the efficiency and competitiveness of the knowledge economy. To this end, a disruption index (CD) was recently developed and applied to publication and patent citation networks ( Wu et al., 2019 ; Park et al., 2023 ). Here we show that CD systematically decreases over time due to secular growth in research production, following two distinct mechanisms unrelated to innovation – one behavioral and the other structural. Whereas the behavioral explanation reflects shifts associated with techno-social factors (e.g. self-citation practices), the structural explanation follows from ‘citation inflation’ (CI), an inextricable feature of real citation networks attributable to increasing reference list lengths, which causes CD to systematically decrease. We demonstrate this causal link by way of mathematical deduction, computational simulation, multi-variate regression, and quasi-experimental comparison of the disruptiveness of PNAS versus PNAS Plus articles, which differ primarily in their lengths. Accordingly, we analyze CD data available in the SciSciNet database and find that disruptiveness incrementally increased from 2005-2015, and that the negative relationship between disruption and team-size is remarkably small in overall magnitude effect size, and shifts from negative to positive for team size ≥ 8 coauthors. • The disruption index (CD) is biased by secular growth, which limits the accuracy of cross- temporal analysis. • We apply mixed methods: deductive reasoning, computational modeling and empirical multivariate regression. • CD converges to 0 as the number of references increases, which explains extremely narrow empirical distribution. • Temporal CD trend indicates that disruptiveness increased from 2005-2015. • Relationship between CD and team size is non-monotonic, weak, and increases for team size ≥8.

Citations

Cited by

Related