2015/02/13 by Giovanni Abramo, Ciriaco Andrea D’Angelo, Ciriaco Andrea D'Angelo
Computer Science · Decision Sciences · Mathematics · Psychology · #Accounting #Bibliometrics #Business #Computer science #Data mining #Data science #Economics #Information retrieval #Marketing #Mathematics #Operations research #Psychology #Ranking (information retrieval) #Set (abstract data type) #Social psychology #Strengths and weaknesses #Valuation (finance) #cs.DL #scientometrics and bibliometrics research
paper · pdf · doi:10.1002/asi.23323
published as Abramo, G., D'Angelo, C.A. (2015). The VQR, Italy's second national research assessment: Methodological failures and ranking distortions. Journal of the Association for Information Science and Technology, 66(11), 2202-2214
openalex publication_date 2015/02/13 · openalex created_date 2016/06/24 · arxiv created 2018/10/31 · arxiv updated 2018/11/01 · openalex updated_date 2026/08/05
The 2004–2010 VQR ( R esearch Q uality E valuation), completed in J uly 2013, was I taly's second national research assessment exercise. The VQR performance evaluation followed a pattern also seen in other nations, as it was based on a selected subset of products. In this work, we identify the exercise's methodological weaknesses and measure the distortions that result from them in the university performance rankings. First, we create a scenario in which we assume the efficient selection of the products to be submitted by the universities and, from this, simulate a set of rankings applying the precise VQR rating criteria. Next, we compare these “ VQR rankings” with those that would derive from the application of more‐appropriate bibliometrics. Finally, we extend the comparison to university rankings based on the entire scientific production for the period, as indexed in the Web of Science.