The metric tide and its hangover: the British experience in research evaluation and management
Article Sidebar
Main Article Content
Abstract
This article reviews the British experience with the use of metrics in the evaluation and management of research, moving from an initial period of “enthusiastic” implementation to a period of cautious review and concern about the distorting effects they have had on scientific practices and their results. There is currently a concern and a need for a cultural change in the use of metrics that reverses the harmful consequences of their irresponsible use for the scientific-academic system, its institutions and individuals. Following the implementation of New Public Management, the UK was one of the first countries to establish an audit culture based on managerial criteria brought in from the private sector. What the evidence analysed in the reports of a review of independent experts convened by the British government shows is that managerial rationality ends up, in many cases, prevailing over the rationality of science itself. Members of the scientific community end up having to adapt, deviating from their objectives and, sometimes, having to take shortcuts to the good practices that the community itself has established. In response to this situation, a number of initiatives are being promoted in the UK to counteract this trend. As this situation is also found in other countries, other recent proposals to restore to the academic community and research cultures effective leadership and control over their own management are reviewed.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
References
Arbix, G., Salerno, M. S., Toledo, D., Miranda, Z. & Álvarez, R. (2010). Inovação: estratégias de sete países. Brasília: ABDI.
Beer, D. (2016). Metric power. London: Palgrave Macmillan.
Benedictus, R., Miedema, F. & Ferguson, M. W. (2016). Fewer numbers, better science. Nature, 538, 453-455. https://doi.org/10.1038/538453a
Colquhoun, D. (1 de diciembre de 2014). Publish and perish at Imperial College London: the death of Stefan Grimm [mensaje en el blog DC’s Improbable Science]. Recuperado de http://www.dcscience.net/2014/12/01/publish-and-perish-at-imperial-college-london-the-death-of-stefan-grimm/
Curry, S., de Rijcke, S., Hatch, A., Pillay, D., van der Weijden, I. & Wilsdon, J. (2020). The changing role of funders in responsible research assessment: progress, obstacles & the way ahead. RoRI Working Paper N° 3. http://doi.org/10.6084/m9.figshare.13227914
Declaration on Research Assessment (DORA). (2018). DORA roadmap: a two-year strategic plan for advancing global research assessment reform at the institutional, national, and funder level. Recuperado de https://sfdora.org/2018/06/27/dora-roadmap-a-two-year-strategic-plan-for-advancing-global-research-assessment-reform-at-the-institutional-national-and-funder-level/
Delgado López-Cózar, E. & Martí-Martín, A. (2022). Detectando patrones anómalos de publicación científica en España: más sobre el impacto del sistema de evaluación científica. ResearchGate [Preprint, Septiembre 2022]. Recuperado de https://www.researchgate.net/publication/363535388
Delgado López-Cózar, E., Ràfols, I. & Abadal, E. (2021). Letter: A call for a radical change in research evaluation in Spain. El profesional de la información, 30(3), 2-30. https://doi.org/10.3145/epi.2021.may.09
Dijstelbloem, H., Huisman, F., Miedema, F. & Mijnhardt, W. (2013). Why science does not work as it should? And what to do about it. Recuperado de http://www.scienceintransition.nl/app/uploads/2013/10/Science-in-Transition-Position-Paper-final.pdf
European Commission, Directorate General for Research and Innovation. (2019). Future of scholarly publishing and scholarly communication: report of the Expert Group to the European Commission. Publications Office. Recuperado de https://data.europa.eu/doi/10.2777/836532
European Commission, Directorate-General for Research and Innovation, Peters, I., Frodeman, R. & Wilsdon, J. (2017). Next-generation metrics: responsible metrics and evaluation for open science. Publications Office. Recuperado de https://data.europa.eu/doi/10.2777/337729
Forum for Responsible Research Metrics (FFRRM). (2018a). UK Progress towards the use of metrics responsibly. Three years on from The metric tide report. Recuperado de https://dera.ioe.ac.uk//31945/
Forum for Responsible Research Metrics (FFRRM). (2018b). FFRRM’s advice to the Research Excellence Framework (REF) 2021 panels, 26 jul. 2018.
Hazelkorn, E. (2015). The Obsession with rankings in tertiary education: implications for public policy. Recuperado de https://hepru.files.wordpress.com/2015/01/the-obsession-with-rankings-in-tertiary-education_wb_0115.pdf
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520, 429-431. https://doi.org/10.1038/520429a
Higher Education Funding Council for England (HEFCE). (2015a). The metric tide: correlation analysis of REF2014 scores and metrics. Supplementary report II to the independent review of the role of metrics in research assessment and management. https://doi.org/10.13140/RG.2.1.3362.4162
Higher Education Funding Council for England (HEFCE). (2015b). The metric tide: report of the independent review of the role of metrics in research assessment [entrevista de Steven Hill a James Wilsdon, 9 jul. 2015]. Recuperado de https://www.youtube.com/watch?v=hhci90s-WPw
López Ruiz, O. J. (2010). As inovações institucionais no Reino Unido. En G. Arbix, M. S. Salerno, D. Toledo, Z. Miranda & R. Álvarez (Eds.). Inovação: estratégias de sete países (pp. 246-280). Brasília: ABDI.
Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., Coriat, A. M., Foeger, N. & Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: fostering research integrity. PLoS biology, 18(7), 1-14. https://doi.org/10.1371/journal.pbio.3000737
Muller, J. (2018). The tyranny of metrics. Princeton: Princeton University Press.
Plackett, B. (2020). Five better ways to assess cience. Hong Kong principles seek to replace “public or perish” culture. Nature index. Recuperado de https://www.nature.com/nature-index/news-blog/five-better-ways-to-assess-science-research-metrics
Salatino, M. & López Ruiz, O. (2021). El fetichismo de la indexación. Una crítica latinoamericana a los regímenes de evaluación de la ciencia mundial. Revista iberoamericana de ciencia, tecnología y sociedad (CTS), 16(46), 73-100. Recuperado de http://www.revistacts.net/contenido/numero-46/el-fetichismo-de-la-indexacion-una-critica-latinoamericana-a-los-regimenes-de-evaluacion-de-la-ciencia-mundial/
Shore, C. (2008). Audit culture and liberal governance: universities and the politics of accountability. Anthropological theory, 8(3), 278-298. https://doi.org/10.1177/1463499608093815
Shore, C. & Wright, S. (2000). Coercive accountability: the rise of audit culture in higher education. En M. Strathern (Ed), Audit cultures: anthropological studies in accountability, ethics and the academy (pp. 57-89). London: Routledge.
Strathern, M. (Ed). (2000). Audit cultures: anthropological studies in accountability, ethics and the academy. London: Routledge.
Tickell, A. (2022). Independent review of research bureaucracy. Recuperado de https://www.gov.uk/government/publications/review-of-research-bureaucracy
Wilsdon, J., Allen, L., Belfiore, E. Campbell, Ph., Curry, S.,… Johnson, B. (2015). The metric tide: report of the independent review of the role of metrics in research assessment and management. https://doi.org/10.13140/RG.2.1.4929.1363
Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S.,… Franssen, T. (2015). The metric tide: literature review (Supplementary report I to the independent review of the role of metrics in research assessment and management). https://doi.org/10.13140/RG.2.1.5066.3520
Zhang, L. & Sivertsen, G. (2020). The new research assessment reform in China and its implementation. Scholarly assessment reports, 2(1), 1-7. https://doi.org/10.29024/sar.15