Technopolis Group

Technopolis completed two studies on the UK’s Research Excellence Framework 2021 (REF 2021). REF 2021 was the UK’s largest national research assessment exercise to date and encompassed submissions from 157 higher education institutions (HEIs) and more than 76,000 researchers. It will determine the selective distribution of a substantial volume of institutional research funding, which could amount to more than £15bn between 2022/23 and 2029/30.

Technopolis analysis of REF 2021 outputs explored ways in which the micro data generated by the latest cycle might yield insights into the health of disciplines or research areas. The study included four areas of experimental analyses of REF 2021 data:

In bringing together the four analyses, we found that on-going developments in data science allow implementing automated processes (many of which rely on machine learning) to analyse certain concepts, at scale. However, the implementation of the different projects also showcases that (i) the successful implementation of data science methods requires not only the use of specialised skills but also a great of prior work in terms of structuring and curating the data information in a way that is meaningful to the analysis; (ii) these methods do not replace the need for thematic expertise and peer-reviewed assessment.

Through this project, the funding bodies gained insights into the feasibility of carrying out a more granular analysis of research assessment data, or analysis of cross-cutting themes in future exercises.

Technopolis also delivered a study to estimate REF 2021 costs and analyse the balance of costs against benefits played out in REF 2021. We estimate the total cost to the UK of running REF 2021 was around £471M, which amounts to around £3m per HEI on average for the 7-year cycle and around £6k for each researcher submitted. This figure constitutes a substantial increase in the costs borne by the higher education (+50%) community when compared with the costs of submitting to the preceding national assessment exercise, REF 2014. The overall cost amounts to around 3-4% of the funding expected to be distributed, which is more economical than the costs of the peer-review procedures (c. 12% of funds distributed) used by the grant-awarding research councils.

The higher cost of REF 2021 was driven in large part by an expansion in the scope of the exercise and in particular by the inclusion of a substantially larger cohort of researchers (+50%). From this perspective, the increased costs are in part mirrored by increased benefits in the shape of a more inclusive process that has provided greater equity to individual researchers. UK HEIs value many of the cultural shifts resulting from these changes, such as greater ability to deal with researchers’ individual circumstances or more open research practices. Indeed, the great majority of HEIs surveyed by Technopolis believe REF 2021 represented moderate or high value for money.

Both Technopolis studies are available here.

These two Technopolis studies are already making a difference, and have informed some of the key decisions on the high-level design of the next research assessment exercise (REF2028), the outline design for which published in July 2023 by JISC on behalf of the four UK higher education funding bodies here.

The team: Anete Vingre, Cristina Rosemberg Montes, Paul Simmonds.

O que há de novo?

Todos os artigos Todas as notícias