On 12 May 2022, the results of the United Kingdom’s Research Excellence Framework 2021 (REF2021) were released. Generally known as ‘the REF’, this framework exists to assess and hold universities to account for the quality of the research they produce (Stockenhammer et al., 2021). The REF results are important as they are used to distribute Quality Research (QR) funding to these universities (Kelly, 2023). Additionally, they are seen as a ‘reputational yardstick’ (Torrance, 2020, p. 772) that can boost or weaken universities’ perceived strengths among fellow academics, students and the wider public.

The REF2021 operated through Units of Assessment that recruited leading colleagues to evaluate research work submitted by universities. For Education, submissions had to be made to Unit of Assessment 23 (UoA23) in the Social Sciences Panel C. Each submission had to include a Research Environment Statement, a selection of staff’s publications and a number of Impact Case Studies, demonstrating how examples of research led to real-world influencing of, for example, policy and practice.

Given the REF comes with high stakes, both reputationally and financially, departments across the country reflect on research strategies that can enhance the quality of research undertaken by its staff members. One central topic in these discussions relates to the generation of income in the form of grants and fellowships to carry out one’s research agenda. Research borrowing data from the previous REF in 2014 by Pinar and Unlu (2020) revealed that in most Units of Assessment, higher levels of departmental research income correlated with higher REF scores. Their analyses, however, were based on overall research income and did not distinguish between different types of funders.

After the publication of REF2021 results, I started a deeper investigation into the relationship between the generation of research income and REF results, especially for the 83 submissions to UoA23 Education (Boeren, 2023). Research income data per submission are publicly available on the REF2021 website under the Environment section of the results and are listed per type of research funder based on categories as used by the Higher Education Statistics Agency (HESA). My methodological approach consisted of a recalculation of research income per Full-Time Equivalent (FTE) of staff submitted per institution. This was done to reflect variation in size of submissions. Average amounts of research income in Sterling per FTE for each unit and for each income stream from UK, Europe, non-European and industry funders were correlated with REF results. Grade Point Averages (GPAs) were used for REF results. The highest possible score for either research environment, outputs or impact is a 4, better known as ‘4* world leading’. GPAs for Education in REF2021 sat in the range 1.47 and 3.60.

A first exploration of the data revealed wide variation in income reported by different universities. In total, 79 per cent of all research income in Education came from UK funders; European grants represented 9 per cent of the funding total. Income from industry funders tended to be low across the submissions. Unsurprisingly, research-intensive institutions – referred to in the UK as the ‘Russell Group’ – captured more funding than newer teaching-intensive institutions. Digging deeper into the data, my analyses showed that the highest-scoring submissions in terms of their GPAs tended to capture more research income from prestigious funders such as the Economic and Social Research Council (ESRC) or the British Academy. Interestingly, several newer teaching-intensive universities with smaller numbers of staff managed to generate good grant capture per FTE but without this necessarily translating into high REF scores. The results revealed that the lowest correlations between the proportion of world-leading outputs and a submission’s reported research income were found for ‘UK government’ and ‘EU government’ funding schemes. The latter category included funding from more practitioner-oriented opportunities such as the European Commission’s Erasmus+ programme.

‘In total, 79 per cent of all research income in Education came from UK funders; European grants represented 9 per cent of the funding total.’

I concluded my analyses by stating that while generating research income is an important part of academic life, scoring high in REF is more complicated than simply following the money that is available to researchers. For example, producing outputs that are not underpinned by original ideas, not carried out through rigorous methodological procedures or that lack significance, are unlikely to be assessed favourably by the REF panel.

This blog post is based on the article ‘Investigating the relationship between research income and research excellence in education: Evidence from the REF2021-UoA23 data’ by Ellen Boeren, published in the British Educational Research Journal.

References

Boeren, E. (2023). Investigating the relationship between research income and research excellence in education: Evidence from the REF2021-UoA23 data. British Educational Research Journal. Advance online publication.

Kelly, A. (2023). The end of the line? A quantitative analysis of two decades of competitive assessment and research funding in UK higher education. Assessment & Evaluation in Higher Education, 48(5), 701–722. https://doi.org/10.1080/02602938.2022.2120961 

Pinar, M., & Unlu, E. (2020). Determinants of quality of research environment: An assessment of the environment submissions in the UK’s research excellence framework in 2014. Research Evaluation, 29(3), 231–244. https://doi.org/10.1093/reseval/rvaa003 

Torrance, H. (2020). The research excellence framework in the United Kingdom: Processes, consequences, and incentives to engage. Qualitative Inquiry, 26(7), 771–779. https://doi.org/10.1177/1077800419878748