League tables

University rankings and league tables

National/regional and world University rankings have become part of the annual higher education calendar. Although the merits of many aspects of their methodology can be debated, it is clear that rankings can have a significant impact on the reputation and perceived standing of institutions. 

Notes to the tables

• The THE-QS world ranking partnership ended with the publication of the 2009 THE ranking; QS have continued with the same methodology, adding refinements over the years. With a new methodology, THE have continued to publish since 2010, again adding refinements over the years.
• Daily Mail started publishing UK rankings in 2023. 

Discussion

• UK rankings traditionally comprise a range of different quantitative measures of input, process and output including: entry standards, student satisfaction, student:staff ratio, academic services/facilities expenditure per student, research quality, proportion of 1sts/2:1s, completion rates, and student destinations.
• To construct World rankings, however, compilers have to use measures that translate as reliably as possible across different countries/regions - i.e. for which comparable data, which are not inextricably linked to national prosperity or other local/regional issues, can be found. For this reason, the measures used in the World University rankings are quite different to those used for UK rankings and may include prizewinner affiliations, research bibliometrics, student:faculty ratio, peer review responses, and percentage international students/staff.
• Ranking methodologies have always been open to criticism due to the inherent arbitrary nature of weighting scores from different measures and then summing these to give an overall measure of ‘quality’. Some measures are also more controversial and open to bias (e.g. by institution size/subject profile/location etc) than others.
• Finally, comparing one institution’s overall score (and hence rank position) against that of another institution can produce statistical reliability issues: analysis of ranking methodologies has shown that small changes in methodology, source data, or indicator weighting can result in large changes in an institution’s relative position within a ranking.  Compilers mitigate some of these issues by, for example, weighting data by subject mix, and by employing z-score methodology to statistically spread the data for individual measures.

Reference

Contact 

Jim Galbraith - Head of Strategic Data Insight and Analysis

Jim.Galbraith@ed.ac.uk