International Efforts to Improve Ranking Systems

As national and international ranking systems have grown in number, each with distinct methodologies and very different goals, the need for sharing of ideas and information among those engaged in this work has also grown. To meet this need, ranking systems researchers, institutional representatives, and the rankers themselves have come together in recent years to learn from each other and develop shared understandings of the purposes, methodologies, and promise of ranking systems.

International Rankings Experts Group

The first international meeting focused on rankings, an invitational round table on "Statistical Indicators for Quality Assessment of Higher/Tertiary Education Institutions - Ranking and League Tables Methodologies" was held in 2002 in Warsaw, Poland, and sponsored by the UNESCO European Centre for Higher Education (UNESCO-CEPES, headquartered in Bucharest, Romania). Through the meeting, participants from 12 countries noted the continued need to improve the conceptual frameworks, methodologies, and organizational aspects of college rankings.

A follow-up meeting, held in December 2004 in Washington, D.C., was hosted jointly by UNESCO-CEPES and IHEP. The meeting led to the creation of the International Rankings Expert Group (IREG), composed largely of the participants in the Washington meeting made up of leading experts and from around the world who either conduct rankings or analyze those ranking systems.

IREG held its second formal meeting on rankings in Berlin, Germany, in May 2006. That meeting was organized by the Centre for Higher Education Development (Centrum für Hochschulentwicklung) in Germany, UNESCO-CEPES, and IHEP, where participants established the Berlin Principles on Ranking of Higher Education Institutions. The third meeting of IREG was held October 2007, in Shanghai, China and was organized by Shanghai Jiao Tong University’s Institute for Education, UNESCO-CEPES and IHEP.

Berlin Principles on Ranking of Higher Education Institutions

A key outcome of IREG’s 2006 meeting was the creation of the Berlin Principles on Ranking of Higher Education Institutions, whose purpose is to promote a system of continuous improvement and refinement of the methodologies used to conduct these rankings. The Berlin Principles focus on good practice that will be useful for the improvement and evaluation of ranking systems over time. The principles emphasize the purposes and goals of rankings, the design and weighting of indicators, the collection and processing of data, and the presentation of ranking results.

The 16 principles articulate several important standards of good practice, including recommending that rankings should:

Recognize the diversity of institutions and take the different missions and goals of institutions into account;
Be transparent regarding the methodology used for creating the rankings;
Measure outcomes in preference to inputs whenever possible;
Use audited and verifiable data whenever possible;
Provide consumers with a clear understanding of all of the factors used to develop a ranking, and offer them a choice in how rankings are displayed.

The 16 principles have been discussed at leading higher education meetings around the world in the last year and have figured prominently in the dialogue about accountability in higher education in several countries.

Other Key International Rankings Meetings

Other international rankings meetings of note include two rankings symposiums hosted by Leiden University in the Netherlands: The Challenges of University Ranking: How can we identify the best universities in the world, held February 2006 and the 2nd Leiden University International Symposium on Ranking, held February 2007.