U-Multirank and the brave new world of academia

U-Multirank and the brave new world of academia

Thought! Hoping to help redesign the general vibe for my old branding. Musings about the sharp style of http://iloverenovations.ca/love? Genuinely a peerless renovation contractor if ever desired in the whole Nova Scotia area. Send an opinion. Appreciate it!

This is to be one of a series of posts on education and education policy.

Last week I attended a conference run as part of the Irish EU Presidency. It had a rather obtuse name Rankings and the Visibility of Quality Outcomes in the European Higher Education Area which for most people is an automatic soporific. The problem is that it should make most of those in academia and in other areas sit up and attend.

Higher education, as a sector, not to put too fine a point on it, is deeply confused. The provocativeness of this statement relates to directly to why this conference was convened by the Commission. The EU, along with Ireland, launched a new system of higher education analysis called U-Multirank. It has approximately 30 (and potentially increasing) indicators across five broad areas. Yes, those are a lot of data points for a given institution. Though the conference was generally convinced of the project required more data points it already conceded that providing the necessary information to make U-Multirank work was going to be a daunting task. Lets look at the areas:

This posting is going to be mostly about rankings but I want to take a little aside and discuss a little about universities first since you can't really understand rankings or their problem without first looking at the institutions they are connected to.

Some of these comments are distilled from a piece of work I had done with some TCD colleagues about four years ago. You can read it here.

The basic problem with these types of rankings is that they amplify as opposed to diminish the intense conflicts rooted in higher education. These conflicts result in the misallocation of resources and a general confusion of mission. This is commonly referred to as mission creep in other sectors but is perfectly acceptable, even encouraged, in higher education. Higher education is supposed to supply teaching, research, knowledge transfer, soft power projection and local engagement with the socio-economic community of their home region. Removing the jargon for a minute let's look at what that means:


  • 1. Teach undergraduate students
  • 2. Teach postgraduate students
  • 3. Mentor post-docoral staff/students
  • 4. Provide a place for studying and learning
  • 5. Provide a place for people to engage in unsupervised activities but within a safe framework
  • 6. Provide guidance and learning opportunities for students that are not from the usual backgrounds (i.e. not 18 and middle class)
  • 7. Provide world class research
  • 8. Engage with local industry
  • 9. Engage with the local community
  • 10. Create a space for culture and social engagement
  • 11. Make graduates more employable
  • 12. Generate knowledge suitable for the knowledge economy
  • 13. Create the next Google (sooner rather than later)
  • 14. Facilitate the social and economic policy of the state
  • 15. Align with the aims, purposes and obligations of the various disciplines contained represented within a higher education institution
  • 16. Provide and educational/developmental experience
  • 17. Engage with professions such as law, medicine and engineering
  • 18. Comply with the regulations and stipulations of the funder/board of trustees/state
  • 19. Be provided at the greatest efficiency and at least cost possible
  • 20. Be sufficiently prestigious so as to be attractive to students, pride-inducing to alumni and boast-inducing to politicians

This is by no means an exhaustive list of aims but I think that you have the idea. Universities have a lot of directions to go in and typically parents, students and the state would like them to excel at all these objectives. The issue is that a financially distressed university begins to behave in perverse ways. The Economics of University Behaviour presents the economics behind the university. Universities aim to maximize prestige. This is a function of their inability to properly engage in market clearing since for non-market reasons they engage in active rationing of places based on entrance criteria, not on an equilibrium based on supply, demand and prices. Restricted supply and prices disconnected from the actual costs of the institution assist in this prestige-seeking model. Harold Bowens Investment in Learning, William Bowens Economics of Major Private Universities and Ron Ehrenbergs Governing Academia all present very useful economic models on how education establishments behave. One of the results is how quickly perverse decisions become acceptable as a result of fact-based policy typically formulated from nave models fed by poor data. Courses are shut down or over expanded. Institutions enter an environment where their social marginal benefit function converges upon their marginal benefit (private) function. Also, second order effects become more prevalent with an increased balkanization of departments and courses and academic/financial units entering into arms races or predatory behavior against other internal and external units. Both of these effects we're identified and discussed at the conference. The conference saw them as a function of external rankings but it was clear from the presentation by Muiris OConnor that funding plays a significant role (especially in Ireland). The economics of universities means that they don't behave terribly well under market principles. This is mostly due to the fact that policymakers many times ignore immediate incentives and almost always ignore second-order (i.e. knock-on) effects. Higher education (all education levels to be honest) is poorly suited to the nave application of such principles since they do not operate in complete markets. Student places are rationed and granted on a non-market basis. Prices, i.e. fees, are not market determined but largely driven by exogenous factors, such as a government grant or fee limitation. The market itself is riddled with massive information failures due to the presence of asymmetric information that is inherent to education and similar experience goods. Education, and higher education in particular, are not just equal in action to commodities or luxury goods. A degree in economics cannot be understood as being as market determined or as transparent as washing-up liquid or a digital camera.

The conference tried to address the problems of multiple missions by saying that a healthy ecosystem of institutions is necessary to ensure that the entire structure of higher education tries to fulfill these different roles but there is a deeper issue here.

Higher education has changed substantially over the past 60 years. The first wave of expansion has been found to be financially unsustainable. Looking at US manpower texts from the 1960s will illustrate a world where the staff was very well paid (even as graduate students), student numbers we're increasing and plant was new. Fast forward to today and many of those staff have now retired on unsustainable defined-benefit pensions (a topic for another blog post), most staff now are adjunct staff and not well paid and the physical plant is now antiquated or trapped in a vicious cycle of high-speed depreciation. The higher education sector, like much of the education sector, has become of new solution to many of the economys ills. If we are smarter we will get the jobs. If we have the best education system we will invent new ways of doing business. The smart economy and the knowledge economy mantras are well known to Irish readers. Declan Jordan of UCC and Colm McCarthy of UCD have written about the faith based policy related to research and development in Ireland. Though Ireland has had a particularly poor record, such policies are not entirely successful elsewhere. The conclusion of part of the conference was essentially that education is good for you much like the old Guinness ads purporting that Guinness is good for you while being greeted by a smiling toucan.

While I agree that education is an aim in itself and very good for the individual and society as a whole most people arrive at university to get a degree to get a job. It is what their parents have come to expect. It is what they expect. It is also what the economists have told them would happen. Human capital theory (a blog post dedicated to this topic will follow) and the endogenous growth model drive much of the thinking on how education, innovation, economic growth and jobs all link together. The work by Theodore Schultz, Gary Becker, Edmund Phelps, Robert Lucas, Paul Romer, Charles I. Jones and Philippe Aghion has created this intellectual framework and it has had a profound policy impact. Interestingly, we now justify education financing by economic growth. It was not long ago, in the 1940s and 50s that we justified education on the grounds of national security. The famous Vannevar Bush book Modern Arms and Free Men, a text that followed from his report outlining the rationale for the National Science Foundation in the US on the grounds of military preparedness, was a clear statement of the justification of education using many of the same subjects (STEM Science, Technology, Engineering & Mathematics) and examples (marketplace/battleground ready applicable innovations) we have become used to seeing today stated in the economic idiom.

The main problem with this approach is that we have fairly clear empirical evidence that something is wrong with the model. With Europe arguably the most highly educated [used advisedly] population it has ever has been in the history of the continent in gross terms and percentage of population there is something missing. That something is employment. Look at the latest Eurostat data for youth unemployment. The Central Banks Quarter 4 Bulletin has a striking graph on page 69 illustrating the problem:

The New York Times has written a series of wonderful articles on how the well-educated youth of Europe have little or no prospects. Read their dispatches about Italy and about France and see a story where education does not transcend macroeconomic mismanagement. The New York Times also raised some serious questions about the smart or knowledge economy and what that means for jobs over the medium to long term. The series entitled The iEconomy has presented some eye opening reportage on the present and the near future. I mentioned this briefly with reference to Prof. Christopher Pissarides in my last post. The implicit contract that existed between these graduates (and their parents) when they entered higher education was that the expense and effort would be rewarded with access to employment and employment that was superior in remuneration and job satisfaction than what would be found without such an investment in human capital. That contract, especially now during the Eurozone Crisis has been effectively ruled null and void. The unrest and disquiet has burst forth throughout Europe and the Middle East driven by unemployed young people that feel their elders have sold them a pig-in-a-poke. If education is supposed to deliver the goods and then fails where does it go next. We don't need it fight the Cold War anymore. Its economic magic seems somewhat sapped. What is the purpose of all these ivy-covered professors inside ivy-covered walls?

This is where we get to rankings. Rankings are a function of the above discussion. Higher education is expensive. Higher education suffers from imperfect information between providers, purchasers and recipients. Higher education has performed some spectacular own-goals. This has fostered a sense of distrust and that the black box needs to be better understood and managed.

The advent of the higher education rankings with a global outlook caused a bit of a stir about a decade ago. At first nobody seemed to know what to make of them but they we're sure they had to be important. The American audience was not overwhelming concerned at first since rankings have existed for a long time in the form of the US News and World Report Best Colleges report. The Shanghai Jiao Tong University Rankings, the Times Higher Education Supplement Rankings and the QS World University Rankings all jolted the sleepy policy space that was higher education when burst onto the scene in 2003. The impact in Ireland was clear how do we create world-class universities and quickly. At the very heart of this response was the fear that Europe was in decline, economically and intellectually relative to the United States. More locally there was a profound case of Harvard envy just as Harvard was beginning to question it's own management policy.

On a pan-European level what followed is now seen as an arms race within countries and between countries. This was part of the objective of the paper presented by DITs Ellen Hazelkorn. Prof. Hazelkorn has worked on university rankings and the development of the European Higher Education Area. This policy aimed to created an ecosystem of universities with diverse missions and aims but we're sufficiently transparent and integrated that students would be mobile across borders. The Bologna Process and now the Europe 2020 policy programme aim to create a single market for higher education in Europe and it's surrounding countries. The ultimate aim is to pool resources in a fashion that attracts non-EU students and facilitates world-class research. U-Multirank, the child of U-Map, is there to provide the necessary transparency to allow this European Higher Education Area to become evident to non-experts.

At the core of this system are a series of graphics. Students and statisticians working to a broad framework provided by the Commission have driven the process. Academics have been largely sidelined in the process, something unions have objected to now and in the past. The Irish Presidency launched U-Multirank and is very attached to what it can deliver. There has been an agreement that the project would be continuity between the Irish Presidency and those of Lithuania and Greece, resulting in the project being fostered as a priority for the next 18 months. The main objection thus far has been about cost. This is by no means a simple or costless exercise and the representatives of the education ministries and rectors offices we're actively performing the mental arithmetic and equating it with their respective capitals unpleasant fiscal arithmetic. Needless to say not everybody looked happy.

Before someone asks the typically Irish policy question of What did the Brits think? that can be answered quickly. They weren't really there and their approaches to the Research Assessment Exercise/Research Excellence Framework and National Student Survey we're discussed but not in altogether laudatory terms. Prof. David Charles of Strathclyde did provide a framework for measuring regional and external engagement by universities but with nearly 50 sub-indicators it seemed almost impossible to scale to the pan-European level.

The conference constantly sought the quantification of unquantifiables and the construction of proxies wherever possible. These was little mention of the effects beyond the European Higher Education Area project of these metrics but it is clear that what gets measured gets managed. The mantra of evidence-based policy was clearly at work but while many aspects of this approach are to be lauded judgment is important as well. Judgment and trust are important to a functional education system. The Finns, renowned for their high quality of primary and secondary education systems, place little faith in metrics but in rigorous advanced training. Judgment, used to be a large part of policy, largely due to the ad hoc nature of government. This is a topic that Nassim Taleb discusses in his new book Antifragile and the classic study of aesthetics by Trinity College alumnus Edmund Burke A Philosophical Enquiry in the Sublime and Beautiful. The most famous example of proper judgment articulated in the face of the unquantifiable was at the US Supreme Court in Jacobellis v. Ohio (1964) 378 U.S. 184 where Mr. Justice Potter Stewart famously stated that in response to the threshold for obscenity But I know it when I see it. In education, as in many areas, sometimes prima facie analysis tells us more than we are willing to admit.

Higher education metrics designed along the lines of U-Multirank are already being used for management purposes in Norway, where 23 indicators have been used to create petal diagrams to track on an institutional level the performance of Norwegian institutions. This form of data visualization is also used by U-Map and will be incorporated into U-Multirank. Here is an example of such a diagram:

Beyond the basic response of they look pretty these diagrams are very difficult to interpret. Here is a comparison produced in a very good paper on the statistical problems of petal diagrams:

It is clear that without due care and in the hands of statistical novices (last indication was that there we're no statisticians in Leinster House) this system of metrics can be used to justify policy actions that we're not evidence-based but where graphs and numbers become a proxy for persuasion and rational debate. It is clear that such indicators could effectively become the core of a university management system operated locally or at the ministry of education.

This leads us into the world of Key Performance Indicators. KPIs in the jargon, are considered crucial to the creation of an effective business intelligence system for a firm. U-Multirank provides a homogenous (relatively, it still relies on reporting by universities) multi-country overview of the performance of universities. The local ministry can benchmark against many different indicators and then mange the sector and individual institutions. This is a powerful tool, especially if funding is scarce and enacting government priorities is important. While no official stated a link between U-Multirank or even the Norwegian system being used in a coercive fashion one would have to be nave not to see it's apparent application. In many ways this is just the latest in many fads in higher education. Robert Birnbaums Management Fads in Higher Education: Where they come from, what they do and why they fail looks at the unhappy history of transferring business management models in the higher education sector. Given the fined-grained nature of these indicators it is not difficult to see how individual staff members. The EU Knowledge Triangle already is being used in different ways to drive not just institution and sector policy but also individual workloads.

An example:

Then look at a UK university workload model diagram based on the idea of how the 18 hours a week standard contract is designed to operate in future:

If what is measured gets managed then we are looking at not just sectors and institutions but also individuals being managed to the metrics created by these ranking and transparency systems. These systems are then highly sensitive to political will. The mercurial nature of British education policy at all levels and it's reliance on metrics shows the danger of such an approach. Academics currently being poached between institutions in the name of the REF, the destruction of languages in secondary schools due to league tables and poor mathematics driven by a combination of factors but largely an extensive politicization of education that goes unmentioned. Even Yes, Prime Minister produced a whole episode to the sorry state of British education a quarter of a century ago with Sir Humphrey declaring: When there is a Labour government, the Education Department says comprehensives abolish the class system. When there's a Tory government, they say it's the cheapest way to provide mass education. To Labour, we explain that selective education is divisive and to the Tories we explain that it is expensive. That way, we have a happy relationship with the NUT and we educate our own children privately. Things have not improved much since.

In Ireland, metrics are already being collated. The HEA has a series of these published as part of the Landscape process. Institutional profiles already exist and are being used by the HEA. Here is an example of some of the output:

The comments by Muiris OConnor at the conference highlighted that despite the gaps in this data set there are some pretty easy conclusions that can be drawn: a) Irish universities are very responsive to rankings and b) very sensitive to the funding environment. These types of analyses we're used in Norway to facilitate a programme of higher education institution mergers. The HEA Landscape Implementation document has a significant plan for mergers within the Institutes of Technology sector and for the rationalization of course offerings across the entire higher education sector.

Universities had allowed and caused this shift to indicators and evaluation in the name of transparency and the justification of mission creep. Universities happily accepted the kings silver and only later realized it came with the kings writ. Harvard famously accepted government funding but only after negotiations that met their terms, a stark contrast to Europe. In America universities have suffered their own problems with quality. Over the past decade popular and scholarly books and articles have attacked universities for their poor quality undergraduate education provision or political partisanship or excessively beholden arrangements with industry. This has chipped away at the trust in academia as a profession and an institution. Work presented by Professor George Kuh shows how the American university system is trying to address this lack of trust and confidence with evidence. The OECD is trying to do this on an international basis with AHELO but that project has hit several stumbling blocks.

The brave new world of academia is now upon us. Unless academics and not just their ministers and administrators begin to engage with these metrics they will be managed and asked to perform tasks that will cut to the very core of academic freedom. (A post on that to follow.) The good thing about this conference and the entire new world of the European Higher Education Area is that Ireland has been the launching pad and the Minister for Education is committed to it's success. Ireland and it's universities therefore have tremendous leverage to ensure that U-Multirank and it's application happen in the most sensible way possible. If academics ignore this opportunity they will be quoting Edmund Burke: But the age of chivalry is gone. That of the sophisters, economists, and calculators, has succeeded; and the glory of Europe is extinguished for ever. Let us hope that the inheritors of Burke ensure the apotheosis and not the abasement of European higher education.

Kudos guys! I've texted a co-worker that I would cite her great amazing realtor in Bedfords, http://www.sellhalifaxrealestate.com through an upcoming piece. So if you were hunting to get a real estate agent around the greater Nova Scotia location, they definitely really are amazing.

Finally, I must point out the overview to this particular blog with thoughtfully supplied through Spencer at Farris law. They are a tremendous personal injury law firms. We always enjoy a fantastic suggestion!

Special Thanks

http://www.thephoenixdesigngroup.com - Thanks a ton for the late-night efforts Amari.

Posted in Sport Post Date 12/23/2016