The Problem with Ranking Universities (League Tables) - Sixty Symbols
Vložit
- čas přidán 11. 09. 2024
- We discuss the problem with ranking universities using so-called "league tables". More links and info below ↓ ↓ ↓
In this video we hear from Professor Philip Moriarty, Meghan Gray and Mike Merrifield... All are from the University of Nottingham and are expressing their personal views on league tables.
This video features scientists from The University of Nottingham School of Physics and Astronomy
bit.ly/NottsPhy...
Also in this little collection of interviews...
Failure: • It's okay to FAIL exam...
Clearing: • The Clearing System (l...
Some Extra Clips that didn't make it but are interesting: • Extra Bit - Sixty Symbols
Catch also this video from some years back with Mike Merrifield about applying to university: • Secrets of Applying fo...
Phil's blog post about league tables: muircheartblog...
Mike Merrifield blog post on the same topic: blogs.nottingh...
In the interests of fairness, here are some popular league table websites...
The Guardian: www.theguardia...
Times Higher Education: www.timeshighe...
Complete University Guide: www.thecomplet...
umultirank (as mentioned by Mike n the video): www.umultirank...
Brady's blog post about these videos: www.bradyharan...
Patreon: / sixtysymbols
We're on Facebook at / sixtysymbols
And Twitter at / sixtysymbols
Sixty Symbols videos by Brady Haran
www.bradyharanb...
Email list: eepurl.com/YdjL9
The "nobody smiled"" argument is SO important and I have a feeling not a single league table or equivalent ranking system takes it into account. If you're gonna be spending several years in a learning institution, it doesn't matter how prestigious or well-equipped it is if you're having a miserable time with miserable people all the way through! Obviously there's personal opinion in this, but I'd take a #12 university where most people are enjoying the daily grind and the atmosphere is welcoming and inclusive over a #1-3 university where every day is tense and stressful a million times out of a million. 😸
What about the million-and-oneth time?
If you are judging people whether they smile or not, that's also a problem. Plenty of people who are not extroverted and overly friendly would give you their coat off their back if you needed it, and other people will smile and be nice to your face, but stab you in the back for a buck.
Congrats to Professor Merrifield on retirement!
Thank you!
Emeritus discussion ~17:00 or so!
@@AstroMikeMerri I presume that "retirement" means "getting to spend more time looking at interesting bits of science that I like, and less time writing grant applications and attending committee meetings"?!
That makes me kinda sad, I really enjoyed the "pandemic" episodes with him discussing astrophysics.
I have deeply mixed feelings on hearing Professor Merrifield's retirement as I have followed his posts for some years I found his manner, along with his knowledge, a treat.
Best wishes, Professor. Hopefully we haven't seen the last of you...
Re: the sports table analogy. It’s like the premier league wasn’t just decided on win/draw/loss points and goal difference, but it also included, all equally ranked with the points: a style of play rating; a jersey design rating; a sponsorship dollar index; best signing of the season rating; a fan song contest; and a stadium comfort ranking.
I think that’s kind of the answer I was fishing for. ;)
@@sixtysymbols :-)
And then millions of people freak out and spent huge amounts of money or go into debt to buy the athletes' jerseys based on who's ranked the best, governments use those rankings in part to decide which athletes get better food and training areas, and hiring managers will check to make sure that you supported high ranking athletes when you apply for jobs.
Yep, and on top of that, different leagues cherry-pick different measures (potentially depending on who they want to win!).
Immediately sent it to my teenage son. He is so fixated on those tables. I keep trying to tell him that there is no way he can correctly decide what’s good for him based on some website rankings.
Well done.... this ranking business is really invidious.... At the best, broadly we can say that something in the top 20s is better than something ranking between 100 and 150. Atleast for undergraduate... for post-graduate, maybe not even that.
If he's fixated on these tables, then tell him to work hard enough to go to Oxbridge. If he can't, then all other universities would matter much less than what effort he's able to put into studying.
@@R_V_ Actually this is a very good point. One way I put it is that: half of a young person's development depends on the institution's quality and the other half on the calibre and effort the young person exerts (Also subject some indeterminate lower threshold of both parties). Of course the three or four years don't have to be nose to the grindstone always, good to also have some flexible space to overall development.
it is often important to know details and exceptions before trustinh broad statistics.
ok?
In a race, every news source will report the same results: who won, the final score, elapsed time, etc. Results are objective, at least within the framework of the sport's rules. League tables are obviously subjective when the tables from the various sources don't agree with one another in the least.
Additionally, in sports, trends tend to be the norm. The are plenty of exceptions, of course, but good teams or good athletes tend to stay good over a period of time. No athlete who has never been able to run 100 meters in less than 12 seconds is going to suddenly one day beat Usain Bolt in a race. Similarly, how can a University or an academic program be ranked number four one year, twenty nine the next, fifteen after that, and so on? It's not reasonable. Nor objective.
Lastly, how can two institutions or academic programs be ranked, when one of them is research-oriented while the other is for future educators, for example?
The fact that there are so many league table publishers and their rankings differ so wildly tells you how subjective (and pointless) these numbers are.
We really need a ranking system to decide which league table is best.
@@tbird81We should get started on that immediately!
@@tbird81We should have several independently run league table rankings.
And then we could...
@@tbird81 We need league table for league tables
@@auroravuitton90 Didn't I already say that? But in grown up talk you possibly don't understand?
Seriously; speaking 'eloquently*' on CZcams and the like seems pointless now.
I'll probably continue to do so, as it's part of who I am.
On the other hand, WGAF. 🤷♂️
* coherently. With real sentences an ting.
It’s hard because there’s more than one objective. If I had a university where it’s poor at job placement for undergrads but the grad students find a cure for a different cancer each year and another university has bad research but 100% job placement, which is better? Giving an equal score to both isn’t helpful
Absolutely correct. From personal experience: I studied chemistry at a university with a small chemistry faculty that is not well known (scored place 50 in some national ranking for chemistry I just pulled up) and now work as a post graduate researcher at a famous university (scored place 4 in the same ranking). And I must say that the education we got was an order of magnitude better at the smaller faculty. I know, I sound very biased because that's where I studied, but I have seen it from both sides. The level we were at when conducting our lab courses directly before the bachelor's and the level the students were at when I supervised the lab course at the other place. It was a massive difference.
My hypothesis is that the small faculty is not full of famous professors who just want to do their research and don't care as much about the education. The complete system was organized and structured far better going as far as even taking the feedback of us students of what could be better and actually implementing it.
In a nutshell: Forget rankings. If you have the chance, look at the place and see if you get the impression that people care about your education.
I've been ranting about the nonsense of league tables since I first applied to university. You only have to read a few before you notice the lack of logic to them. What's far more important to students and wider society is reputation. If your uni has a good rep then people know about it, particularly with certain subjects. If you say you studied medicine at Nottingham or classics at Oxford, or art at The Slade School, or biology at Imperial; people understand that's impressive. Observing talented students choosing universities, for them it's about the city or campus, how close they are to home, and just the vibes from their visits. So I wouldn't worry about it. I'd worry more about getting the vibes right.
i thought they were saying “lead” tables the whole time. makes way more sense now 😂
ok?
This is really potent the way it's edited. Can't wait for the discourse when it's released in full!
Brady does a top notch job playing devil's advocate. Great way to handle interviewing about it. That being said, as someone who works in higher ed, university rankings publications are BS and people internally all know it. We play the game for public perception and nothing else.
John stossel is the best at it, bradys arguments are quite bad and negging
As Prof. Merrifield alludes to at 11:30 and 17:40, the fundamental quantitative flaw is they collapse multiple value dimensions into a single metric - that’s not just difficult, it’s inherently impossible without an objective conversion factor among the input variables. “We looked at new cars’ acceleration and petrol-efficiency and ranked the best cars as follows…” not doable unless you know the *subjective* value of a unit of acceleration per liter/kilometer of fuel. And that’s subjective to the consumer, just as students are subjective consumers who value different inputs to the rankings differently. If you had interviewed the Nottingham economics department they’d probably have ranted about this flaw more than about error bars.
Adding to this, universities sometimes optimize for the metrics of the league tables and ignore aspects that are important but not considered.
One big flaw with Brady's Olympics analogy, even with more subjective events like gymnastics, is that very few people, if any, make important life decisions based on who won the gold medal. Damn straight I'd want error bars on the gymnastics scores if people were using those results to decide something like what university to attend.
I just wanted to say I really appreciate Brady's critical points around 8:50. Really the best interviewer I have ever seen.
People just want to know what's the best to be with rather than scientifically what happened from behind to the foreground, and who's won and that's all.
I'm really glad he brought that point up. Made the discussion quite interesting
A bit of push back was great
I appreciate that you always try to get a bit of a rise out of professor Moriarty, and that he always gives you one. The man is passionate!
I've recently been grumbling quite vigorously about job interviews and recruitment processes recently. And many of my grumbles have incredible resonance with the many of the issues raised here. Phil's comments about peer-review and grant proposals and actually engaging with the material really struck a cord.
Also when you're asking Mike if he could come up with an alternative league table is similar to a Veritasium video a while back, where Derek was talking about expertise and how certain environments just aren't "valid enough" to ever allow a real expert to exist.
A student chooses a school based on these rankings. In that student’s second year at the selected university, the school’s ranking drops 15 positions. What to do now.
Fair point but there are some schools the thing you describe just won't happen.
Transfer. People do this all the time. Start at a lower-ranked university that will accept them, transfer to a higher ranked university once they start acing classes and prove their capabilities.
Always thought that the fact that at primary and secondary school level the worse a school performs the less resources they get is ass backwards.
Surely you should want the best teachers at the worst schools? And that costs money.
Never realised they were such a problem at higher education level.
when a metric becomes a target is stops being a useful metric.
There's two things I dislike about these kinds of lists:
The schools with the highest rating would get the "best" students, so I would assume they would scale higher for things like average grades.
You're looking at the schools rated at the highest when there are like 4000 higher ed schools (in America). So people are comparing the, like, 0.25%of the schools people actually go to.
Don't worry too much, often the best students are not really the best, but ones who simply score well in exams.... something to that, but it isn't everything.
Brady, go do the 100 m sprints "league table" for the top 25 sprinters this year for the last 5 years, and see how it varies, take the best times they do during 2 or 3-month periods during this time. I dare you. You will not see the same type of variance-to-trend curves.
I think this is part of a more general issue that people need to think about for themselves: any given set of data has no meaning until you yourself impose a meaning on it. Maybe to you the league tables are valuable, but to another, maybe not
I really liked using sports for an analogy. What I would say to Brady though, (besides thanks for the interesting points) is that the League Table isn’t about which team had the most goals in a season, which is a well defined metric that can be studied. But doesn’t necessarily mean they’re the best team. Maybe it should be the team that won the most games? Should it be the team that won the championship game despite being 3rd in goals and 2nd in wins? So it’s not about who won a race or scored the most in a season, which is objectively measurable. But how athletes or teams should be ranked by some subjective combination of metrics (which is what these media orgs are doing with the universities).
Not too sure about this. Many selective companies choose graduates from roughly the top 10 consistently high ranked universities far more often, and these students tend to have been from more selective schooling or private schooling, this is what originates a lot of the class divide in the UK.
Another thing that's not taken into account here is location. I chose my school not just because of the programs and scholarships, but because it was 5 minutes from my house. The best school in human history by every metric wouldn't have detracted me from staying home.
Great video with great challenges from Brady. While I completely recognise the flaws in these rankings, I take issue with 'don't base your decision on the league tables at all'. Rightly or wrongly, the quality of your university matters to many employers (particularly for your 1st job out of university). You could ignore the league tables entirely and study at a low ranked university on issues that matter to you but some employers will bin your application as a result.
Change "University rankings" to "company stock" and you perfectly sum up Wall-street. 14:13 --> "What did we do differently, to move from 18th to 5th?"...
It is like IQ. A single number evaluates intelligence, a quantity that is really ill defined to begin. 😋
Interesing discussion on league tables, but what really caught my eye was the Marillion t-shirt on prof. Moriarty 😁
Ohhh, well spotted. It's difficult to tell, but, although it's the Marillion jester, the t-shirt is merchandise from a wonderful Fish-era tribute band called Cillirion. Well worth checking them out. They even do Grendel....
Those first four Marillion albums -- classics, every one.
Philip
Excellent video! Especially the bit about why the numbers change, because how the ranking is caluclated changes from year to year.
Fantastic discussion, way to challenge the profs! I think it boils down to:
1. Are the rankings valuable for future employment despite being ambiguous and volatile? Yes. Subjective perceptions and reputation will always matter for jobs and academic applications.
2. Are they a good way for a student to find a place to do the science they love? Absolutely not. A superbly ranked school may be missing many groups and facilities for the sub-field or topic the student is actually interested in.
For me it is mostly the fact that at more highly thought of universities they tend to be far more competitive and full of more dedicated and talented people because the admissions processes are so difficult. As someone who wants to go into academia I view it as paramount that I can interact with the best people possible. However I completely agree that the variations in places is nonsense - largely most universities have students of similar aptitude it is just that Oxbridge and maybe a few others are made more valuable in a completely circular way in that the main reason they are valuable is because many of the best people go there and many of the best people go there because many of the best people go there. Hence I think that the opinion of people on a universities reputation doesn’t matter in theory it will mean that the year group is more likely to be more competitive so if that is what an individual wants then there is a circular value to the reputation of unis. But yeah league tables are bollocks.
When I started university mine was rated 8th and when I left it was at 24th. I like to think I had some impact on that during my time there.
Two things can be true at the same time: the quantitative approaches used by ranking organizations should be more scientifically rigorous and transparent, and quantitative data should be a part of the decision-making process. Most of the arguments presented highlight that the rankings are not ideal, not that they should be removed. This becomes evident when the professors are asked to provide alternatives. They either came up empty or give extremely subjective measures, such as: "Everyone was smiling". The feeling of welcomeness is undoubtedly important, but giving such an emotional and subjective response after criticizing the quantitative approach for its lack of rigor leaves a bitter aftertaste.
It's almost as if they are trying to argue that all choices are equally 'correct.' While no one can predict the future, that doesn't mean you shouldn't hedge your bets on the most statistically sound choice, especially when the differences are significant. Given the choice between Harvard and a local community college, the former will almost certainly provide a better all-around experience.
The discussion on bias also felt off, despite some valid points being raised. It’s as if the professors, who are from the University of Nottingham (which is indisputably not even in the top 5 in the country), have nothing to gain from the removal of rankings. In fact, I would argue that newspapers, whose rank publications barely affect their year-over-year revenue, have less vested interest than the professors do.
as someone who applied to do physics at nottingham, they ABSOLUTELY flaunted their university rankings repeatedly.
The most important thing I learned in college was that you don't need anyone but yourself to learn things.
Really? Isn't the point of language, and education which I really do think are correspondent in many ways, the relations we build and strength with the world and one another, one more rigid than me might say 'necessary' to life, and of course to the process of learning to stride along it while in it.
@@enimol1 Perhaps it depends upon your field of study. It certainly was not my experience.
It's better to look at ratings and reviews of professors to determine which school to go to.
I've been saying something similar for years - when universities jump tons of places in a single year, that indicates all the close ranks were in a margin of error. I think they can be useful if taken in a very general sense - position 5 is almost certainly better than position 50, and that is almost certainly better than position 150.
Unfortunately I think the league table number does in itself matter, as an employer may consult one when trying to evaluate an application.
I got a BSc in Physics from my local university, and I'm a very grateful 50 year old, for the hard work I did as a young adult. It's hard to quantify the value of an education. I can do more as a hobbyist now than most people can do in a profession.
"You're physicists"
Me, a theater kid working in video games "Yes! I'm with you!"
The Universities all try to game the system, using inventive interpretations of the statistics that need to be submitted. They have teams whose job it is to influence things like cross institution reputation in order to boost their rankings. Not to mention the corruption of the system through premium packages and advertising deals the Universities can pay to the ranking bodies to feature them.
Most of the 'popular' rankings require Universities to submit data to be ranked, so it is well within Universities ability to agree to opt out. Some in fact do, but most are so competitive at the executive level that they refuse to forego the chance of appearing to be better than their competitors.
Everybody knows the only university ranking that matter is coolness of their mascot.
Congratulations to Prof Merrifield. I've really enjoyed your videos the whole time you've been doing them with Brady.
I guess only north American universities count then.
@@SillieWous It's not our fault the rest of the world doesn't understand marketing. ;)
@@CheddarKungPao It is your fault that NA actually believes marketing makes the product better. Yet, interestingly, you also came up with the phrase "polishing a turd".
@@CheddarKungPao ha ha.... well put.
There is some value in providing these rankings to prospective students, and some methodology to rank universities is better than no methodology. Treat the numbers with a huge grain of salt, but if the methodology is scrutable and transparent, they have _some_value. I'm also in a unique position of having transferred mid-degree from a lower ranked to a higher ranked institution. From my own experience, the ranking clearly reflects the quality of the education I am receiving, the assessment is more rigorous, the teaching is of higher quality and the resources available to me are far more comprehensive and the employability of graduates is much higher.
I think because almost no one ever transfers, people rarely get the kind of perspective you have in your comment. They look back on their studies with rose-tinted glasses and say "well I wouldn't have had it any other way, how can these tables say otherwise?"
I did my undergrad at a mid-ranked uni (Reading), and my masters at a high ranked uni (UCL). I felt the quality of education/organisation was far superior at Reading. At Reading they seemed to put more effort in, while I always felt that at UCL they knew they could rely on their prestige alone.
The story about the smiles: "I can be happy here".
Extending the time you need to finish your PhD (especially when you have a time-limited scholarship) is not an easy decision, and sadly is a rather common one to make. Having a place where you feel you can keep on working (on your own, like many PhDs), is very important in my opinion. Especially for not falling into the mentality of "let's just publish anything".
Personally, I continued to a PhD after a master's, because I felt safe in my laboratory. I knew I could continue my research and at the same time, continue my personal life. After you make your research about "graduate student life", you come to know that it can be a pretty scary place, pretty easily.
Brady, the race analogy totally erroneous. The university rankings is like ranking the fastest sprinter by the average of their seat placement on the airplane there , their training times when they were 12, how good they look and the apparent placings but judges by people watching 144p black and white video feed from 400 yards.
Somethings are clearly under defined, some are irrelevant to "good sprinting times" and some are just guesstimates.
What about gymnastics or diving which I switched to pretty quickly?
8:21 when “precise” and so other termas are thrown around you know its a good conversation
I definitely agree with Prof Moriarty on the sports analogy and don't understand Brady's hangup. A race produces one number: time. The error bars surrounding each racer hearing the starting gun, each millionth of a second on the clock, each frame of the slow-mo cam at the finish line, adding up to the error in a racer's time are absolutely *miniscule* compared to the difference between each racer crossing the finish line. Therefore, if a racer has a lower time than anyone else in the race, they won. ZERO interpretation required. These tables, by contrast, may *produce* one number at the end, like a race, but how that number is calculated is completely arbitrary. What metrics did they use, what did they leave out, and why? How did they average them, what weights did each metric have and how was that weight determined? The objective of a race is to have the lowest time. I think everyone can agree on that. The objective of a university is... what? Student satisfaction, research developments, cost efficiency, job outlook? Every single person would value these things differently, so the average is meaningless to everyone but the person choosing the weights.
Hmm. I’m not entirely sure you fully appreciate my role in these interviews. :)
It’s to get the people to say things like what you just said. :)
PHILIP IS WEARING A MARILLION T-SHIRT!!!!!! ❤
The exact rankings are obviously nonsense, but I believe that the vast majority of prospective uni students don't make their decision based on if a uni is 2 places above another.. What ppl use these tables for is to get a general idea abt the quality of the schools they are considering. For example if there is a good school in a different country and there is another school closer to home they might check how big the difference actually is. And if it's like 15-20 places between them it's not really a big deal, but if the closer uni is like 100 places behind the better one, then it makes more of a case for moving. And my point is that for these rough estimates these rankings ARE actually useful. Without them there would be nothing to base these decisions on.
Quite the lineup. You went all out on this one.
A main issue is that these rankings often include student opinions. In case they do there comes a huge interest conflict into play: When the students rate the own University higher (so their own education), then they have an advantage in that (e.g. coming from a better university might lead to better employment opportunities). As such the grading is worthless unless people can guarantee that they are not biased.
(And in the good old days the students regularly boycotted the questionnaires for these rankings, because the outcome was misused, while the universities begged the students to fill them out to not get down ranked)
Great questions. Thanks for pushing back. Loved hearing different perspectives.
My biggest fear for universities is them increasingly becoming vocational institutions. Far more students are concerned with job prospects than academia nowadays... I know we technically got rid of polytechnics but it does feel more and more like we are getting rid of universities.
This video is great, why is it unlisted?
I’m surprised no one mentioned types of validity in research - namely, construct validity: does a test/measurement actually measure the concept it was intended to measure? It doesn’t matter whether the measurements are “objective” - they can be as objective as they like, but it doesn’t mean that they are measuring the “goodness” of universities. That is not something that can be measured directly and these ranking don’t seem to provide much justification for their metrics or how combining them creates a good proxy for “university goodness”.
Thank you Brady et al. Hopefully this video is helpful to programs under pressure to compete.
Time to crossing the finish line is directly observable, and being the best athlete is not directly observable. If a ranking is to help us learn anything other than the ranking itself, then we must acknowledge all the subtleties the professors are telling us
I don't know what we can realistically do about it, but it's a bit worrying how much decision-making is expected of 16, 17 and 18 year olds at a point in their lives when they're only just starting to enter the adult world. I remember being 18 and in retrospect I had such a warped view of the world and my place in it, though at the time it felt like I had everything figured out.
The thing is in league tables in sports you tend to measure a competition over a single definable metric „points scored“ where you are really trying to estimate the playing strength of each team or player. The issue with university league tables is that there is no single definite playing strength you are trying to estimate. And really these tables should be under heavier scrutiny because this is a self reinforcing system the metrics chosen will be optimized.
One of the biggest problems with these league tables, that I felt like Prof. Merrifield was nearly going to state, is that even though you can sort some of the rankings into subrankings (how good the Physics or Chemistry or Philosophy, etc. departments are) almost none ever sort them more fine grained into subfields. Some universities might have the best physics department ever, but if you're interested in Planetary Science and not Quantum studies or Galactic Astronomy, you will think that department is useless or the worst ever.
hm then instead of a single number describing "How good" a given university is, maybe we should consider a whole tensor which tells you how good a university is!
From the student's point of view, maybe all that's needed is a list of universities/ faculties whose degrees are a waste of time? If you know what to avoid, you'll be OK in the long run. Maybe you make an imperfect first degree choice, but if you go on into the higher degrees where the the academic credentials of an institution matter, you probably already have your own idea of where's a good place to study. And maybe most of the time that's simply under the right individual supervisor/teacher?
As an undergraduate, if the course structure or presentation isn't good enough (I think they're generally pretty standardized?) you can fill in the gaps with visits to the library to try a different text book, for instance. You can keep that up for a lifetime, and make yourself equivalently basic-educated to anyone else, in principle.
And if you're going to be a graduate student, you probably already don't need things like teaching that's so good it's almost like they hold your hand all the way? You're going to have to do without that when you get to your ultimate academic goal, so a very hands off university with courses that lack much has to be pretty bad before it's not fit for purpose.
Or do you have to be a Cambridge graduate to get into a Cambridge PhD, in practical terms, in the real world?
Why should you bother looking into the rankings so deeply from a student's perspective? The main purpose of universities in society is to have an impact on the economy. If the employer is using these rankings, then surely the students should as well.
I don’t think the main purpose of a uni is to have an impact on the economy. Or if it is, it’s shouldn’t be. I think it’s a nice benefit or side-effect, but shouldn’t be the main purpose.
The problem I have with the sports analogy is that when you are choosing a university that is a choice you are making for the long term if i pick university A because it is the "best" is year and then it drops by 10 rankings the next year I mean shouldn’t it matter how the preformed in the past? If there is so volatility in rankings not just across one newspaper but all of them how am i supposed to make any long term decisions.
My chemistry teacher always asked one question when presented with numbers:
What's the unit of these numbers? No unit, no meaning.
fine structure constant = 1/137
Unitless, therefore meaningless?
@@johnblankenhorn9730 it's a ratio and it's not even constant (at higher energy levels). Check wikipedia!
Explain it's meaning and a nobel price might be waiting for you.
@@johnblankenhorn9730 it is a ratio and it is not even constant (check wikipedia).
Explain its meaning and you might win a nobel price.
Completely agree. Please, offer a criteria, an educated alternative criteria, that helps an student to look for the right information to make decisions. Just three clear tips to look for that are all related to academic potency and quality of education, that's it. Not about facilities, grants, impact, number of people in the department, size or location of the University, etc. Just pure logical syllabus, programme, topic background checks, department provenance that should be able to teach such topics in deep if needed. That sort of thing. The students then can check the online videos of the Universities selling the accommodation, the clubs, etc.
I think there is one advantage to these random league rankings every year. The best students shuffle into different universities every year so no one university gets everything
the difference is that sports rankings are only for fun, it's well known that tournament is not a true sorting algorithm for instance
8:03 "I get knocked down, but I get up again, you are never gonna keep me down..."
Hi Brady, with all due respect to Phil and mike they didn't explain very well why the sports analogy is different to university league tables. University league tables based on GPA (for example) does indeed have an associated error bar as it is a single number calculated as the mean GPA value but with an associated spread due to some pupils getting much higher or lower than the average (standard deviation). A 100m race does not have error bars unless you decide to take an average over several races by the same runner.
The medical research arena is filled with drug effectiveness rankings and many have been criticised due to not including error bars so that a significant difference to 2 or 3 sigma can be stated.
For the analogy to hold university league tables would have to be baeed on a single student rather than an average if all students.
Rank universities on the sum of teaching staff salaries per student, the number of drop outs per 1000 students, and average debt a student has when they graduate.
The first and by far most important criterion these global university rankings seem to evaluate is what unis are located in the US. That's it. I mean, 24 out of the 25 best rated are supposed to be located in a country where engineers spend their entire first year learning basic European late-middle-school to first year high school maths? Come on...
Even if they don't actually mean much, they mean something to employers and it's a lie to pretend otherwise
*Which* league table means something to employers? Would it be the annual Graduate Market report, for example, whose league table placements differ from those of other sources by 20 places or more?
But Nottingham is placed very highly in that particular table this year so, yes, you're absolutely right -- that's definitely the ranking to plumb for! :-)
Philip
It is all bs.
Greek public universities rank high even though they don't have the basics(eg. Libraries)
And every year they go on about being amongst the "best" in the world,most of which are private for profit businesses.
I have learned that the Ivy League is not that great and Harvard has 14,000 graduate students. The large number is good for publications.
Yes uni rankings are based, but let’s be honest here, there are certain unis that have smarter students, certain unis that have the most influential professors in x or y.
I've not watched for a while, but good to see Professor Moriarty again. I always love hearing his passion. And sorry Brady, but I totally agree with him through this whole video.
Many years ago, when looking at how the rankings are produced for most lists, a few friends and I tried to work out how many factors were primarily proxies for money. The US News one has a single factor that wasn't. (Pay for faculty? Floor space in classroom buildings? faculty ratio? cost per student? Athletic scholarships? Real estate holdings? And so on) The conclusion: Uni's are primarily ranked like porn stars: by the size of the endowment.
Why is this topic not discussed about more? Because most media that will talk about this topic are producing league tables of their own.
When several of the smartest people in the room are getting angry about something, it's worth listening to what they're on about.
Has anyone tried using U-multirank? It doesn't seem helpful. The comparison tool wont load. The "Best matching universities" feature only allows a choice of 2 variables and seems to only include 4 UK universities?
Looking at those tables was the biggest mistake of my teenage years. I should have focused on the specific programs I am interested in, and how the culture fits me. The rankings are pure marketing, nothing more. That endowment is more or less a hedge fund, and hardly a source of money for the student body. Now I'm doing a PhD at a well ranked university, but frankly it doesn't even matter, because what I care about is the work I'm doing, not the school I'm at.
Instead of only conveying critique, I'd argue for another metric. A rival list. Perhaps a quality index. Make a thing of it being a metric of quality instead of the other's more quantity based list.
It is annoying that rankings are inherently unscientific. I don't think you can probably make something that is fully able to rank universities 'correctly,' but we could probably do a lot better
the error bar for the 100m sprint is the error in the measurement, which is very small.
Regarding league tables. That table is measuring only one single thing, goals made during matches and in what distribution they were made.
That would be like ranking universities on Nobel prize laureates and nothing else. Sure, that's easy to quantify, but like the professor said, Universities are more than one thing. Unlike football, where the league is Only based on one thing.
Imagine premier league rankings based on singular most spectacular goal, best fighting spirit, most merchandise sold, and then.. lastly, goals made and games won. Then you have to figure out "which is the most important thing for a "best team"". It's doesn't require a lot of contemplation to figure out that this new "all in one" table will be useless for most purposes.
You can;t choose a school based on the tables because that's last year's result, and next year will be different.
And more often than not, only some departments in a university are worth studying in, while the others could be immaterial.
Merrifeld is gone, and these “grasping at straws” type of videos. Copeland is our only hope and is the only thing that keeps me interested in this channel. The Era of Nottingham University that more or less raised me and cultivated my thoughts on chemistry and physics is passing…
as a postdoc researcher, this ep is so funny. brutally honest qtns as well. thanks!
Brady, this doesn't compare to the league of any one sport. This is the merger of mens & womens leagues, rugby to bowling to chess, who has the better equipment/stadiums, AND who has the better uniforms -- all boilded down to ONE number to cover all those stats!
Loving the amount of pushback and discussion that Brady is bringing to this one.
You just need a publication that ranks league tables.
Does professor Tom of Oxford University (also in numberphile videos)too agrees with the statement made in this video ??
If I had to keep only one thing from the whole video, it would be the "smile" factor. I for one find it to be the best indicator to know if I can thrive in a new team/company/group of people... We're social animals, first and foremost. You do better work when you're happy, and with happy people.
By the way in France, in engineering schools, we have students writing a booklet to present the life of the school (I don't know if it's something widely used in other countries). And they're published on the website of the schools, so that candidates can read about the school from other students and alumni: job perspective, student associations, culture & mentality of the school etc. And it's FAR more useful than engineering schools league tables.
Another thing, barely touched upon, is that the universities look at those made-up ranking categories and game it, so that they do well in them. The measure becomes the target, but stops being a good measure.
Saying that as a St Andrews physics graduate. The tables are nonsense. Not that everyone at our School wouldn't be excellent and lovely, they were. But i really didn't care about the refurbished library or gym. And the student satisfaction survey - also pure nonsense. I couldn't put a number on my own student satisfaction 😅 How do they turn those free text answers into a number anyway?
I think the academics are letting their egos get the better of them over something which has no real effect on a vast majority of students' decisions. If a student is smart enough and accomplished enough to be worrying about which are the best schools, they and their parents are most likely going to use the tables as just one small data point. Prof. Gray even said that the thousands of students she talked to had different motivations.
I'm sure there are ramifications career wise. Moronic administrators probably use the rankings as a cudgel during meetings, "Why aren't we higher ranked!?", but in terms of its effect on students? Zero to nil.
Goodhart's Law: When a measurement becomes a target, it's no longer a useful measurement.
Most rankings use numbers of publications, funding etc. for their statistics ... in Germany most reasearch happens in institutes. They are run and staffed by university-personnel, students and cooperate with those universities. Basically their research-centers, but on paper they don't belong to them. You won't find a german university in the upper ranks. Probably not because they're that bad.
I'm frustrated by how "concerned" you are about how physicists that know how to work with numbers trust these tables without thinking through the quantitative validity.
The education system thinks that it teaches critical thinking, but first and foremost it teaches you to blindly trust in institutions and authority.
Just look what has happened to anyone who didn't "trust the Science™" over the last few years. Being skeptical and asking reasonable questions gets you labeled "anti-science," and you wonder why university students blindly trust the authorities that push these tables when universities love to brag about how well they scored on them.
Brady always seems harder on Professor Moriarty than he does on the others.
Can't be hard on Emeritus Professor Merrifield, though. Congrats on retirement! I'm envious.
I first read this as "The Problem with Ranking Universes"