The Meaningfulness (or Lack Thereof) of College Rankings

The Washington Post recently ran an article about some colleges taking themselves out of U.S. News and World Report's annual rankings. These rankings lead to deep feelings of conflict within portions of higher ed. Gettysburg College, for example, tends to get this in spades because we are usually ranked in the tail end of the 40's for liberal arts colleges and the front page of the listings goes up to 50. to fall off the front page is perceived as disastrous for recruiting, even while we poo-poo their claimed meaningfulness.

On the one hand, you hear these ranking frequently disparaged on several grounds. First, the criteria used in U.S. News' algorithm do not really tell you about the quality of education that you would receive. If you use more adjuncts, your score goes down even if you are hiring in retired faculty who are master teachers instead of research-heads who resent being in the classroom. A significant factor is reputation, so whether you deserve it or not, the fact that people who have no idea what is happening on your campus have a positive or negative knee-jerk reaction to your name pretty much cements you close to your traditional rating. Second, the numbers do not mean anything. That you are 33rd this year and not 36th, implies that you are 3 better at something this year than you were last year, but of course that is not true. Thirdly, the factors are known by institutions who then craft policy to fool the rankings, not in line with best practices. It makes schools look better, but hampers their efforts to get better. Finally, you cannot give a simple linear ranking. Different schools are more or less appropriate for individual students. It's about fit. Schools have vastly divergent cultures, emphases, and characteristics that would enhance some students' experiences and diminish others. Trying to shove that into some line-up from best to worst is a category mistake.

On the other hand, schools are obsessed with the rankings because they make a real difference in the world. Parents and students take them EXTREMELY seriously, more so than they should. The stories are frequent about the prospective student who says that I was accepted to both this school and institution x, and I am going there even though I like this campus better because they are ranked four places higher than here. When you appear in certain ranges, numbers of applications go up and allows you to be more selective. More selective means better students, and that means an improved campus culture, more fellowships and grants to brag about, and all sorts of other improvements. The rankings are like money, they are valuable solely because people think they are valuable and that value can then be cashed in for goods and services desired.

What makes the objections a bit less than honest is that we do buy into a sense of the rankings ourselves and only admit it behind closed doors. We talk about colleges which we partition in "reach," "like," and "watch" classes. Our reach schools are our older siblings whom we idolize and hope to grow up to be just like someday. The like schools are the ones we sit with in the cafeteria and call our friends even if we gossip about them behind their backs and it wouldn't break our heart to see them drop their tray and embarrass themselves in front of everyone. The watch schools are the ones we might have a couple beers with, but would never admit to waking up next to the following morning. As faculty, we say of certain positions, but not others, that it is a "good job" and when talking over spinach dip at conferences, you know when your peers approve of your home institution and when they figure this must be a fling while you wait to find "department right."

So it is interesting that the colleges leading the charge here, the ones opting out of the academic version of keeping up with the Joneses, are some of the Heathers of the lot, very prestigious schools who are doing quite well in the rankings. Is it the sort of principled stance that idealistic academics are supposed to take? No doubt, in part. Could it also be sour grapes that they are doing well, but not as well as they think they should be doing? Maybe. Is it maybe the sense that not only are they doing well, but that they are so good, they don't even need to play that silly game? Rankings are for losers who need some stupid magazine to pat them on the head. Maybe. Is it resentment that non-academics have the nerve to judge us who sit so high above them? Dunno.

Whatever the motivation, it ain't going to work. U.S. News rankings are and will remain important because the families of high school students looking at colleges are among the worst informed consumers out there. They don't know exactly what it is they are shopping for and have virtually no sense of how to compare the different options. Most people only know of a few universities, usually those that are local or have well-known football or basketball teams. These are major research universities whose teaching record is far from stellar. "But there are several Nobel Prize winners there, doesn't that mean that they will be taught by the best and therefore become the best?" No, it doesn't work that way. "But they have a new athletic center with a bowling alley!" Oy. You can't blame them for this because they do not know -- and to some cannot know -- what to check out. Who knew their daughter would major in anthropology? Who knew their best anthro teacher would get denied tenure in a power play because of factional in-fighting? Who knew that her course load and advisees would be picked up by a senior member who would have retired years ago but for his collapsed 401k and despises his students? The most important aspects are often things you would never know to look for nor would be able to find out even if you did figure out what you wanted to know.

So, what's left? U.S. News and world Reports' rankings. It's all they've got. No, they don't understand them. Yes, they think they mean something they don't. But the fact is they are real and given a sense they don't deserve. They are the Moody's of higher ed and just as we are reforming the financial system whose collapse was due in part to incestuous ratings mechanisms that could be manipulated, so too we need to reform the rankings.