Published by Town&Country
On July 8, US News & World Report announced that it was taking away Columbia’s number two ranking on its 2022 Best Colleges list. The university had shared the spot with Harvard and MIT but was dropped after it declined to provide additional information the publisher requested for its 2022 and 2023 survey. Earlier in the year, one of Columbia's own mathematics professors published a 21-page paper questioning the data behind the ranking. In late June, the school’s provost announced that the university would not respond because it was still assessing the claims. “The ongoing review is a matter of integrity. We will take no shortcuts in getting it right.”
It wasn’t the first or even last time a college was dropped from the list. Indeed, Villanova was removed a few days later when it reported errors in data it had submitted, making it the tenth school to be taken off this year. But the incident involving a prominent Ivy League university put into sharp relief a battle that’s been brewing for decades.
…
In his February paper, An Investigation of the Facts Behind Columbia’s U.S. News Ranking, Columbia’s Michael Thaddeus laid out broad concerns about their accuracy and usefulness and then offered a detailed analysis of the data U.S. News had used (some which Columbia had provided) to arrive at that number two spot. His calculations painted a different picture of where the school should have rated in various categories, including class size and spending on instruction. It was a new type of critique—a mathematician correcting the math—but many of his subsequent conclusions echoed points voiced earlier by others: Specifically, schools can manipulate data to improve their ranking; the lists don’t fully capture important differences between colleges and universities; and their prominence can have a negative effect on students and educational institutions.
…
Diver’s next piece of advice may sound counterintuitive given the subject of his book. “Once they start to answer those questions, I tell them to go look at lists to find schools that might be a good match,” he says. Though rather than relying on U.S. News, he suggests they start with College Scorecard, the Department of Education's consumer-oriented website that allows users to search by categories, including geography, speciality, and affordability. “But by all means also Google ‘Best School for B Students’ or ‘Best Engineering Schools’—whatever it is that fits your criteria,” he says. The point is not to blindly accept rankings from any one place, but to instead use all the resources out there to “create your own list of schools that you can then start to plow through.”
“Plowing through” means whittling down options to a manageable number and making your ranking. Visiting schools in person can be a crucial part of the process. “College tours aren't perfect but students so often tell me things like, ‘I just took one step on that college campus and I knew it wasn't the place for me,’ or vice versa,” Diver says. “In the end, college is a socializing experience. It’s important to ask, ‘Could I feel at home here? Could I hang out with these people and in this kind of a place?’” He admits none of this advice is particularly earth-shattering—many applicants and their families already do all of the above. “But what I tell people is that this early process of gathering information and making assessments about yourself and the schools you see is a vital part of your education—don’t give that up by basing your decision on a list.”
“I just don't buy that any kind of ranking is actually possible in a way that is helpful,” says David Strauss, another nonadmirer of the practice who is a principal at Art & Science Group, a Baltimore-based consulting and research firm that works with universities and colleges. “Each student has his or her own character, needs, and aspirations.” He agrees with Diver that rankings hold tremendous sway at the graduate level. “When it comes to law, business, and med schools, the they are indeed sometimes the most powerful determinant of choice.”
But the story changes when you look at college applicants. Art & Science Group has performed research and conducted surveys on rankings on and off since the mid-1990s. “It has remained the case that most people [applying to college] didn't look at them meaningfully. The students and their parents who did used them more as validation than as the basis of choice. In other words, “‘I'm going to one of the top 15 institutions,’ not, ‘I chose it because it was one of the top institutions,’” Strauss says. Why is this important? “I think we exaggerate the rankers’ influence at a cost to everyone but U.S. News and other providers. And by doing so we validate them more and more.”
Columbia’s removal from the U.S. News & World Report list continues to generate conversation about rankings. But neither Diver nor Strauss hold out much hope that it will have a lasting effect. “You’d have to have a lot more scandal for people to say, I can't brag at a cocktail party anymore about the ranking of my child’s school,” Strauss says. Diver is a bit more circumspect: “Yes, these criticisms have been made many times before but perhaps it will lead to broader examination at the high school level—maybe even the introduction of academic courses—on what students should know when they apply to schools.”