Beyond the Rankings Colleges search for a more meaningful equation

Author: Tess Russell

Each August, millions of Americans anxiously await the release of U.S. News and World Report's "America's Best Colleges" for a wide variety of reasons. High school students and their parents look to the rankings in hopes of gaining helpful insight into the trying college process, university administrators simultaneously curse the methodology and prepare to manage the fallout if their schools do not perform as expected, and nerds of all ages revel in another opportunity to indulge their fetish for lists.

Indeed, in an age where the fate of print journalism hangs perilously in jeopardy, U.S. News has found in the "Best Colleges" edition not just an annual bestseller, but what can only be described as a cultural phenomenon. In addition, a franchise which includes an extensive college guide and the offshoots "America's Best Graduate Schools" and, most recently, "America's Best High Schools."

Robert Morse, Director of Data Research for U.S. News, has been charged with compiling the college rankings since their third installment in 1987. He explained that the magazine's original goal when it conceived the idea for "America's Best Colleges" was simply to produce a consumer product for readers.

"Obviously the scope has vastly expanded since then," said Morse, "but the main goal is still to provide information for prospective students and their parents that is not readily available elsewhere. That is why we do the rankings - it's not to produce benchmarking for academics so that colleges can have an opportunity to show that they've succeeded in some way, or to create debate on college campuses."

But the U.S. News issue has always created debate, mainly among educators who feel that the rankings attempt to quantify something immeasurable - the quality of learning. Lloyd Thacker, the author of "College Unranked" and the Director of the Education Conservancy (EC), a nonprofit organization committed to reforming the college admissions process, explained the negative impact of the rankings on higher education.

"The rankings have not contributed in any way to educational betterment in America," said Thacker. "They have distorted the way education is perceived and reduced it to a system of variables that are both not reliable and not valid - that is, they don't measure what matters and they aren't accurate."

Morse countered that, while U.S. News doesn't actually encourage rank steering, college students themselves are often the beneficiaries of initiatives designed to increase a school's cache.

"If a school decides to target the ranking variables, assuming that they understand the rankings, that benefits the current students," explained Morse. "If they are able to get a larger portion of alumni to give, that gets fed back into the school budget. If a school targets doing a better job retaining freshmen, graduating more students, having smaller classes, starting more student programs - then I think those things produce a higher quality of education for students and they're the winners in that sense."

Still, as Thacker and other detractors are quick to point out, the "consumer report" mentality that Morse acknowledged seems to suggest that choosing a college is comparable to choosing a car, a dangerous assumption that reduces the output of educational institutions to a single numerical value. To combat this trend, Thacker and the EC drafted a letter in May of 2007 that encourages colleges and universities to make all the raw data that has not been subjected to the U.S. News algorithm publicly available, to refrain from using their ranking in promotional materials and to refuse participation in the "peer survey" component of the U.S. News rankings.

Morse responded to the influence of the EC with skepticism, given the continued increase in demand for the rankings. (The day the rankings went live this August, the U.S. News website received a record 15 million hits.)

"Their aim is to discredit U.S. News in some way," said Morse, "but it's hard to tell whether they're still active or not because they haven't gotten any recent publicity. I don't know how anyone can cite evidence that the boycott of the peer survey has succeeded but maybe I'm missing something."

In fact, the evidence suggests that Thacker's movement is gaining momentum. Since the letter's inception, sixty-seven college presidents have pledged their support. In addition, nineteen liberal arts colleges, including Middlebury, have signed on to a similar agreement put forth by the presidents of Williams and Amherst.

The main difference between the two letters is the refusal to complete the reputational survey of peer institutions. Bob Clagett, Dean of Admissions for the College, attested that boycotting the survey is ineffectual because U.S. News can extrapolate the data regardless, but strongly endorsed the College's decision not to discuss the rankings in any media releases. (In fact, he initially declined to be interviewed for this article, but ultimately agreed to "comment on the College's refusal to comment.")

"We have chosen not to comment because of the overall concern that all of us have that undue significance is being attached to these rankings," said Clagett. "The rankings strengthen a tendency that is already strong enough to approach the process of college selection with a name-brand mentality that ultimately detracts from what should be defining these decisions, which is fit."

Thacker agreed that the "indicators and predictors of good education are easy to know and tough to measure."

He has received hundreds of thousands of dollars from over 30 universities and is currently working to develop a "robust system of information that will guide kids and encourage colleges to compete to be more responsible in terms of the information they provide."

Indeed, the importance of freely available information is the one thing that Morse and Thacker can agree on. U.S. News should certainly be credited with standardizing the research process and giving schools a reason to compile data that they largely ignored until the 1980s when the rankings originated. And colleges, including Middlebury, should be lauded for releasing not only the data they provide to U.S. News, but also the results of participation in the National Survey of Student Engagement, which panels students in an attempt to compile more meaningful, less dubiously "precise" data than the U.S. News rankings.

Still, the rankings will continue to sell, noted Clagett, whether or not those in education support their methodology, precisely because they are so sensational.

"I don't think the desire for this kind of data is going to decrease," said Clagett. "All we can do is continue to try to educate the public about its limited value."