Inside the Numbers?

Your friends may have joked about it for a few days. You certainly read about it in the news. Your parents might have asked you if you knew anyone who went to "that Caltech school."

With the Aug. 30 release of U.S. News & World Report's annual college guide, the college rankings craze that began over a decade ago has continued. There are magazines that evaluate the per-dollar value of your education, there are books that rank graduate schools, and there are top 10 lists of everything from "Best Libraries" to "Best Party Schools."

According to U.S. News & World Report, Harvard, which held the top rank last year, is now the number two school in the country. The California Institute of Technology (Caltech) jumped from number four to number one.

But that doesn't mean you should peel the sticker off the back of your car just yet. According to officials at Harvard and several other schools, numerical rankings such as those in U.S. News & World Report are not an accurate way to compare colleges.

"These simple lists are necessarily very arbitrary," wrote Dean of the Faculty Jeremy R. Knowles in an e-mail message. "The position of a given institution is greatly affected by how the particular calculation 'weights' [various criteria]."

Caltech Provost Stephen R. Koonin says that, while his school appreciates the positive publicity that comes with being ranked number one, he recognizes that the listings cannot be taken at face value.

"It's like using IQ as a measure of intelligence," he says. "It doesn't capture all the factors or all the dimensions of a school."

Administration officials at several schools have recently begun to question to methodology used by U.S. News & World Report and other systems that rank colleges, accusing the magazine of shuffling the rankings each year in order to sell more magazines.

Surveys such as those in U.S. News & World Report consider selectivity, faculty, graduation and retention rates, financial resources and costs, though they often change the relative weightings of these criteria from year to year.

"The fact that the rankings change each year is proof that this isn't an accurate measurement," says Justin Harman, director of communications at Princeton. Princeton dropped from a first-place tie with Harvard last year into a fourth-place tie with Yale this year.

For example, Caltech's jump from number four to number one can largely be attributed to change in the way school spending per student was evaluated. Rather than using a numerical ranking for a school's per capita expenditure, the survey now considers the actual dollar amount spent by the school on each student.

Caltech, with its wealth of lab facilities and a 3:1 student-to-faculty ratio, spends more than double what Harvard, Princeton and Yale spend on each student.

"Nothing changed at Caltech from last year to this year," says Caltech Vice Provost David L. Goodstein. "Things just changed at U.S. News."

Koonin says he agrees with Goodstein, and that he expects the rankings to change again next year.

"Institutions change on a much longer time scale than the news cycle," he said. "It's clear they changed the weightings and could change them again next year."

While U.S. News & World Report's issue may receive the most fanfare, other ranking systems come out each year as well. Kiplinger's and Money magazine each release annual lists of the best values among American colleges.

Harvard generally ranks lower in such "value-based" lists than it does in U.S. News & World Report. For example, the College ranked 19 in Kiplinger's and 43 in Money.

According to Director of Admissions Marlyn McGrath Lewis '70-'73, Harvard has not had a problem attracting lower- and middle-income students as a result of these rating systems.

"We try to address the perception that we're socioeconomically out of reach," Lewis says. "These rankings give very little credit to generous schools for the enormous scholarship assistance we give to students in need."

Provost David Minter of Rice University, which ranked number one in Kiplinger's and number two in Money, says that lesser-known schools such as Rice reap greater benefit from college rankings than schools like Harvard.

"Most of the top schools are on the coasts....We're smaller and younger, so it's hard to get recognition," Minter says. "As long as [ratings] exist, it's gratifying to be number one."

The Princeton Review, known for its SAT preparatory courses and books, also publishes a book entitled The Best 331 Colleges. While it lacks an overall-ranking scheme, the book has a section of top 10 lists based on statistics and surveys of students on campus.

Harvard was ranked sixth on the list for "Best Academic Experience," and had the seventh-best "Quality of Life." Harvard also received the top rating for its libraries and was named the "Toughest to Get Into."

Many college guidebooks, including the Princeton Review's Best 331 Colleges, advise students to select a college that fits their preferences rather than choose one simply because it tops the book's rankings.

While most school officials say that the ratings aren't important, the public--including prospective students and alumni--continues to pay attention to them.

Charlene Leibau, director of admissions at Caltech, says her office has already received three calls from students who had "selected other schools over Caltech but wanted to have their admission reactivated."

"In one case, I asked the student whether this had anything to do with a recently released ranking," she says. "The answer was, 'Yes.'"

Leibau says that while her office will not mention the rankings in the material it sends to prospective students, she still expects the publicity from the survey to translate into an increase in applications from last year.

"Schools that go to the Rose Bowl also receive an increase in applications," she says.

Theresa Davis, director of the Caltech Alumni Fund, says that her office has actively brought the ranking to the attention of alumni when it solicits contributions.

"Our alumni have responded in a very positive way," Davis says. "We make sure to tell them that it's their gifts that helped to make this ranking possible."

Harman says that Princeton has received some fallout from its drop in the rankings this year.

"There are some number of students who will say no to Princeton because they've been accepted by the number one school," Harman says. He said his office has also received inquiries from alumni since Princeton dropped to fourth on the U.S. News list.

Harvard Dean of Admissions and Financial Aid William R. Fitzsimmons '67 advises that students not use college ratings to decide where to attend. He says, however, that students should take advantage of the wealth of information that college guides provide.

"This is a flesh-and-blood decision, choosing a real college with real people," he says. "There's no way a ranking can truly capture that."

"More information really is a good thing, and guidebooks have really democratized the information out there," Fitzsimmons adds.

Fitzsimmons also notes that the admissions office has not seen an increase in inquiries from prospective students or their parents since the U.S. News & World Report guide was released.

According to Knowles, Harvard does not use college ranking systems as a way to evaluate the effectiveness of its policies.

"We continually strive to offer the best educational experience that we can, and what defines 'best' is much more subtle than the fixed (and necessarily arbitrary) weights given...to the many factors that contribute to that experience," Knowles wrote.