Undergraduate Use of Consumer Course Guides Expands

While undergraduates will be able to offer input on the hot issue of curricular reforms in the coming months through the Student/Faculty Committee of Undergraduate Education (CUE), there is little doubt that structural academic changes will both be authored and decided on by Faculty members.

However, the one forum that considers every student's opinion on his or her academic experience--and influences the educational experience of almost every student--is the Course Evaluation Guide published yearly by the CUE.

Since 1973, the CUE Guide has used results from questionnaires distributed to the vast majority of Harvard classes. The results are used to compile statistical information and short written reviews of Harvard's undergraduate courses and instructors.

According to the introduction to the Guide, its aim is "to help students make the most of their academic experience by granting them ready access to their peer's advice."

But the CUE Guide is more than just the Harvard undergraduate's heavily-utilized consumer report on academic programs.

Faculty, administrators and teaching fellows all say that they use the statistics and written comments to improve their own courses and evaluate junior Faculty and graduate students for hiring and promotion.

Student Use

While the majority of students say that they use the CUE Guide when planning their course schedule, most say that the Guide could be improved.

Adriana E. Abdenur '97 says that the categories in the rating are often too narrow.

"My interests aren't always what the class interests are," she says. "For example, I'm an East Asian Studies concentrator, and I'm interested in inner Asia, which a lot of people aren't interested in."

Abdenur adds that although she is more likely to shop a course with a high rating, she does not necessarily rule out a low-rated course.

Students say they feel that many of the categories are confusing and even overlapping. They add that the most useful categories are the overall course ratings and the workload ratings.

"Because all the Moral Reasoning classes sounded the same, I took Justice because it had the lowest workload," says Nick C. Malis '99.

Mixed Reactions from Faculty

While students undoubtedly use the Guide extensively, Faculty and administrators say that they have mixed opinions on the usefulness of the Guide.

Some professors say that they find that the written comments on the back of the form are most useful for improving their courses.

"I take very seriously what's written on the back, positive and negative," says William M. Todd III, incoming dean of undergraduate education.

Owen J. Gingerich, professor of astronomy and the history of science, says he's used the student reports as well.

"Certainly in my own courses we've made significant changes because of the questionnaires," he says.

Faculty and administrators say that the CUE numbers are often used to help decide which teaching fellows to hire and which junior Faculty to promote.

"Evaluations have to be a part of every promotion case," Todd says.

But most professors agree that they cannot rely exclusively on the CUE numbers to evaluate teaching ability.

Todd says that he supplements the CUE evaluations for TFs with visits to the class sections.

"I balance my own evaluation with that of the students," he says, adding that students cannot always perceive the qualities that make a good teacher.

Some instructors take the view that the statistics provide much more valuable information than written comments do.

"I felt that the numbers were useful, and the narrative was useless at best, and usually mostly confusing," says James E. Davis, head tutor of the Department of Chemistry and Chemical Biology.

"They should try to write meaningful prose or let the numbers speak for themselves," he adds.

Professors who prefer the written comments in the Guide acknowledge that it can be difficult to interpret.

"I'm not thrilled with the presentation of the results in the CUE. In the past three or four years it's become incredibly mechanical," says Professor of History Mark A. Kishlansky.

Recently, worries that students misinterpret the statistics presented in the Guide has led the editors to include a translation of "CUEspeak" and further clarification of what the numbers mean in their introduction.

Despite the warnings, however, Davis says that students continue to misread the Guide--for instance, many falsely assume that because one-third of respondants praised the textbook, two-thirds did not like it. As the introduction would explain, the two-thirds likely had no opinion at all.

Graduate students say that they also doubt the accuracy of the CUE Guide.

"To a certain extent, the ratings reflect quality of teaching, but also may reflect other things--popularity of [the] TF in places beyond teaching," Lee says.

Objectivity

Despite criticisms, CUE officials say the Guide strives for impartiality.

"The CUE Guide is meticulously edited to avoid biases of any kind. We make every effort to assure that the publication is as representative as possible of student opinion," writes CUE Guide writer/editor Jared H. Beck '99 in an e-mail.

Although students say that their own responses are not always representative of their own opinions, Pilbeam says the Guide is as accurate as it can be.

"[The CUE Guide] is accurate in that it represents exactly what people write on the forms," he says.

A number of factors affect accuracy and objectivity before the questionnaires ever reach the Guide's writers, students say.

For example, small seminar classes invariably get higher ratings than large lecture courses. Part of the ratings' disparity is most likely due to student preference for classes with professor contact, but Carlton F.W. Larson '97 suggests a different reason.

"The ratings in the small classes tend to be pretty high because the students in them need to get recommendations from the professor, and so they're not going to tear them apart," Larson says.

In addition, the CUE questionnaires are usually passed around in the last minutes of lecture, a time when many students need to get to another class or are simply fatigued.

"There's definitely a problem with wanting to rush and get the thing done with," Larson says.

Even though the statistical data may be accurate, some students say they are not as careful with the written portion of the evaluation.

Abdenur says she feels she is objective when rating elements of the course on the statistical portion, but her commentary on the course may be more subjective.

"It's just for the written responses that I'm affected by the amount of time I have or my mood," she says.

Because students fill out the questionnaires toward the end of the course, the last few lectures are the most fresh in students' minds, and may unduly influence their evaluation of the entire course, some students say.

"There are times I regret having written something needlessly harsh--not often, but it's happened," Larson says.

History of the CUE Guide

The CUE has been influential in improving courses in its 25-year existence.

The Committee on Undergraduate Education launched its first course evaluation project in 1972 as one of several educational reforms instituted by former president Derek C. Bok during his first year in office.

CUE Guide forms are photo-copied and distributed to instructors after grades are determined.

Before the CUE Guide, the only publication that reviewed Harvard classes was The Crimson's Confidential Guide, published yearly since 1925.

The early Confidential Guide consisted of a written review of each course and the recommendation "Stop, Caution or Go."

In the early years, the Confidential Guide was published under the auspices of the University.

However, professors and administrators became dissatisfied with the Confidential Guide's reviews. In 1972, Bok suggested that the CUE publish reviews of courses that were to be based on "statistics rather than common knowledge," says Jeffrey Wolcowitz, associate dean of Undergraduate Education.

"The CUE Guide was my idea," Bok says. "We needed something more reliable and something that would be taken more seriously than the Confi Guide. The professors are more willing to listen to an official University document."

According to Gingerich, Bok believed the Confidential Guide was irresponsible and needlessly harsh.

Gingerich says that Bok once read a scathing evaluation in the Confi Guide of two assistant professors. According to Gingerich, when Bok read the statistics about the same assistant professors in the CUE Guide, their evaluation was much more "acceptable."

According to Gingerich, Bok then called the Confi Guide to complain about its harsh treatment of the junior Faculty.

"Then [Bok]...asked [the Confi editors], 'What are you doing? What you think is a wonderful joke is a deadly thing for people who are nontenured--it could seriously jeopardize their career,'" Gingerich says.

Similar previous conflicts had prompted Bok and the CUE to start an objective, statistical review of Harvard courses in 1972.

In its earliest days, the CUE Guide represented a collaborative effort by students and professors, but by 1977 its editors believed that its editorial independence was crucial.

"What is and what is not printed in these pages is determined solely by the student editors and not by an administrative agency," the editor-in-chief wrote in a preface to the 1977 edition.

But in the summer of 1985, directly before the CUE Guide went to press, Dean K. Whitla, director of the office of instructional research and evaluation, ordered editors to delete or temper criticisms of several instructors.

Editor-in-chief Barbara S. Okun '86 told The Crimson that Whitla ordered the editors to eliminate the words "condescending" and "arrogant" from descriptions of professors.

Whitla, who had consulted an advance copy of the Guide in order to predict course enrollment, threatened to cancel the book and fire the staff if the criticisms were not omitted, Okun said.

Okun made the changes, but protested administration "censorship" in a preface to the book inserted without approval.

Now, the CUE Guide is entirely written and edited by undergraduates, but the policy guidelines, including acceptable language and the format of the questionaire, are set by the CUE.

Most recently, the Guide's Faculty supervisors decided to review the questionnaire of the CUE because it had been met with several criticisms.

According to Dean of Undergraduate Education David Pilbeam, the CUE noticed this year that some of the questions on the form could be interpreted in a number of ways.

"For the first time in living memory, it became clear that many questions were ambiguous," Pilbeam says.

Pilbeam adds that although the Committee will not change the form this year, it will be testing some new questions in a focus group.