Software Eases Essay Grading

From first-grade teachers to overworked TFs, grading student essays consumes a huge portion of instructors' time. Multiple-choice tests and oral exams are imperfect solutions to the problem, and students' writing skills go untried.

But a new computer program promises to help change all that. Developers of the Intelligent Essay Assessor claim it can grade essays as well as a human professor--but far more quickly and reliably.

"What we've done is to create a system that would grade in the same way I would grade," said Peter W. Foltz, an assistant professor of psychology at New Mexico State University and one of the project's leaders.

The researchers plan to offer the essay grader to public schools, colleges and national examiners like the Educational Testing Service.

But some educators expressed skepticism.

"I'd have to be convinced of the quality," said Dean K. Whitla, a lecturer on education at the Graduate School of Education. "Automation certainly can help, but it has to be seen as a tool...It shouldn't dominate [students'] lives."

But developers said test trials of the program have been positive.

"The people we've actually worked with have really enjoyed it," said Darrell Laham, a doctoral candidate at the University of Colorado at Boulder, who has worked on the software for three years.

"This will give people the opportunity to assign essays."

Foltz allowed students in his psycholinguistics class to have their essay drafts graded by the computer, and then offered to re-grade the final essays if students were dissatisfied with the assessment. None of them took him up on his offer, he said.

The program plots millions of words in a "high-dimensional space"; synonyms are close together, while words rarely used in combination are far apart. That allows the software to recognize synonyms, making no distinction between "doctor" and "physician," for instance.

"It's not really looking for reasoning ability but at the choice of words and how those words are expressed," Foltz said.

To use the program, a professor must grade several papers ranging from excellent to poor and input them into the computer. The essay grader assigns high scores to those essays that use word combinations similar to the sample high-scoring papers.

But the essay grader can't not judge style or originality--the program works best with focused, fact-or content-based essay topics in disciplines like history, science and economics, Foltz said. In fact, highly original essays are rejected by the computer as irregular and must be graded by hand.

Don't expect to see the essay grader make an appearance at Harvard anytime soon, Whitla said.

"The Faculty here...like to personalize student activities and work much more than these programs do," he said.