Testing Irresponsibility

>Picture it: It's senior year in high school, the morning of your second crack at the SAT. You're nervous but optimistic. You've put in all that time all over again, relearning the ins and outs of the test, mastering every kind of math question, memorizing countless vocabulary words. You arrive at the test site early but not too early, clutching a box filled with more number two pencils than you know what to do with. After a tense half-hour of rule-reading and circle-filling, the moment of truth arrives. The proctor announces, "begin," the test book flies open and almost immediately, the questions in section one look remarkably familiar. Perplexed, you flip to the analogies. You've definitely seen them before. As a tentative smile surfaces on your face, you turn to the first reading comprehension passage. Yes, it's that same one about insect mating rituals!

A college-applicant's daydream? An ill-timed practical joke? An exercise in deja vu? No, actually it's a true story, brought to you by the Educational Testing Service (ETS) and the College Board. On Sunday October 22, (a testing day for observant Jews and Jehovah's Witnesses) the test originally given Sunday April 2 was repeated in its entirety. Approximately 100 of the 2500 to 3000 students taking the test (between three and four percent) were seeing the three-hour test for the second time.

But ETS and the College Board didn't respond to the incident with a profound apology, or even an indifferent "oops." In fact, this was neither a freak accident, nor a bureaucratic blunder. "It was not a slip-up," asserted Ray Nicosia of ETS. Defending the practice of reusing old exams, he said that ETS has been repeating versions of the SAT for "a great many years." Janice Gams of the College Board, adding to the don't-worry-we-know-what-we're-doing defense, suggested that when students have no knowledge that the exam will be repeated, "seeing the test form a second time does not guarantee a score increase." (Neither does bringing a pocket dictionary.) Gams added a statistical punch to her rationale, explaining that, "On average, students who repeat the same test form tend to get one more verbal and/or one more math question correct than if they had repeated with a different form."

Now it's a relief to know that Mr. Nicosia and Ms. Gams see no real problem with students repeating the same test, but their position is worth considering nonetheless. For starters, Mr. Nicosia's appeal to precedent does little to legitimize the practice of repeating exams. And Ms. Gams can cite whatever statistics she likes. Regardless of her findings, it is irresponsible for the College Board to pretend that those seeing the SAT for the second time do not have an advantage. Think of the student who vividly remembers one of the reading comprehension passages and breezes through it, or the one who has looked up all the analogy words he didn't recognize during his first test. Granted, as far as test-takers go, I'm on the obsessive side, but after my first SAT experience, suddenly 'convalescence' (antonym: attrition) was popping up all over the place. I sure as hell wasn't going to get that word wrong again.

While 100 students operating under unfair conditions amount to but a tiny fraction of the 1.8 million who take the exam each year, there is a broader concern here about the only truly national college-entrance test. Indeed within the national pre-college community there is an almost reverential attitude towards the SAT. It goes unquestioned that colleges want you to have scholastic aptitude--whatever that is--and the SAT tells you how much of it you have. But is that the end of the story?

The SAT is a yearly rite of passage for the college-bound, and as the only nationally comparative factor, thousands of colleges and universities weigh scores heavily in their admission processes. It is used to narrow the field for designating Presidential Scholars, and (in the PSAT form) to hand out thousands of National Merit Scholarships. If the noble souls who write this test can't even figure out that giving an identical exam to the same people twice is unacceptable, should they be entrusted by students and learning institutions with such extraordinary power?

The writing of tests is "a very expensive process," contested Nicosia, which would be "prohibitively expensive" if special administrations such as the Sunday test always used fresh exams. Given what is at stake, this concern about finances is troubling. We'd like to think that the people who make the SAT work in an economic vacuum. But attention to the costs is also instructive, as ETS and the College Board have recently introduced a number of changes that will be sure to increase revenue generated by pre-college testing.

Along with the revision of the SAT last year, Achievement tests were renamed SAT II. ("If you liked the SAT, you'll love SAT II.") Whereas in past years every uncanceled Achievement score became a part of each student's permanent testing record, students can now choose to send colleges only specific test results. So students with the time and money can take the new SAT II three or four times until they hit their target scores. Of course, this new service is accompanied by a "retention fee" for those who opt for anything but full disclosure. Along with the opportunities to receive rushed score reports, test-return, and to buy books with sample tests, all those companion industry dollars flow right into the College Board-ETS establishment. So were these recent changes in policy designed to provide colleges with better information about their applicants? Or is it about sucking more money out of high school students looking for the best possible numbers to put down on their transcripts?

We would like to believe that neither ETS nor the College Board would let its educational scruples become hostage to concerns about the bottom line. And yet this recent test repetition incident makes it apparent that here, just as in another business, there is a trade-off between quality and cash-flow. The question becomes whether the trade-off results in anything less than fairness. Is the SAT the well-constructed, well-administered test that its marketers say it is, or is it just the best predictor of college success that can be afforded in a world of "prohibitive costs"?

With such an important role in the admissions decisions of colleges around the country and the lives of generations of students, we ought to hope that the SAT fulfills its educational mission. We ought to hope that those folks over at ETS and the College Board know what they are doing. But this recent incident indicates above all things that we shouldn't count on it.