Hoxby: Article Presents Slanted Veiw of Academic Debate

To the editors:

I am writing in response to the news article “Star Ec Prof Caught in Academic Feud” (July 8), in which The Crimson portrays me as a school choice advocate in a “feud” with an assistant professor at Princeton named Jesse Rothstein. Actually, there is merely an intellectual discussion, and the Crimson should have reported about it rather than engaging in tabloid style journalism. Let me set the record straight.

Substance. The paper in question investigates what happens when families choose a residence partly because of the public schools associated with it. The Crimson reports Rothstein’s claim that when he uses alternative measures an area’s number of streams, my results change substantially (when school districts were originally drawn, it was common to align their boundaries with natural barriers like streams. Thus, areas with more streams tend to have more school districts, and this makes for a “natural experiment” in which some families have more districts to choose among than others). Rothstein’s claim is incorrect. The streams are a red herring: measuring them in alternative ways does not affect the results.

What Rothstein does is discard information on students’ true public schools and residential locations. He substitutes locations that he estimates by cross-referencing characteristics of the school’s zipcode. The cross-referencing procedure misidentifies a student’s true public school district more than 30 percent of the time. Only by substituting erroneous for true data does he get different results.

Restricted-Access Data. The Crimson portrays the fact that restricted access provisions apply to some of my data as extraordinary, but such provisions are routine for school, health, wage, tax, and investment records. I made an informal survey of recent studies in the economics of education and found that 70 percent of them use data that are not entirely public.

Researchers do not impose restricted-access provisions. Rather, they are imposed on us by data owners (in my case, a federal statistical agency), who cannot gather confidential information if they do not protect respondents from identity theft and privacy violations. Restricted-access laws prevent a researcher handing data directly to others. If another researcher wants the data, he must go through the data owner. (the agency that supplied my data works with any legitimate education researcher). The federal government is serious about researchers not distributing restricted-access data. The penalty for a violation is up to $250,000 or five years in prison; and I face periodic, random inspections.

Journals routinely publish papers with an agreement, like mine with the American Economic Review, that recognizes that the researcher can make only the non-restricted part of his data freely available. After my paper was published, I gave out the non-restricted data. Over the years, however, I found that nearly everyone who contacted me wanted the entire dataset, so I worked to have the agency make the whole dataset, the raw data, and the code (more than a gigabyte) available to all legitimate researchers. I was the first researcher ever to do this with the agency and new protocol had to be invented. There are hundreds of papers based on just this one agency’s data for which authors have made no such effort. I was referring to my efforts to make the data available when I told the Crimson reporter that I had gone “above and beyond the call of duty” (the Crimson article puts the quotation in another context where it is misleading).

Replication. In economics, a replication that confirms an existing study is virtually unpublishable; when another researcher confirms my findings, I might get the results, but that is the last I’ll see of them. To get a publication, a replicator must find non-confirmatory results. This situation creates perverse incentives. It also discourages interesting, original work since researchers who do such work can look forward to many hours working with replicators with nothing to show for them. Because economics students only end up reading non-confirmatory replications, they can become confused and think that they are being assigned to find flaws when they are assigned to do replication. The Crimson quotes me as saying Rothstein “may suffer from [sic] confusion,” as though I had made a general statement about his intellect. The omission of the word “such” was strategic, however. If the Crimson had included the word, it would have had to say that I was merely expressing the idea that Rothstein might share the confusion about replication that I’ve described (the quotation is from my writin).

Misleading Statements and Lack of Substance. The Crimson article contains a number of misrepresentations and strategic omissions, most problematically omitting to mention any of my substantive responses to Rothstein’s claims while giving the claims themselves coverage. This is not for a lack of information. I spent a long time talking to the Crimson reporter about substance and gave him the names of other economists who were prepared to talk about substance. He had information about how he could learn about the data; read the restricted-access agreement; reach the agency; contact people to whom I’d given the non-restricted data; and find research similar to mine that had come to similar conclusions.

An exhaustive list of problems in the article would be tedious. Consider, though, the quotation from an anonymous faculty member who said that I told him that I was not conveying the data even to prestigious people. The law on restricted-access data has nothing to do with prestige, so the quotation is misleading. Consider the reporter’s harping on race and gender bias. When asked why Rothstein was so critical of me, I suggested self-interest and ideological bias. The Crimson reporter, who was intent on race and gender bias, inverted my emphasis, entirely omitted self-interest, and suggested that “bias” (rather my substantive reply) was my response to Rothstein’s claims. Consider the fact that I asked Rothstein to communicate with me through the journal so that he could ask legitimate questions but not miscommunicate. Somehow, the Crimson reporter managed to turn the situation on its head and make it sound as though Rothstein decided to stop calling me because I would not answer legitimate questions.

The Crimson article does not refer to a single point of substance that I make in my reply to Rothstein’s comments. This is despite the fact that I early on expressed concern to the Crimson’s President, Lauren Schuker, about the reporter being intent on relaying ill-founded gossip rather than conducting responsible journalism. Ms. Schuker undertook to ensure that the reporter adopted an intellectually serious, responsible approach. Readers can see for themselves this did not happen.

CAROLINE M. HOXBY ’88

Cambridge, Mass.

July 14, 2005

The writer is a professor of economics.