If you’ve clicked around the economics blogosphere over the past few days, you’ve probably seen something about the Reinhart-Rogoff critique released on Monday. Researchers at the University of Massachusetts – Amherst attempted to replicate the conclusions of a paper published by famous economists Kenneth Rogoff and Carmen Reinhart that concluded that debt-to-GDP ratios over 90% were associated with significantly lower (and possibly negative) growth rates. The UMass researchers found that coding errors led to the accidental exclusion of some countries from the analysis, leading to skewed and inaccurate averages. More about that here.
While the debate over these new findings is important in its own right, the hullabaloo over the replication study reminded me of a fascinating episode in the history of a highly-unrelated branch of economics – the economics of education, specifically, the contentious issue of school choice.
School choice is one of the perennial tentpole issues of education reform. Basically, advocates of school choice want to provide students and families with a greater range of school options, above and beyond the default local public school. One argument for increased school choice is that choice induces competition among schools to retain students and funding.
This is where economic theory comes in. Normally, we do not think of public goods (like public education) as open to competitive forces (ignore private schools for the moment). This assumption changes if the consumption of that public good is tied to a specific location – is you can’t get exactly the same public education everywhere, but instead the specific type of public education you get is contingent on where you are. In this case, people will sort into different locations depending on their preferences for local public goods. In the case of education, you might expect to see people sorting into school districts with good schools and leaving the districts with bad schools. This creates the kind of competition advocates of school choice want. The original model of location choice based on the provision of local public goods is credited to Charles Tiebout, and choice of this kind is called Tiebout choice.
In 2000, economist Caroline Hoxby published an influential study in which she attempted to quantify the impact of Tiebout choice on public school quality in U.S. metropolitan areas. Her original hypothesis was that cities with a larger number of school districts had a greater amount of Tiebout choice, but this hypothesis posed a problem – the number of districts in a city might be related to school quality, if local governments decided to break up or merge districts with failing schools. To solve the problem of reverse causality, Hoxby discovered an ingenious solution – look at the geography of school districts.
Hoxby noticed that geographic characteristics of metropolitan areas affected the number of school districts in the area – specifically, she noticed that the number of large streams in an area was related to the number of school districts in the area. Streams seemed to be natural boundaries that affected the ways the original school districts were drawn. Because the number of streams in an area has no plausible effect on school quality except for its effect on the number of districts, Hoxby was able to use the number of streams in an area to isolate the variation in the number of school districts caused by nature, and thereby estimate the effect of increased Tiebout choice on school quality (or as she called it, school productivity). She found that increased Tiebout choice is associated with improved school quality and a lower the number of private schools in an area – a huge win for school choice advocates.
Five years later, however, a young economist named Jesse Rothstein published a comment (much like this week’s comment on Reinhart and Rogoff) contesting the results. Rothstein claimed that Hoxby’s measure of streams was flawed and subjective, and that after constructing his own measure of streams, Hoxby’s results became insignificant. “The exercise makes clear that Hoxby’s larger-streams variable is subjective and unverifiable,” Rothstein claims in the paper. Rothstein also complained that Hoxby had not made the data she used available.
Hoxby published a passionate reply to Rothstein’s reply, writing “I discuss every claim of any importance in the comments. I show that every claim is wrong.” The tenor of the debate became so contentious that the story made the front page of the Wall Street Journal, with both economists on the record with sharp comments about the other.
Unfortunately, the core issue of school choice was lost in the fray. The debate exposed one of the truths of education reform and the economics of education – many of the methods activists tout as quick fixes for the education system are difficult to assess and have subtle effects that are hard to test. Recent evidence suggests choice might have moderate positive effects on later life outcomes, but the jury is likely to be out on school choice for many years.