About / June08 / Yun criticizes report claiming voucher program success in Florida

June 3, 2008
For immediate release

John Yun of UC Santa Barbara’s Gevirtz School criticizes report that declares Florida voucher system for special education a success

Professor John Yun of UC Santa Barbara’s The Gevirtz School has reviewed a recent report that claimed a Florida program providing vouchers for special education students was effective; Yun claims the report offered vague and flawed statistical analyses. Yun concludes that the Manhattan Institute report offers policymakers little guidance: “Any attempt to use this report for decision-making or policy evaluation, prior to validation using different methods and more robust approaches, should be viewed with extreme skepticism.”

The Manhattan Institute for Policy Research article “The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida’s McKay Scholarship Program” was written by Jay P. Greene and Marcus Winters. Professor Yun reviewed the study for the Think Tank Review Project.

Florida’s McKay Scholarship Program provides vouchers for special education students to attend private schools. The scholarship program is open to any Florida student classified as having a learning disability; as of the 2006-2007 school year, about 4.5% of Florida’s special education students received these vouchers. The Manhattan Institute analysis appears to cover an earlier period, from school years 2000-2001 through 2004-2005, during which time the program enrolled fewer students.

The Manhattan Institute report is based on statistical analyses that, its authors conclude, shows that the McKay program spurred public schools to improve achievement for those special education students who remained in public schools during that time. The report presents relatively small effect sizes (a small competition benefit), but asserts that these results are probably understated.

Yun’s review finds the statistical evidence weak. Among his criticisms are the following:

The conclusions rest on multiple unsubstantiated assumptions about the direction of possible “selection bias.” This bias occurs because the students using the vouchers are not randomly selected, meaning that important differences likely exist between “average” special education students and those who use vouchers. In each case, the report presumes that the selection bias in Florida results in an understating of the competition effects. But Yun points out serious problems with those presumptions. The most important one is that the report completely ignores other plausible sources of possible section bias that might lead to overestimating the effect of the voucher program.

The model specification for the report’s analysis of achievement measures includes several questionable and difficult-to-justify choices that are unexplained and unaddressed in the report.

The Manhattan Institute report defines competition by the number of private schools willing to accept vouchers within a 5- or 10-mile radius of the public school. Yet, Yun argues, if the loss of students is the critical signal to public schools, the report should have considered the number of available spots relative to the number of students who could fill them. This would more accurately capture the magnitude of the competition pressure felt by public schools.

The report’s approach conflates exposure to vouchers with urban, rural, or suburban location. Because the analysis did not directly include urbanicity, and because urban schools generally perform worse than their suburban and many of their rural counterparts (and thus may have an easier time improving their scores), the results could reflect factors unrelated to voucher exposure and be specific to changes occurring in urban schools.

The report does not explain how public school officials would know how many private schools in the local area were accepting vouchers or the level of capacity in these private schools to enroll additional students with disabilities, nor how very small percentages of students leaving public schools for private schools under the program is likely to encourage widespread change in public schools. Nor does it explain how the mere presence of schools would trigger immediate changes in public school behavior that would be quickly reflected in student test scores.

In reality, Yun points out, “the number of voucher recipients was quite modest until nearly halfway through the sampled time period. In order for the hypothesized competitive effects to have caused improvements in nearby public schools, those schools would have had to almost immediately receive the signal that special education students were leaving their schools and then adjust their practices accordingly, with the effects of these changes then very quickly having an impact on test scores. Such a series of events seems unlikely.”

Yun’s article can be read on-line.

[John Yun is available for interviews; contact George Yatchisin at 805 893 5789]
– end –

School-wide Links
Courses Apply	Computing Contact

News

John Yun of UC Santa Barbara’s Gevirtz School criticizes report that declares Florida voucher system for special education a success