How Well Do Structural Demand Models Work? Counterfactual Predictions in School Choice

New Findings, School Assignment, November 2017

In 2013, Boston Public Schools con­sid­ered sev­eral new choice plans that dif­fer in where appli­cants can
apply. At the request of the mayor and dis­trict, we fore­cast the alter­na­tives’ effects by esti­mat­ing
dis­crete choice mod­els. This work led to the adop­tion of a plan which sig­nif­i­cantly altered choice
sets for thou­sands of appli­cants. Pathak and Shi (2014) update fore­casts prior to the pol­icy
change and describe pre­dic­tion tar­gets involv­ing access, travel, and unas­signed stu­dents. Here,
we assess how well these ex ante coun­ter­fac­tual pre­dic­tions com­pare to actual out­come under the
new choice sets.