How Well Do Structural Demand Models Work? Counterfactual Predictions in School Choice

New Findings, School Assignment, November 2017

In 2013, Boston Public Schools con­sid­ered sev­er­al new choice plans that dif­fer in where appli­cants can
apply. At the request of the may­or and dis­trict, we fore­cast the alter­na­tives’ effects by esti­mat­ing
dis­crete choice mod­els. This work led to the adop­tion of a plan which sig­nif­i­cant­ly altered choice
sets for thou­sands of appli­cants. Pathak and Shi (2014) update fore­casts pri­or to the pol­i­cy
change and describe pre­dic­tion tar­gets involv­ing access, trav­el, and unas­signed stu­dents. Here,
we assess how well these ex ante coun­ter­fac­tu­al pre­dic­tions com­pare to actu­al out­come under the
new choice sets.