Impact Evaluation in Matching Markets with General Tie-Breaking

New Findings, School Assignment, School Reform, January 2018

Many cen­tral­ized match­ing schemes incor­po­rate a mix of ran­dom lot­tery and non-lot­tery tiebreak­ing. A lead­ing exam­ple is the New York City pub­lic school dis­trict, which uses cri­te­ria like test scores and inter­views to gen­er­ate appli­cant rank­ings for some schools, com­bined with lot­tery tie-break­ing at oth­er schools. We devel­op meth­ods that iden­ti­fy causal effects of assign­ment in such set­tings. Our approach gen­er­al­izes the stan­dard regres­sion dis­con­ti­nu­ity design to allow for many run­ning vari­ables and treat­ments, some of which are ran­dom­ly assigned. We show that lot­tery vari­a­tion gen­er­ates assign­ment risk at non-lot­tery pro­grams for appli­cants away from non-lot­tery cut­offs, while non-lot­tery vari­a­tion ran­dom­izes appli­cants near cut­offs regard­less of lot­tery risk. These meth­ods are applied to eval­u­ate New York City’s school progress assess­ments, which give schools let­ter grades as a sum­ma­ry mea­sure of qual­i­ty. Our esti­mates reveal that although Grade A schools boost achieve­ment, these gains emerge only for stu­dents who attend lot­tery schools. Attendance at a cov­et­ed Grade A screened school, includ­ing some of the high­est per­form­ing in the dis­trict, gen­er­ates no mea­sur­able effects. Evaluation meth­ods that fail to take advan­tage of both lot­tery and non-lot­tery vari­a­tion miss this dif­fer­ence in impact.