n.V1-7.17 | CONTRACT TEACHERS AND TEACHING AT THE RIGHT LEVEL

01| Two education reforms, both demonstrably effective, both handed to governments to run, both producing nothing. This chapter is about the handover, because the handover turns out to be the intervention.

02| Bold, Kimenyi, Mwabu, Ng'ang'a and Sandefur embedded a randomised trial inside a nationwide Kenyan teacher-hiring reform. Contract teachers — hired on fixed terms outside the civil service — had already shown strong results when an NGO ran the scheme. The trial put the identical contract in government hands in one arm.

"New teachers offered a fixed-term contract by an international NGO significantly raised student test scores, while teachers offered identical contracts by the Kenyan government produced zero impact."— Bold, Kimenyi, Mwabu, Ng'ang'a & Sandefur (Experimental Evidence on Scaling Up Education Reforms in Kenya, Journal of Public Economics, 2018, Vol. 168, pp. 1–20)

03| Observable teacher characteristics explained little of the gap. The authors attribute it to bureaucratic and political opposition to the contract reform, which produced implementation delays and divergent interpretation of the same written terms. The contract said the same thing. It meant something different depending on who was holding it.

04| The second case is Teaching at the Right Level, an approach with as strong an evidence base as anything in development education: group children by what they can actually do rather than by grade, teach to that level, and test results rise substantially. Pratham had demonstrated it repeatedly with community volunteers working outside school hours.

05| Scaling it into government school systems across four Indian states produced, in the first two — Bihar and Uttarakhand — no adoption at all. Government teachers did not use the methodology, despite training sessions that were well received and ongoing organisational support from the originating organisation. Not a diluted effect. Non-adoption.

06| Two independent cases, two countries, two sectors, identical structure: the intervention works, the transfer does not. This is the strongest evidence in this book part for a proposition that sounds like an excuse when a programme states it about itself — that the failure is in the system carrying the intervention rather than in the intervention.

07| It also disciplines the word "scale", which does an enormous amount of unexamined work. Muralidharan and Niehaus decompose it into three distinct dimensions: the size of the sampling frame, the number of units treated, and the size of the unit of randomisation. Each implies a different failure mode. A result may not generalise because the sample was unrepresentative; or because effects change when many are treated at once, through market and spillover channels; or because the intervention behaves differently when the deciding unit is a district rather than a village. "It doesn't scale" collapses three mechanisms into one phrase and points at no fix.

08| What happened in Kenya and Bihar is arguably a fourth mechanism that the taxonomy does not contain: the implementer's political economy. The intervention was unchanged, the population was unchanged, the unit was unchanged. What changed was who carried it, and what that carrier's staff had to gain or lose from carrying it well.

09| The limit of this evidence should be stated because it is often overstated. Both cases are NGO-to-government transfers, and government bureaucracies have specific features — tenure, unions, political exposure, career incentives detached from measured performance — that may explain the whole result. There is no comparable evidence on NGO-to-NGO or community-to-community transfer. Anyone arguing that a working design can be carried from one place to another by some other kind of carrier is arguing beyond what these studies establish.

10| The practical consequence is uncomfortable for the entire evaluation enterprise. If an effect size is a property of the pairing — this intervention, this implementer — then a trial result is not a portable fact about the intervention, and twenty years of trials have been measuring something less general than they reported.