n.V1-9.01 | The Test That Never Closes

01| When a disaster hits, the question has an answer. Did the food arrive? Were the tents up before the rain came? Did people live through the week? It is a brutal question, but it closes. Somebody can stand up at the end and say yes or no.

02| Poverty work outside a crisis has no such moment. An organisation can work the same district for twenty years, raise money every year, publish a report every year, and never arrive at a point where anyone says plainly whether it worked. There is no week in which people either eat or do not. There is only next year's proposal.

03| That is the standard indictment of the sector, and it is where most critics stop. Stopping there is a mistake, because the comparison holding it up does not survive inspection.

04| ALNAP is the humanitarian system's own evaluation network — not an outside critic, the sector's own instrument for looking at itself. For years its recurring report has documented weak evidence and thin accountability to the people being helped. In a companion study bluntly titled Insufficient Evidence? it argued that the failure to generate and use evidence makes humanitarian action less effective, less ethical and less accountable. Later ALNAP work describes accountability to affected people as having become a professionalised function, treated as tangential to the main job rather than as the main job.

05| So the emergency sector is not the tested one. It is the one where failure is more embarrassing. The historian Michael Barnett makes the related argument in Empire of Humanity: humanitarianism's picture of itself as a clean delivery operation was constructed over time, by the sector, for the sector. It is not a property of the work.

06| The correction matters more than it looks. The problem is not that long-term poverty work escaped a test that crisis work passes. The test is weak almost everywhere. In long-term work its absence is simply harder to hide, because nothing forces the question to be asked on a deadline.

07| With that in place, the standard criticisms are worth going through one at a time. Each has a serious version. Each has a serious person arguing the other way.

Answering to the wrong people

08| A company answers to customers who can walk away. A government answers, in theory, to voters who can remove it. An organisation working on poverty answers to its funders, and its funders are not the people it serves. The accountability runs upward, away from the poor, toward whoever writes the cheque.

09| The two audiences want different things. Funders want clear stories and countable results inside a grant cycle. Durable change is slow, hard to photograph, and usually cannot be attributed to any one organisation. Given the choice, an organisation that wants to survive to next year reports the countable thing.

10| This is not a fringe complaint. It is the settled description inside the field's own literature. Alnoor Ebrahim named the consequence in 2005 as accountability myopia: the mechanisms pointing upward and outward — to donors, regulators, the public — are well developed, while the mechanisms pointing downward to beneficiaries and inward to the organisation's own learning stay weak. The imbalance quietly destroys the organisation's ability to find out it is wrong about anything. Michael Edwards and David Hulme had set the formulation a decade earlier, and it has not been improved on: too close to the donor, too far from the poor.

11| Where the argument goes open is the next step. Almost everyone agrees the asymmetry exists. Very few can point to a downward accountability mechanism that has been shown to work — a feedback system that changed a decision, a complaint channel with consequences, a beneficiary who could withdraw something. The gap between a diagnosis everyone accepts and a cure nobody has demonstrated is one of the more interesting silences in the field.

Nobody is checking

12| Closely related and harder to research: who verifies the claims. Much of this work happens across borders, in places where regulation is thin and where no outside body has both the authority and the local knowledge to check whether a stated result is real. An organisation reports that ten thousand people were lifted out of poverty. Very few people are positioned to ask what that sentence means, how it was counted, or what would have happened anyway.

13| This is not a moral failing peculiar to anyone. It is a structural feature of a field in which the people paying, the people delivering and the people being counted are in three different countries, and only one of them reads the report.

Whose agenda is it

14| A large share of organisations working in poor countries draw funding from foreign governments, and money carries priorities. So: is the organisation pursuing the community's agenda or the foreign policy agenda of the government paying for it? William Easterly argued a version of this in The White Man's Burden and The Tyranny of Experts; Dambisa Moyo argued a harder version in Dead Aid. Both are forceful, widely read, and heavily contested — the Easterly–Sachs dispute has run for two decades without resolving.

15| More interesting is a critique that turns the question on the critics. In 2021, Peace Direct, with Adeso, the Alliance for Peacebuilding and Women of Color Advancing Peace and Security, published Time to Decolonise Aid after a three-day consultation with 158 practitioners, activists, academics and decision-makers. Its finding on localisation — the sector's own commitment to shift money and power to local organisations — was that it had been a major disappointment to the people it was meant to serve, with donors continuing to fund familiar international intermediaries rather than accept the procedural and reputational risk of funding local ones. It is testimony, not measurement, and should be read that way. But it puts a hard question to the whole northern critique: the people closest to the problem still have least control over the money meant to solve it, and that includes the money spent on criticising how the money is spent.

What is left standing

16| None of this makes these organisations worthless. They have raised issues governments and markets ignored, argued for people with nobody else arguing for them, proposed solutions nobody else was proposing. Several of the sharpest criticisms in this book came from inside the sector, in its own journals, from people who spent careers there. A field that produces its own indictments is not a field without self-knowledge.

17| But watch what happened as we went down the list. The founding comparison collapsed. The accountability complaint held, and its cure has never been demonstrated. The verification complaint held, and largely explains the others. The agenda complaint held, and landed partly on the people making it.

18| What connects them is not corruption. It is a field that moves very large sums without a reliable way of finding out whether any of it worked — and without much agreement among its own critics about what going wrong would even look like.

19| We have left one complaint out on purpose: that the money never arrives. It is the most popular of them, it is the one most often stated wrongly, and it needs a chapter of its own.