Most charity evaluation measures the wrong thing
This is Part 2 of 5 of the Giving With Intention series. Read Part 1 and Part 3
Singer’s argument, which I wrote about in the first post in this series, answers the whether question. It does not answer the how. Once I accepted that giving substantially was both morally sensible and practically required, I found myself facing a different and considerably harder problem: how do you know whether a charity is actually worth giving to? If I really wanted to ensure I am making as much impact with my donations as possible, how do I determine which causes, charities or institutions to focus on?
My first instinct was to look for external validation. Charity watchdogs exist precisely for this purpose: Charity Commission filings in the UK, Charity Navigator in the US, assorted rating schemes that purport to separate responsible organisations from careless ones. What I found was a methodology built almost entirely on the wrong question.
The overhead ratio is not a measure of impact
The dominant heuristic in mainstream charity evaluation is the overhead ratio: what percentage of income does an organisation pass through to programmes versus spending on administration, fundraising, and staff? Charities that exceed roughly 20 to 25% overhead are treated with suspicion. Those below it are treated as virtuous.
While this superficially looks like a sensible indicator to optimise for, it is a measure of organisational tidiness. It is not a measure of impact, and treating them as interchangeable produces real distortions.
A charity that spends 5% on overheads and delivers a programme with no evidence of effectiveness is worse than one that spends 30% on overheads to run a rigorously evaluated intervention that demonstrably works. The overhead figure tells you nothing about what the money achieves. It tells you something about how the accounts are structured.
There is a deeper problem. Overhead ratios actively penalise the activities that produce good outcomes: monitoring, evaluation, randomised controlled trials, and staff capable of designing interventions and interpreting evidence. A charity that invests in understanding whether it works is, under the overhead metric, a worse charity than one that spends all its money on inputs without ever asking about outputs. The incentive runs exactly backwards and effectively penalises investments into outcome optimisation.
It became very clear to me that treating overhead as the primary signal is clearly not helpful and I realised that quite early in the process. Finding a better approach took a bit longer.
What cost-effectiveness actually measures
The question I eventually settled on is simple to state: what does £1 buy, in terms of human welfare? Answering it requires having some mechanism to measure welfare, which is not a trivial challenge.
In practice, two units are commonly used, and they measure the same thing from opposite directions. A QALY (Quality-Adjusted Life Year) represents one year of full health gained; it is the standard unit in high-income healthcare systems, including the NHS. A DALY (Disability-Adjusted Life Year) represents one year of healthy life lost, either to premature death or to illness and disability; it is the standard unit in global health economics, used by the WHO and the research bodies that evaluate interventions in low-income settings. One averted DALY corresponds roughly to one QALY gained. The distinction matters in practice because the contexts in which each metric is used are quite different.
The UK’s National Institute for Health and Care Excellence (NICE) typically approves treatments costing up to £20,000 to £30,000 per QALY gained. This is roughly what UK society has decided a healthy year of life is worth in a domestic healthcare context. Naive idealists may find the idea that you can assign a monetary amount to a year of someone’s life somewhat repulsive but, in the real world, governments and organisations do this (and have to do it) all the time to make decisions about different policies.
Now contrast the number above with what follows. The Disease Control Priorities project, a large academic collaboration that evaluates interventions across low-income settings, estimates that many of the most effective global health programmes avert a DALY for well under $200. GiveWell, whose methodology I will come to, puts the cost per life saved at its top-rated charities in the range of $4,000 to $6,500, depending on geography and year. The implicit cost per DALY at that level is substantially lower than the NICE threshold, often by a factor of 50 or more.
This is not a rounding error. It is a structural consequence of two facts: disease burden is concentrated in low-income contexts, and intervention costs are structurally lower where wages and supply chains are different. The arithmetic does not change based on whether you find it comfortable. It’s clear that impact of one’s donation can be orders of magnitude larger if directed in the right way.
The scale of the asymmetry
Abstract multipliers are easy to discount so let’s look at concrete examples.
Guide Dogs for the Blind UK costs approximately £50,000 to train and place a single guide dog. That investment helps one person navigate the world with greater independence; the benefit is real and not trivial. By contrast, the cost of cataract surgery in a high-burden developing country context, through organisations such as Sightsavers, runs to roughly £25 to £50 per operation, restoring functional sight to a person who had lost it. At those prices, the same £50,000 would restore sight to somewhere between 1,000 and 2,000 people. Both interventions address visual impairment. The cost-effectiveness ratio between them is approximately three orders of magnitude.
The comparison is not a criticism of guide dog charities. It is an illustration of what geographic and economic context does to cost-effectiveness, independent of how well-run or well-intentioned an organisation is.
The same logic applies when comparing interventions across health contexts more broadly. Despite significant progress in recent decades, Malaria still kills around 600,000 people every year, many of them children. Long-lasting insecticide-treated bednets distributed in high-malaria-burden sub-Saharan Africa, at roughly $5 to $7 per net, prevent a substantial fraction of malaria deaths in the households where they are used. The cost per life saved, at scale, sits in the low thousands of dollars. Most charitable interventions operating in high-income countries cannot approach that ratio, not because they are doing less valuable work in absolute terms, but because the baseline cost structure of operating in those environments is entirely different.
This is the central asymmetry that evidence-based giving tries to navigate.
Counterfactual impact
Cost-effectiveness is necessary but not sufficient. A charity with a strong evidence base and favourable cost estimates still needs to pass a different question: Does my donation change what actually happens?
This is the counterfactual test. If a charity has more funds than it can productively deploy, or is delivering activities that would occur regardless of marginal donations, then additional money has no causal effect on outcomes. The organisation’s impact is genuine; my contribution to that impact is zero.
In practice, this means asking what GiveWell calls “room for more funding”: whether a charity can absorb additional donations, translate them into additional programme delivery, and do so within a reasonable timeframe. A charity with a funding gap and established operational capacity is a materially different bet from one that cannot deploy capital without degrading quality or accumulating reserves with no deployment plan.
I treat room for more funding as a filter, not a bonus. Evidence of effectiveness matters. So does evidence that my donation is adding to a distribution, not releasing pressure somewhere else.
GiveWell, Giving What We Can, and the case for independent evaluation
The most systematic attempt I know of to apply this framework at scale is GiveWell. Founded in 2007, it produces detailed cost-effectiveness analyses of a small number of charities that meet a high evidential bar. The methodology involves primary data, model-based projections, explicit uncertainty quantification, and published reasoning that can be audited and challenged. Their models are available in full; the assumptions are stated, not buried.
Giving What We Can (GWWC) takes a complementary approach. Originally a pledge community built around Toby Ord’s commitment to give 10% of lifetime income to effective charities, it has evolved into a broader recommendation platform that draws on multiple evaluators. Its Global Health and Wellbeing Fundcurrently allocates to GiveWell’s research, reflecting the degree to which GiveWell has become the reference standard in this domain. GWWC also maintains its own recommended charity list and extends into cause areas beyond global health, including animal welfare and global catastrophic risk.
Neither GiveWell nor GWWC is the only organisation doing this work. Founders Pledge produces comparable analysis for founders and high earners considering large donations. The Copenhagen Consensus has run systematic cost-effectiveness comparisons across a broader set of global priorities. For an individual donor making evidence-based decisions in global health, GiveWell remains the most thorough publicly available resource, but the ecosystem around it matters.
GiveWell’s current top charities, as of its most recent update, are the Against Malaria Foundation (bednet distribution in high-burden malaria regions), Malaria Consortium (seasonal malaria chemoprevention), Helen Keller International(vitamin A supplementation), and New Incentives (conditional cash transfers to increase childhood vaccination uptake in Nigeria). All operate in high-mortality, low-income contexts where the disease burden is concentrated and the cost per intervention is low.
A methodology that corrects itself
One feature of this approach that I find more compelling than any single cost-effectiveness estimate is the commitment to self-correction.
For instance, GiveWell maintains a public page titled “Our Mistakes,” updated regularly, that documents errors in research, modelling, and funding recommendations, along with what changed as a result. I find GiveWell’s epistemic honesty here genuinely compelling: this is not a public relations gesture, it is a live log of methodological revision and course correction. It’s impossible to never make a mistake, and having a robust mechanism that looks for errors and actively working to address them is a great way to maximise learning.
The most instructive example of an error made and subsequent course correction is No Lean Season. GiveWell recommended the programme, run by Evidence Action in Bangladesh, beginning in 2017. The intervention subsidised seasonal migration during the lean season, and two earlier randomised controlled trials had found significant effects on migration, income, and consumption. A third RCT, conducted in 2017 and reported to GiveWell in late 2018, found no statistically significant impact on migration. GiveWell removed the recommendation publicly in November 2018, published a detailed account of what the evidence showed and why it had updated, and Evidence Action shut the programme down in 2019.
This is the scientific method applied to philanthropy. The original recommendation was not made carelessly and naively. It was grounded in replicable evidence. And the removal of the charity was not defensive or delayed; it followed the evidence as soon as the evidence arrived. The whole process was documented publicly so donors could see the reasoning at every stage. That’s how it should be done.
Giving What We Can applies similar standards to its own recommendations, updating and revising as evaluator research changes. The output of both organisations at any given moment is not a definitive answer; it is the current best estimate, subject to revision.
I find this more credible, not less, than organisations that do not acknowledge errors. A methodology that cannot be falsified is not rigorous. One that actively invites falsification is.
What this framework does not answer
Running a charity through this framework, cost-effectiveness per DALY, evidence quality, counterfactual impact, room for more funding, changes the answer considerably from what instinct alone would produce. Most well-known UK (and indeed Western) charities perform poorly against it: not because they do bad work, but because they operate in high-income contexts where the cost of each intervention is structurally high. The charities that perform best are, almost without exception, absent from the fundraising solicitations in my inbox or in the mainstream public discourse.
This framework answers how to evaluate charities within a cause area. It is an approach that ought to be taken much more seriously by governments and public institutions alike, and it provides a rigorous (within what the available data allows) and systemic, self-correcting model for individuals to make informed decisions about their charitable giving. But it does not answer which cause areas deserve priority in the first place. Deciding how to weight global health against animal welfare against existential risk against climate change requires a different kind of reasoning, one that involves comparing not just cost-effectiveness but also scope, tractability, and how neglected a problem already is relative to its scale. That framework, and how I personally apply it, is the subject of the next post.
Giving with intention is a five-part series documenting how I think about charitable giving: the framework I use, how I evaluate charities, how I have structured my portfolio across cause areas, and what I have got wrong.