Poor targeting: a response to Development Pathways’ paper on how best to reach those in poverty

The original version of this blog appeared on Development Pathways

Sometimes in life there are things that you know, instinctively, to be true; but you lack the proof with which to convince others. It is then particularly gratifying when the necessary proof emerges.

I have experienced just such a moment of gratification with the appearance of the Development Pathways’ paper on Targeting Effectiveness, "Hit and Miss"! I have always felt that poverty-targeting, in all its forms, is a fundamentally flawed approach to deliver social protection – see my other blogs on this site. The new paper provides incontrovertible proof.

In an ideal world, poverty-targeting would be sensible, which is why it intuitively appeals to those who are new to social protection, or who don’t understand the true complexities of poverty. I would even go so far as to accept that, if all of five conditions were met, there are circumstances in which poverty-targeting might be the optimal choice. But we don’t live in an ideal world, and the Development Pathways’ paper clearly demonstrates how very unlikely it is – in the real world – to fulfil each one of those five conditions, let alone to meet all five simultaneously!

For poverty-targeting to be the best option, all of the following five assumptions need to hold true:

• The poor represent a small residual group
• The poor will remain poor (and the non-poor will remain non-poor)
• Inequitable outcomes are acceptable
• The resource envelope is fixed
• You can accurately identify the poor

Let’s look at each of these in turn, in the light of the evidence presented in the Development Pathways’ paper.

1) The poor represent a small residual group

It only makes sense to undertake the expensive and complicated attempt to target the poor if they represent a relatively small portion of the population. And yet, paradoxically, it is in exactly these circumstances that poverty-targeting is most difficult, as the paper shows. In any case, it is absolutely clear from the paper that in all lower- and most middle-income countries, the number of those who can be considered poor by any comprehensive metric is high: usually in the region of 80 percent. It illustrates this with graphs of the wealth distribution in Brazil and South Africa, and with a telling figure showing incomes in purchasing power parity for five such countries:

In the USA, anyone living on less that US$10 a day would be considered extremely poor. If some 80 percent of the population is poor, it makes no sense to waste money on targeting: the cost of doing so would probably exceed the cost of including the extra 20 percent in the programme, and you would lose many of the political economy benefits of an inclusive universal approach. Conversely, if a much smaller proportion of the overall population is poor, then you still have a serious problem, because the paper shows clearly that poverty-targeting is less and less accurate the lower the coverage – see point 5 below.

2) The poor will remain poor (and the non-poor will remain non-poor)

Poverty is extremely dynamic. The paper gives two examples of such churn in and out of poverty, from Uganda and Indonesia:

These graphs show that in Uganda, only 46 percent of households that were in the poorest quintile in 2013 had been in the poorest quintile in 2011; and, in Indonesia, 38 percent of households in the poorest quintile in 2015 had not been in that quintile just one year earlier. As the paper makes clear, all forms of poverty-targeting would miss this: even the very best such programmes only undertake retargeting every three or four years, and in the vast majority the interval is between five and ten years (for instance, neither Pakistan nor the Philippines has updated its PMT since 2009). There is simply no extant example of a poverty-targeted approach that could capture such volatility in wealth.

3) Inequitable outcomes are acceptable

Even if it could, that would create another challenge. The main reason for such volatility is that, in most low- and middle-income countries, there is very little difference in the income (or consumption) of the poorest 80 percent of the population[1], so even a small change in income, caused by only a minor shock, can knock a household a long way down the wealth distribution. Conversely – and this is where the next challenge comes – making a cash grant to a selected poor household will inevitably propel it far up the wealth distribution, leap-frogging a number of almost equally poor households who are now poorer than the beneficiary household. And yet the beneficiary household is likely to go on receiving the same benefit for a number of years, while the now-poorer households get nothing. This is inequitable, and frequently causes the resentment, jealousy and social tension that are the inevitable by-products of poverty-targeting.

4) The resource envelope is fixed

If you assume that the fiscal space for social protection is fixed and immutable, then there is some justification in arguing that the limited resources should be focussed on the poor. As Devereux[2] argues when presenting the case for targeting, “Given the reality of budget constraints, scarce public resources must be used optimally and allocated efficiently, where they can achieve the maximum impact. If poverty reduction is an objective of public policy, social spending should be directed towards the poor who need income support, not spread thinly over the entire population including to those who do not need it”. But it is wrong to assume that poverty reduction is the only – or even the main – objective of social assistance, and it is naïve to think that budgets are immutable: investment in social protection is a political choice. If politicians spy electoral benefit, the funding will inevitably follow. As a result, the value of transfers is likely to be much higher with universal programmes than with poverty-targeted ones, meaning – ironically – that the poorest do better out of such inclusive programmes than they do when they are specifically targeted.

5) You can accurately identify the poor

Finally, we come to what the Development Pathways’ paper demonstrates to be the killer assumption: that you can accurately identify the poor. It is abundantly clear that you cannot…or at least that current approaches cannot. The paper systematically analyses national household survey datasets from 23 countries, and examines the targeting accuracy of 38 social protection schemes or targeting registries in those countries. It does this using two metrics of targeting effectiveness: (i) the proportion of households incorrectly excluded, when measured against the scheme’s intended coverage; and (ii) the degree of exclusion of the poorest 20 percent of households from the scheme. Both give results that can only be described as lamentable, especially when considering schemes that aim to reach 25 percent of the population or less (representing two-thirds of the programmes studied). In terms of exclusion against intended coverage, the very best performer amongst these has errors of 47 percent, while 12 such schemes have exclusion errors above 70 percent, 8 above 80 percent and 5 above 90 percent (meaning that fewer than one in ten of intended beneficiaries is actually selected). In terms of exclusion of the poorest 20 percent of households (which should logically all be included in a programme aimed at 25 percent of the population), the results are similar: the very best programme excludes 46 percent of the poorest quintile of households, while 12 schemes exclude more that 70 percent of the poorest wealth quintile, 9 exclude more than 80 percent and 4 exclude more than 90 percent – indeed, the worst-performing manages to correctly identify fewer than one in twenty of these poorest households!

The purpose of the paper is not to shame the worst performers, but to demonstrate that the whole approach is fatally flawed. Yet there is a remedy: to increase programme coverage and – ultimately – move to universality. The paper summarises this on a very instructive chart, showing the relationship between coverage and exclusion error:

The best-targeted programmes, using both metrics applied in the paper, are those which have higher coverage of their target populations. Thus, the best results using a PMT, though still not very good, are for the two such programmes where coverage exceeds 40 percent of the intended population: the old age component of Ecuador’s Bono Desarrollo Humano and Uruguay’s Asignaciones Familiares, both of which exclude “only” some 30 percent of their intended recipients and 17 percent of the poorest quintile of households. Similarly, the best example of community targeting, admittedly out of an alarmingly poor set of results, is Rwanda’s Ubudehe classification, covering 30 percent of the population, although this still excludes over half of those intended and half of the poorest quintile of households.

As coverage increases further, exclusion error decreases. South Africa’s Child Support and Old Age Grant, which are means-tested to exclude the most affluent, have over 70 percent coverage of their target populations: exclusion of the intended beneficiaries is only 13 percent and 8 percent respectively; and exclusion of the poorest quintile of households is zero in both cases – in other words every household in the poorest quintile is included, a remarkable achievement. As we move to the four universal programmes analysed in the paper, this finding is further reinforced. Bolivia’s Renta Dignidad and Bono Juacinto Pinto, Georgia’s Old Age Pension and Mongolia’s Child Money Programme all exclude fewer than one in ten of their intended beneficiaries and fewer than one in ten of the poorest households.

The bottom line is that poverty-targeting does not work. The best way to help the poor is not to target them, but to move to higher-coverage programmes, and ideally to universal life-course approaches! I know I have said this before…but now I feel I have the evidence to prove it! Thank you, Development Pathways!

---

[1] It is much easier to distinguish the wealthiest 20 percent in such cases, which is probably why the high-coverage, affluence-tested programmes studied in the paper – in particular South Africa’s Old Age and Child Support Grants – perform so well.

[2] Devereux, Stephen, 2016. Is targeting ethical? Global Social Policy, I-16.

Will no-one rid me of this troublesome PMT?

The original version of this blog appeared on Development Pathways

Anyone who has worked in social protection knows that the thorniest issue of all is that of “targeting”. The recent polemics on these pages about the inadequacy of the Proxy Means Test (PMT) as a “targeting” mechanism have raised the more fundamental question of whether it is in fact the actual concept of “targeting” that is at the heart of the problem.

Amartya Sen began his famous and oft-cited paper on “The Political Economy of Targeting” (1995) like this:

The use of the term “targeting” in eradicating poverty is based on an analogy–a target is something fired at. It is not altogether clear whether it is an appropriate analogy. The problem is not so much that the word “target” has combative association. This it does of course have, and the relationship it implies certainly seems more adversarial than supportive.

Sen also worried that “targeting” treats the recipient as an object rather than as a human being (“The image is one of a passive receiver rather than of an active agent”). And Stephen Kidd, in his “Pathways Perspective” on “Rethinking ‘Targeting’ in International Development” (2013) argued that:

The concept of viewing recipients of public services as “targets” to be hit corresponds well with a neoliberal concept of social policy, in which people receive “assistance” as a form of government charity. It is much less appropriate within a paradigm in which public services are regarded as “entitlements” offered to “citizens”.

These are all fair points. But there is a further objection to the term. The concept of “targeting” carries with it an implicit assumption that accuracy is possible: the supposition is that we are able to hit a target. But such accuracy is, of course, impossible when we are trying to assess the comparative poverty of different households or individuals. There is no accurate way to capture the multiple facets of relative poverty: where a household lives, what it comprises, the age and capabilities of its members, what it owns, what characteristics and skills it possesses, what social bonds it has, and so on. To imagine otherwise is delusional. We need to be honest, and explain to policy-makers that PMT is not a “targeting” mechanism, it is one of several possible “rationing” mechanisms…and not necessarily the best one.

The concept of targeting should properly be reserved for the policy choice of which vulnerable group is to be the recipient of government support, for example the elderly, children, pregnant women, the working age poor. Here, at least some degree of accuracy of identification is possible. It is rather at the next stage that the problems arise: there is a subsequent choice about whether all of this group, or just some of them, should receive the benefit. Much the best solution of course is to provide the assistance universally, or at least only exclude the very wealthiest, as in South Africa’s self-declared affluence testing. But this decision is usually a function of the resources available: if they are inadequate, then some form of rationing will need to be applied to restrict them to just a subset of the group.

There exist a number of options. The major ones (in broadly decreasing order of acceptability, from best to worst) include:

• Eligibility rationing – where eligibility criteria are highly restricted, for example by setting the eligibility age of a social pension very high (e.g. over 70, as in Lesotho), or of a child grant very low (e.g. under 2 years). The advantage of this approach is that it limits the numbers, but is nonetheless universal, which makes it popular, and which allows the eligibility criteria to be gradually expanded over time (as has happened for example with the Child Support Grant in South Africa).
• Geographical rationing – where specific geographic areas are isolated for receipt of the benefit. Examples of this approach would include support to the riverine chars of Bangladesh, or to specific ethnic groups in tribal areas of Vietnam. The problem is that this is sometimes politically difficult, since it may create or exacerbate regional jealousies.
• Random rationing – where beneficiaries are selected by lottery. This is the practice in a number of instances, for example on the SWAPNO programme in Bangladesh. It may seem a bizarre way to allocate social “entitlements”; but it has the advantage that people understand its arbitrariness, and it is at least honest and transparent.
• Community rationing – where communities themselves are asked to ration the benefits of the programme. One advantage of this approach is that community members frequently opt for an inclusive approach, and simply distribute the total amount of the benefits equally among everyone in the community (as for example in Indonesia’s Raskin programme): universalism through the back door. On the other hand, there can also be significant challenges, where selection may reflect existing patterns of social exclusion within communities, or resources may be captured by the more powerful.
• Temporal rationing – where groups of beneficiaries are selected to receive benefits for a short period (often 1 to 2 years), and are then removed from the programme and replaced by a new set of temporary beneficiaries, as occurs for example in Uzbekistan. This is the approach of so-called “graduation” programmes. It may be fair in that it rotates the benefits, but impacts are seriously constrained as a result.
• PMT rationing – where a fallacious veneer of objectivity and transparency is applied to lottery rationing, permitting the selection process to be attributed to a computer rather than to pure chance. There is nothing inherently wrong with this: what is unethical – as we have seen from the evidence presented recently on these pages – is the charade that PMT results in accurate “targeting”.
• Auction rationing – where potential recipients bid against each other for inclusion on a programme, as for example in Bangladesh, where some beneficiaries of the old age allowance have had to pay up-front bribes of up to one year’s worth of benefits to be included. The most common manifestation of such auction rationing is public works programmes, where “beneficiaries” bid their opportunity costs for the privilege of expending valuable calories on hard labour in exchange for a meagre transfer.
• Patronage rationing – where community leaders or other worthies select beneficiaries based on their patronage relationships. Examples of this would include the constituency funds allocated in some Pacific Island states to Members of Parliament, for them to distribute as social assistance to selected constituents. The approach is not “fair”, but it is at least understandable, and transparent in its unfairness.

There is nothing very scientific about the list above, and the order of preference is strictly personal! But it demonstrates that there are other methods than the awful PMT to cut the cake. It may be that if the use of PMT for poverty-“targeting” is presented to policy-makers for what it really is – i.e. as one of many options for rationing limited social resources – then they will pay greater attention to alternative approaches…most of which are actually much better!

Poxy Means Testing: it’s Official!

The original version of this blog appeared on Development Pathways

“A prox on both your houses” [i]

The World Bank has recently – and some would say belatedly – undertaken a critical review of the Proxy Means Test (PMT)[ii], the approach to targeting that it has been advocating, uncritically, for the past decade.

The results are astonishing. Disguised beneath a splendidly econometric veneer, the raw findings that emerge demonstrate that the PMT is a wholly unsatisfactory targeting mechanism. Based on rigorous analysis of PMTs in nine sub-Saharan African countries (Burkina Faso, Ethiopia, Ghana, Malawi, Mali, Niger, Nigeria, Tanzania and Uganda), it finds the following: when using “Ordinary Least Squares results for Basic PMT” (the most common PMT approach), with a fixed poverty line of 20% of the population, “On average, the rate of inclusion errors implies that 48% of those identified as poor by the Basic PMT method are in fact non-poor”; and “The average exclusion error is sizeable, with 81% of those who are in the poorest 20% in terms of survey-based consumption being incorrectly identified as non-poor by the PMT method”.

Let’s just stop and think about this. What this means is that, if a country is encouraged to establish a poverty-targeted social assistance programme targeting the poorest 20% of its population, then its policy-makers will need to accept two facts: that almost half of the actual beneficiaries of the programme would be from outside the intended sub-group; and that fully four out of every five households intended to benefit would in reality be excluded from the programme. What kind of policy-maker would accept such lamentable targeting performance? In Mali, incidentally, not one single ultra-poor household was correctly identified by the PMT as being ultra-poor: an exclusion rate of 100%!

The paper goes on to suggest that certain refinements can improve the accuracy of econometric targeting. But the improvements are small, and the necessary refinements range from being unlikely to being wholly impractical in reality. At the unlikely end of the scale, one suggestion is to increase the coverage of such poverty-targeted programmes to 40% of the population. Yet there are practically no examples of this in Africa, and the reality is that the vast majority of PMT-based programmes target even fewer than the poorest 20%. At a more common level of 10% coverage, the targeting errors are likely to be significantly higher, especially since, as the paper states: “econometric targeting may have difficulty in identifying those who are very poor” and “PMT is missing many of the poorest households in all countries”. At the wholly impractical end of the scale, the proposal is to run an “Expanded PMT” with “far more data”. But here the paper itself accepts that: (a) “the improvement would have to be judged as modest”; and (b) “the field implementation of a PMT formula with many variables is expensive and difficult”.

Remember too that these underwhelming reported outcomes reflect only the inherent statistical inaccuracy of the PMT approach. As other papers have emphasised[iii], the overall performance of a PMT will inevitably be further compromised by a range of other factors. Many of these are touched on, but not explored, in the World Bank paper. Actually implementing the complex and unintuitive PMT approach is bound to introduce further errors (as the paper coyly admits “Field implementation may introduce idiosyncratic mistakes”, and “Most likely the methods will perform less well than our calculations suggest.”). And there are still further problems: (a) with a PMT’s perverse incentives (e.g. households not wanting to acquire assets or improve their dwelling for fear of being excluded); (b) with moral hazard (e.g. households being encouraged to lie about their situation in order to qualify); (c) with the actual costs involved in the targeting process; and (d) with the damage to social cohesion of an improperly understood and seemingly arbitrary selection procedure. As the paper acknowledges, “We present and compare the best-case results for the various methods reviewed, unaffected by potentially differential costs, ease of implementation, and susceptibility to manipulation and corruption”.

One final reason for the PMT’s inaccuracy – which the paper does explore – is that of its inability to respond to the dynamics of poverty. There is always a degree of churning in and out of poverty, and a PMT is very static: most PMTs are only re-run every five to ten years. The paper looks at the implications of this on targeting accuracy by using panel data and running the analysis with lags of one to two years. This shows that – even with such a small lag – inclusion error increases from 48% to 55%, and exclusion error from 81% to 90%. On this basis (which would become still worse over a longer time-lag), we would now need to be telling our putative policy-maker that his or her poverty-targeted social assistance will consequently include more unintended than intended beneficiaries; and that nine out of ten of the intended beneficiaries will be excluded from the programme. This is crazy: imagine trying to persuade a policy-maker to adopt a criminal justice system that resulted in more than 50% of all jail inmates being innocent, and nine out of ten criminals being found not guilty!

So what are the alternatives to PMTs? Well, the paper helpfully goes on to explore some options. It looks at various permutations of simpler, more transparent and more intuitive targeting approaches, premised on a basic income transfer either to all, or to selected categories of, the population (children, the elderly, widowed, disabled or orphaned). It assumes the same overall budget for all the options (though it doesn’t allow for the additional costs involved in running a PMT), and it looks at the comparative poverty impacts of each. The verdict: “even under seemingly ideal conditions, the ‘high-tech’ solutions to the targeting problem with imperfect information do not do much better than age-old methods using state-contingent transfers or even simpler basic income schemes. We find that an especially simple demographic ‘scorecard’ method can do almost as well as econometric targeting in terms of the impacts on poverty. Indeed, on allowing for likely lags in implementing PMT, the simpler categorical targeting methods perform better on average in bringing down the current poverty rate. This conclusion would undoubtedly be strengthened once the full costs of fine targeting are taken into account”.

The paper thus demonstrates conclusively that, in terms of poverty reduction in the real world, PMT performs worse than simpler categorical approaches or even basic income schemes…as well as being administratively costly, morally reprehensible and socially divisive.

Hurrah! But why has this taken so long? And what are the implications for those countries that the World Bank has already persuaded to sign up to such an execrable model?

---

[i] To misquote Mercutio in “Romeo and Juliet” by William Shakespeare.

[ii] Brown, C, Ravallion, M and van de Walle, D (December 2016), “A Poor Means Test? Econometric Targeting in Africa”, World Bank Policy Research Working Paper 7915, Washington DC.

[iii] See for example, Kidd, S and Wylde, E (September 2011), “Targeting the Poorest: An assessment of the proxy means test methodology”, AusAID, Canberra; and Kidd, S, Gelders, B; Bailey-Athias, D (2017) Exclusion by design: An assessment of the effectiveness of the proxy means test poverty targeting mechanism, International Labour Office, Social Protection Department (SOCPRO), Geneva.

Poxy Means Testing

The original version of this blog appeared on Development Pathways

PMT is a curse! Sisters, you all know that: inescapable, debilitating, emotionally draining, a regular cause of extreme irritability!

But I refer here not to Pre-Menstrual Tension, but rather to a new form of PMT that is sweeping the globe: Proxy Means Testing.

This variant of PMT is a method of selecting poor people to become beneficiaries of social transfer programmes, currently being advocated strongly by, among others, my good friends at the World Bank. Proxy means tests generate a score for each applicant household, based on "fairly easy to observe characteristics of the household such as the location and quality of its dwelling, its ownership of durable goods, demographic structure of the household, and the education and, possibly, the occupations of adult members" (http://go.worldbank.org/SSMKS9WUT0).  The specific indicators used in calculating this score and their relative weights are derived from statistical analysis (usually regression or principal components analysis) of data from detailed household surveys.

PMT is touted as generating "impressive" results; it claims to be based on "statistically rigorous methods"; in Chile (where it all began), it exhibits an "excellent record" of targeting; in Fiji, it has been pushed as being "highly reliable"; in Jamaica, "leakage errors are less than 3 percent"; etc. As a result, the World Bank claims that PMT is "objective", that it has fewer disincentive effects than a true means test, and that it has been "proven to work particularly well in countries with high levels of informality and where personal and household income is difficult to verify with any degree of precision". Overall, the advocates of PMT paint a happy picture of a scientifically sound, technocratically robust and dispassionately objective solution to poverty targeting.

But a recent paper, Targeting the Poorest[i], suggests that the reality is very different, and cautions policymakers strongly against the dangers of being steamrollered into the adoption of PMT. It suggests that the PMT approach is demonstrably deficient in five main areas.

First, the datasets on which PMT is based are not fit for purpose.  The household surveys on which proxy means tests are modelled are designed to build an aggregate picture of poverty at national, regional and - less often - district levels; they are not appropriate for a detailed understanding of poverty in individual households. In addition to the intrinsic sampling errors, household surveys also suffer from substantial non-sampling errors, such as (i) lack of clarity on what constitutes a household, (ii) incomplete coverage, and (iii) the reticence of households to provide accurate information. These difficulties are further compounded by the fact that household surveys typically measure only consumption and expenditure, not income; that household surveys reflect only a single moment in time (often once every five or ten years), while poverty at household level is highly dynamic; and that data on asset-ownership (much used within PMT) is a reflection of past income, not of present income, which would tend to penalise, for example, older households, who have accumulated assets over a long period but whose current income is diminishing.

Second, PMT analysis can be unduly influenced by arbitrary statistical choices. The paper looks at three specific examples:

a)       How equivalence is calculated. Some analysts do not apply equivalence scales (ie they treat a child as having the same level of consumption as an adult), whereas others treat children up to 12 (or 14 or 16) as being equivalent to 0.5 (or 0.8) of an adult.

b)      How missing variables are interpreted. Inevitably there are missing data in household surveys; and analysts must decide whether these non-responses should be imputed or treated as absent. This decision impacts on the estimated value of the coefficients in the regression, and hence on the weights used in the PMT score.

c)       How sampling errors are treated. Household surveys only provide estimates based on a sample; their precision therefore falls within a range, at a given level of confidence. The paper tested two scenarios, one using the lower bound at the 95 percent confidence interval, and one using the upper bound.

The paper looks at the actual datasets for specific countries where PMT is used, and demonstrates that each one of these three sets of assumptions can arbitrarily change the eligibility status of some 10% of households. Cumulatively, this could mean that the eligibility of over 30% of households is determined not by its inherent poverty status, but by the statistical whim of an analyst.

Third, the regressions used in PMT do not provide sufficient clarity to distinguish between poor households. Clearly the choice of variables to be included in the PMT influences the outcome: by selecting only a subset of "fairly easy to observe" variables, the PMT model is inherently less able to reflect the same degree of variation as the more comprehensive list in the full consumption measure. Looking at examples in four countries where PMT is being used, the paper shows that PMT regressions typically only explain about 50% of the variation in consumption between households. What is worse is that they are particularly weak at the poorer end of the scale, thus making it especially difficult to distinguish between the poorest households. In other words, PMT "performs the weakest at the point where it would be expected to find the best correlation between assets and consumption". The paper's analysis of data from four countries (Bangladesh, Rwanda, Sri Lanka and Indonesia) shows that, depending on programme coverage, targeting error in PMT programmes is typically between 35% and 43% at a 30% coverage level, between 44% and 55% when 20% of the population is covered, and a staggering 57% to 71% at a (more common in reality) 10% coverage level.  Interestingly enough, a separate study in Pakistan, this one by the World Bank itself (Report No: 47288-PK, May 8, 2009), reported even higher exclusion errors: of 61% at 20% coverage and of 88% at 10% coverage. And remember that this is just the theoretical error of the PMT regressions: it will inevitably be further compounded by errors connected with the household data, with statistical analysis and with implementation.

Fourth, PMT faces significant challenges at implementation, and the paper cites a number of examples of this. There is the problem of finding the beneficiaries, using either a census or on-demand method, with examples of enumerators not wanting to enter urban slums because of security concerns; male enumerators being barred from  entering households where only females were present; evangelical families refusing to take part in the enumeration process; and nomadic groups, temporary migrants and remote communities being deliberately excluded. Another issue is the objectivity of enumerators, often with excessive demands placed on them, inadequate training and insufficient supervision: examples have been documented of corruption, inadequate time per interview, and deliberate changing of results where PMT was perceived to be wrong. Then there is the question of the verifiability of the indicators: assets can be hidden; ownership is hard to prove or disprove; education, occupation and even age can be falsified; indeed the fear that proxies may be easily manipulated has led advocates of PMT to suggest that proxies and weights should be kept secret - not exactly an advertisement for "transparency"! Other documented weaknesses include the fact that community verification is rarely effective; there is evidence in many countries of political interference; recertification is seldom sufficiently regular to capture the dynamics of poverty; and there is often no effective appeals mechanism - indeed there is an inherent irrationality in even introducing an appeals system against what is claimed to be a fair, objective and transparent system of selection. In this regard, it is notable that the World Bank’s social protection handbook (For Protection and Promotion: The Design and Implementation of Effective Safety Nets) clearly states: “Proxy means tests are most appropriately used where a country has reasonably high administrative capacity” ... which raises the question of why so many countries with relatively weak administrative systems (e.g. Pakistan, Kenya, Nepal, Fiji, Niger) are being encouraged to adopt the PMT methodology.

Finally, PMT does not avoid the social, moral, incentive or political costs of targeting. In terms of social costs, the paper cites qualitative research in Mexico, Nicaragua and Peru indicating that some community members ascribe the omission of poor households to luck or God’s will, describing the PMT methodology as similar to a lottery; the apparent unfairness of selection leads to feelings of despair, frustration, resentment, anger and envy, and there is evidence that this has resulted in a breakdown of community cohesion and even conflict. Morally, there is clearly the issue, as noted by Sen, that people may be rewarded for being deceitful and punished for being honest, which may in time corrode the fabric of society. Nor is it clear why incentive costs should be any less when using PMT: if potential beneficiaries are aware of the proxies, such as possession of animals or farm implements, they will be less likely to invest in them. And among the political costs of PMT is that poverty-targeted programmes, especially when perceived as arbitrary, tend to alienate the middle classes: evidence suggests that programmes using PMT never command as significant a share of GDP as, for example, universal programmes such as child grants and social pensions.

What do these deficiencies mean in practice? All in all, the paper finds that - despite all the grandiose claims of its proponents - PMT performs lamentably in targeting the poor.  It concludes that a striking finding of the analysis was the consistency of the magnitude of errors across countries, suggesting that such levels of error are to be expected using PMT methodologies as currently employed. It counsels policymakers to bear in mind "this combination of theoretical errors means a majority of eligible poor households may be permanently excluded from social grant benefits as a result of PMT scoring".

As I said, sisters, in another context, PMT appears to be "debilitating, emotionally draining, a regular cause of extreme irritability" ... but at least we can put a stop to this variant!

---

[i] AusAID, Targeting the Poorest: An assessment of the proxy means test methodology, September 2011