Poxy Means Testing

The original version of this blog appeared on Development Pathways

PMT is a curse! Sisters, you all know that: inescapable, debilitating, emotionally draining, a regular cause of extreme irritability!

But I refer here not to Pre-Menstrual Tension, but rather to a new form of PMT that is sweeping the globe: Proxy Means Testing.

This variant of PMT is a method of selecting poor people to become beneficiaries of social transfer programmes, currently being advocated strongly by, among others, my good friends at the World Bank. Proxy means tests generate a score for each applicant household, based on "fairly easy to observe characteristics of the household such as the location and quality of its dwelling, its ownership of durable goods, demographic structure of the household, and the education and, possibly, the occupations of adult members" (http://go.worldbank.org/SSMKS9WUT0).  The specific indicators used in calculating this score and their relative weights are derived from statistical analysis (usually regression or principal components analysis) of data from detailed household surveys.

PMT is touted as generating "impressive" results; it claims to be based on "statistically rigorous methods"; in Chile (where it all began), it exhibits an "excellent record" of targeting; in Fiji, it has been pushed as being "highly reliable"; in Jamaica, "leakage errors are less than 3 percent"; etc. As a result, the World Bank claims that PMT is "objective", that it has fewer disincentive effects than a true means test, and that it has been "proven to work particularly well in countries with high levels of informality and where personal and household income is difficult to verify with any degree of precision". Overall, the advocates of PMT paint a happy picture of a scientifically sound, technocratically robust and dispassionately objective solution to poverty targeting.

But a recent paper, Targeting the Poorest[i], suggests that the reality is very different, and cautions policymakers strongly against the dangers of being steamrollered into the adoption of PMT. It suggests that the PMT approach is demonstrably deficient in five main areas.

First, the datasets on which PMT is based are not fit for purpose.  The household surveys on which proxy means tests are modelled are designed to build an aggregate picture of poverty at national, regional and - less often - district levels; they are not appropriate for a detailed understanding of poverty in individual households. In addition to the intrinsic sampling errors, household surveys also suffer from substantial non-sampling errors, such as (i) lack of clarity on what constitutes a household, (ii) incomplete coverage, and (iii) the reticence of households to provide accurate information. These difficulties are further compounded by the fact that household surveys typically measure only consumption and expenditure, not income; that household surveys reflect only a single moment in time (often once every five or ten years), while poverty at household level is highly dynamic; and that data on asset-ownership (much used within PMT) is a reflection of past income, not of present income, which would tend to penalise, for example, older households, who have accumulated assets over a long period but whose current income is diminishing.

Second, PMT analysis can be unduly influenced by arbitrary statistical choices. The paper looks at three specific examples:

a)       How equivalence is calculated. Some analysts do not apply equivalence scales (ie they treat a child as having the same level of consumption as an adult), whereas others treat children up to 12 (or 14 or 16) as being equivalent to 0.5 (or 0.8) of an adult.

b)      How missing variables are interpreted. Inevitably there are missing data in household surveys; and analysts must decide whether these non-responses should be imputed or treated as absent. This decision impacts on the estimated value of the coefficients in the regression, and hence on the weights used in the PMT score.

c)       How sampling errors are treated. Household surveys only provide estimates based on a sample; their precision therefore falls within a range, at a given level of confidence. The paper tested two scenarios, one using the lower bound at the 95 percent confidence interval, and one using the upper bound.

The paper looks at the actual datasets for specific countries where PMT is used, and demonstrates that each one of these three sets of assumptions can arbitrarily change the eligibility status of some 10% of households. Cumulatively, this could mean that the eligibility of over 30% of households is determined not by its inherent poverty status, but by the statistical whim of an analyst.

Third, the regressions used in PMT do not provide sufficient clarity to distinguish between poor households. Clearly the choice of variables to be included in the PMT influences the outcome: by selecting only a subset of "fairly easy to observe" variables, the PMT model is inherently less able to reflect the same degree of variation as the more comprehensive list in the full consumption measure. Looking at examples in four countries where PMT is being used, the paper shows that PMT regressions typically only explain about 50% of the variation in consumption between households. What is worse is that they are particularly weak at the poorer end of the scale, thus making it especially difficult to distinguish between the poorest households. In other words, PMT "performs the weakest at the point where it would be expected to find the best correlation between assets and consumption". The paper's analysis of data from four countries (Bangladesh, Rwanda, Sri Lanka and Indonesia) shows that, depending on programme coverage, targeting error in PMT programmes is typically between 35% and 43% at a 30% coverage level, between 44% and 55% when 20% of the population is covered, and a staggering 57% to 71% at a (more common in reality) 10% coverage level.  Interestingly enough, a separate study in Pakistan, this one by the World Bank itself (Report No: 47288-PK, May 8, 2009), reported even higher exclusion errors: of 61% at 20% coverage and of 88% at 10% coverage. And remember that this is just the theoretical error of the PMT regressions: it will inevitably be further compounded by errors connected with the household data, with statistical analysis and with implementation.

Fourth, PMT faces significant challenges at implementation, and the paper cites a number of examples of this. There is the problem of finding the beneficiaries, using either a census or on-demand method, with examples of enumerators not wanting to enter urban slums because of security concerns; male enumerators being barred from  entering households where only females were present; evangelical families refusing to take part in the enumeration process; and nomadic groups, temporary migrants and remote communities being deliberately excluded. Another issue is the objectivity of enumerators, often with excessive demands placed on them, inadequate training and insufficient supervision: examples have been documented of corruption, inadequate time per interview, and deliberate changing of results where PMT was perceived to be wrong. Then there is the question of the verifiability of the indicators: assets can be hidden; ownership is hard to prove or disprove; education, occupation and even age can be falsified; indeed the fear that proxies may be easily manipulated has led advocates of PMT to suggest that proxies and weights should be kept secret - not exactly an advertisement for "transparency"! Other documented weaknesses include the fact that community verification is rarely effective; there is evidence in many countries of political interference; recertification is seldom sufficiently regular to capture the dynamics of poverty; and there is often no effective appeals mechanism - indeed there is an inherent irrationality in even introducing an appeals system against what is claimed to be a fair, objective and transparent system of selection. In this regard, it is notable that the World Bank’s social protection handbook (For Protection and Promotion: The Design and Implementation of Effective Safety Nets) clearly states: “Proxy means tests are most appropriately used where a country has reasonably high administrative capacity” ... which raises the question of why so many countries with relatively weak administrative systems (e.g. Pakistan, Kenya, Nepal, Fiji, Niger) are being encouraged to adopt the PMT methodology.

Finally, PMT does not avoid the social, moral, incentive or political costs of targeting. In terms of social costs, the paper cites qualitative research in Mexico, Nicaragua and Peru indicating that some community members ascribe the omission of poor households to luck or God’s will, describing the PMT methodology as similar to a lottery; the apparent unfairness of selection leads to feelings of despair, frustration, resentment, anger and envy, and there is evidence that this has resulted in a breakdown of community cohesion and even conflict. Morally, there is clearly the issue, as noted by Sen, that people may be rewarded for being deceitful and punished for being honest, which may in time corrode the fabric of society. Nor is it clear why incentive costs should be any less when using PMT: if potential beneficiaries are aware of the proxies, such as possession of animals or farm implements, they will be less likely to invest in them. And among the political costs of PMT is that poverty-targeted programmes, especially when perceived as arbitrary, tend to alienate the middle classes: evidence suggests that programmes using PMT never command as significant a share of GDP as, for example, universal programmes such as child grants and social pensions.

What do these deficiencies mean in practice? All in all, the paper finds that - despite all the grandiose claims of its proponents - PMT performs lamentably in targeting the poor.  It concludes that a striking finding of the analysis was the consistency of the magnitude of errors across countries, suggesting that such levels of error are to be expected using PMT methodologies as currently employed. It counsels policymakers to bear in mind "this combination of theoretical errors means a majority of eligible poor households may be permanently excluded from social grant benefits as a result of PMT scoring".

As I said, sisters, in another context, PMT appears to be "debilitating, emotionally draining, a regular cause of extreme irritability" ... but at least we can put a stop to this variant!

---

[i] AusAID, Targeting the Poorest: An assessment of the proxy means test methodology, September 2011

Urgent: World Bank safety net needed

The original version of this blog appeared on Development Pathways

Beginning in the early 1980s, the World Bank (with the International Monetary Fund) foisted structural adjustment programmes on a variety of poor countries around the developing world. These programmes, based on the forced application of new ideologies of liberalisation and privatisation, led to massive unemployment, human misery and deprivation. Jobs were lost in government service (which had to be shrunk drastically), in state marketing boards (as trade was thrown open to the private sector) and in public services (where utilisation fell as a result of the introduction of prohibitive access fees).

In partial (and inadequate) response to this, the World Bank adopted the idea of “safety nets” to help the main losers and minimise social unrest. In 1987, to cushion the adverse effects of structural adjustment programmes on the poor, the Bank helped Bolivia to establish a first Emergency Social Fund (ESF) aimed at providing emergency relief through creating temporary employment and improving income. The Bank’s commitment to “safety nets for the poor” was set out in the World Development Report 1990: Poverty. And, after Bolivia’s ESF, the wave of social funds and public works spread to more than 60 countries, throughout Latin America, Africa, Asia and – from the 1990s – eastern Europe. By 2014, a World Bank review[i] counted 94 countries with public works programmes, often cleverly rebranded as “productive safety nets” rather than workfare.

Starting in 2001 and learning from existing programmes in Brazil and Mexico, the Bank began to support Conditional Cash Transfer (CCT) programmes…in Colombia, Jamaica, Turkey, the Philippines, Indonesia, Kenya and elsewhere; the same recent review counted 64 countries with CCTs in 2014. The Bank still confusingly termed these programmes “safety nets”: which was odd, because one of the key arguments it used to get around its own restrictive bylaws on providing loans for direct cash transfers was that CCTs had human development as their main objective. For the Bank, therefore, safety nets gradually became synonymous with social assistance and social transfers, but with distinctively neo-liberal characteristics: small-scale, poverty-targeted (usually using proxy means testing), involving a labour requirement or compliance with human capital conditions, and often with an objective of “graduation”. And so they have remained, rolled out as a kind of blueprint in tens of countries around the world, with technical support from a veritable army of World Bank technical experts with specific skill-sets.

Until now! On 30 June 2015 – seemingly out of the blue – the President of the World Bank, Jim Yong Kim, threw his organisation’s full weight behind universal social protection. In a joint statement with the ILO, he recognised the opportunity “to make universal social protection a reality, for everyone, everywhere”; and he even explicitly recognised that countries might prefer not to adopt the Bank’s established safety net blueprint: “There are many paths towards universal social protection. It belongs to each country to choose its own, and to opt for the means and methods that best suit its circumstances”.

The concept note supporting this bold statement contains no allusion whatsoever to poverty-targeting or proxy means testing; it doesn’t include the word “conditions” (still less the non-word “conditionalities”); it makes only one brief passing reference to public works programmes; the concept of “graduation” is totally absent; and the term “safety net” doesn’t even get a mention. Rather, the concept note recognises that social protection “is a human right that everyone, as a member of society, should enjoy, including children, mothers, persons with disabilities, workers, older persons, migrants, indigenous peoples and minorities”. The Bank’s stated objective is now “to increase the number of countries that can provide universal social protection, supporting countries to design and implement universal and sustainable social protection systems”.

This is of course excellent news for the wider cause of global social protection. But there must be concerns that – like structural adjustment three decades back – the imposition of this new ideology may entail serious human costs. In particular, it risks creating swathes of unemployment among the massed ranks of the World Bank’s safety net experts. What is to become of all those worthy Bankers whose skills, acquired and nurtured over the intervening years, have suddenly become redundant? This would include, for example: the multiple trainers on its annual social safety nets training course; the proponents of public works programmes with their endless discussions over setting the optimal wage rate and calculating the net present value of assets generated; the arcane wizards of proxy means testing, debating the sensitivities of equivalence scales and the merits of ordinary least squares over quintile regression (while still delivering a highly inaccurate targeting methodology); the fanatics of graduation, endlessly reworking their formulas and indicators to exit people from safety nets as fast as possible; and the advocates of conditionality, devising ever more complex experiments to try to demonstrate that attaching conditions to social transfers actually makes any difference (despite the accumulating evidence that it does not). All these, incompatible as they are with rights-based universal social protection, are now apparently relics of the past, consigned – like the poor laws and workhouses before them  – to the dustbin of social policy history.

So how can we help these unfortunate souls now they are no longer useful to the World Bank? With their redundant qualifications and outmoded skills, we must nonetheless hope against hope that they will be able access an adequate safety net. In particular, we must trust:

• That having spent so many years devising and imposing non-human rights based safety nets, they themselves are not now denied their basic human right to social security.
• That the poverty-targeting approaches they have designed are sufficiently responsive to detect their plight rapidly (though unfortunately the next retargeting exercise is not scheduled for another three years and may well be delayed further).
• That the proxy means testing formulas they have devised are sensitive enough to identify them as being particularly worthy of support, ideally without too heavy a weighting against asset-ownership, previous income or educational level of household head.
• That their once leafy, yet now deprived, townships around DC will be geographically targeted for inclusion in a putative Safety Net for Indigent Professionals (SNIP) emergency response programme.
• That they are physically robust enough to toil for hours each day in the torrid heat of the Washington sun, in order to receive a derisory wage set tantalisingly below the statutory minimum.
• That they are located close enough to basic health facilities and public educational establishments that they and their children will be able to comply with any necessary “human capital co-responsibilities”.
• That the graduation criteria for programme support are not set so low that they will have to exit the safety net before they have had time to equip themselves with the new skills needed to re-engage with the labour market.

This promises to be a true test of safety nets!

---

[i] Oddly enough, The State of Social Safety Nets 2015 was published just a week after the President’s statement advocating universal social protection. Let us hope (a) that this was an unfortunate coincidence and not an early indication of dissent and recalcitrance among displaced World Bank staffers, and (b) that the hoped-for safety net is indeed up to the task of defusing any resultant internal unrest. We look forward already to the next edition of this flagship report, entitled, no doubt, The State of Universal Social Protection 2016!

The Ages of Man – Shakespeare and social protection through the life-course

The original version of this blog appeared as a Pathways Perspective on Development Pathways

Introduction

Readers of earlier Perspectives and blogs on this site (see, for example, The Seven Deadly Myths of Social Protection) will recognise that there are two opposing ideologies of social protection. The first is manifested in the neo-liberal “poor relief” approach:  poverty-targeted, conditional, and focused on ameliorating the symptoms of current deprivation. The second is represented by a more inclusive, universal approach that aims to tackle the fundamental causes of poverty, often based on addressing vulnerabilities through the life-course. Let us turn to William Shakespeare, the greatest writer in the English language, to help us understand the overwhelming advantages of the second approach.

In Act 2, Scene 7 of As You Like It, written in 1599 or 1600 at the apogee of his illustrious career, the “Bard of Avon” wrote a famous monologue setting out the seven ages of man. The “ages” are not always depicted in the most flattering light, as befits the nature of the melancholy Jaques who articulates them. But they nonetheless provide a useful framework for thinking about a life-course approach to social protection. So, in this paper, I have taken the Bard’s seven ages to examine vulnerabilities at each stage of the life-course and propose potential social protection interventions that may be used to mitigate them.

Three important messages about social protection already emerge from the opening lines of the soliloquy:

                                    All the world's a stage,
And all the men and women merely players:
They have their exits and their entrances;
And one man in his time plays many parts,
His acts being seven ages.

The first is the emphasis on “all the world” and the implication that social protection should be available to all: in other words, that it should be inclusive and provided universally. The second is the explicit focus on “all men and women”, underlining that social protection needs to be gendered to reflect the fact that men and women may experience different vulnerabilities requiring nuanced responses. And the third is that one man – and “man” is used throughout to denote “woman” as well – “plays many parts” in the course of his or her life and is therefore exposed to changing vulnerabilities and needs over time.

Infancy

                                      At first, the infant,
Mewling and puking in the nurse's arms.

Infancy is perhaps the stage of a person’s life when he or she is most in need of protection. Indeed, such protection should ideally start at conception, and should cover – at an absolute minimum – the “first 1000 days”. The vulnerabilities of the “mewling infant” would include: malnutrition, resulting in permanent physical stunting and reduced cognitive development; missed immunisation and growth monitoring; limited access to ante- and post-natal care; and the possible loss of parental care from bereavement or migration. Social protection responses to counter these vulnerabilities would include: maternity/paternity benefits and leave entitlements through social insurance; and a universal maternity grant, family allowance or child benefit funded by government – since this is such an important investment for the future – possibly in a way that incentivises attendance at birth counselling, facility-supervised birth, birth registration, regular growth monitoring, and vaccinations. They might also encompass access to childcare services for working mothers of young children – like the “nurse” in whose arms this particular Shakespearean infant is mewling – as argued in another of our Blogs.

School age

And then the whining school-boy, with his satchel
And shining morning face, creeping like snail
Unwillingly to school.

Key vulnerabilities of the “whining student” would include child labour, and the inability (for whatever reason) to access school or to have a satisfactory environment in which to study; malnutrition, which, whilst not having the same irreversible consequences as during infancy, can still impede growth, learning and mental development; and, again, the loss of parental care from bereavement or migration. Social assistance is important here, particularly in ensuring access to school (for girls in particular, but also for boys) and in optimising the quality of the learning environment. Instruments would include child grants – such as South Africa’s Child Support Grant – educational stipends, bursaries and school meals. Furthermore, pensions for the elderly would help those children whose parents have migrated or died, an example of how interventions at one stage of the life-course can be extremely beneficial at other stages.

Youth

And then the lover,
Sighing like furnace, with a woeful ballad
Made to his mistress' eyebrow.

The “woeful ballad” of challenges faced by young adolescents would include: a lack of adequate skills to enter the labour market and consequent unemployment or underemployment; the inability to access appropriate training; resultant feelings of inadequacy, alienation and frustration; the risk of being pressured into early marriage; and – for young girls – the dangers of early pregnancy and motherhood. Social assistance responses should include the availability of secondary and tertiary education stipends and support to access free technical vocational and educational training. Helping to keep children in education is one of the most effective methods of reducing teenage pregnancy and early marriage, but social legislation – together with investment in its enforcement – also has a role to play. Here too, it is worth noting that many of the problems of young people listed above could have been avoided through better social protection during infancy and school age.

Working age

Then a soldier,
Full of strange oaths and bearded like the pard,
Jealous in honour, sudden and quick in quarrel,
Seeking the bubble reputation
Even in the cannon's mouth.

For those in physically demanding employment, there are significant risks of work injury, sickness and invalidity. As Shakespeare cautions, however well things are going and however fast one’s “reputation” is rising, the “bubble” can always burst suddenly: the result is reduced income and sometimes dramatically diminished well-being for the household. Where such employment is in the formal sector, like our “bearded soldier” here, such risks should be covered by contributory social insurance schemes: unemployment benefit, work accident compensation, sickness benefit, and invalidity insurance. For those in the informal sector, there needs to be access to Government-financed social assistance in the form of grants for disability and chronic illness. Other gendered vulnerabilities at this stage would include domestic violence, demand for dowry payments, discrimination against women in the labour market, unavailability of childcare services, and the need to look after ageing parents. Many of these can be addressed through social assistance – such as child grants, family allowances and old age pensions – which allow others to care for children and parents so that mothers can return to employment (a further example of how interventions can have benefits across the life-course). Others – such as domestic violence, dowry and sexual discrimination – are best addressed by other policies (or a mix of policies).

Maturity

And then the justice,
In fair round belly with good capon lined,
With eyes severe and beard of formal cut,
Full of wise saws and modern instances;
And so he plays his part.

Maturity is depicted as a time of moderate comfort and in a relatively positive light (perhaps because Shakespeare was at this established stage when he wrote the play!). But during this phase of the life-course there are still risks, not only the continuing ones of accident, sickness, and invalidity but also, increasingly, of unemployment (or at least underemployment), and the inability to find new work. Key social protection responses will include the range of social insurance benefits mentioned above. But social assistance is also required for those outside the formal sector: this might take the form of income tax credits, direct income transfers, an employment guarantee scheme, public works opportunities, or retraining for new employment opportunities.

Old age

The sixth age shifts
Into the lean and slipper'd pantaloon,
With spectacles on nose and pouch on side,
His youthful hose, well saved, a world too wide
For his shrunk shank; and his big manly voice,
Turning again toward childish treble, pipes
And whistles in his sound.

The vulnerabilities faced in old age, as expressed so eloquently by Shakespeare, include increasing frailty (“his big manly voice, turning again toward childish treble”) and a decreasing ability to work (“his shrunk shank”), although it is worth remembering that many older people nonetheless continue to work: for example, some 50% of farmers globally are more than 60 years old. There is also the possible lack of care from family, and discrimination in areas such as accessing the labour force or obtaining credit. At this “sixth age” of the life-course, support to the “slipper’d pantaloon” should consist of an old age pension. Ideally this would be one with three tiers: a foundation tier of a universal non-contributory citizens’ pension; a mandated compulsory contributory old-age benefit funded through social insurance; and voluntary pensions provided by the private sector which would be funded from additional savings made at the stage of having a “fair round belly with good capon lined”. Having access to income allows the elderly to remain active contributors to social networks, rather than becoming dependent on others or – worse still – being excluded from family support as a result of being viewed as a burden.

Death

Last scene of all,
That ends this strange eventful history,
Is second childishness and mere oblivion,
Sans teeth, sans eyes, sans taste, sans everything.

The final stage of each individual’s “eventful history” is indeed “mere oblivion”. But even here social protection has a valuable role to play, in ensuring that the surviving members of the household are not left bereft, or “sans everything”. From a social insurance perspective, this would include death benefits, funeral insurance and survivors’ allowances. Informal systems include savings schemes and burial societies. And government assistance might include a social pension for widows who have not yet attained the qualifying age for an old age pension.

Conclusion

Shakespeare knew a thing or two about life and hardship, and about “shuffling off this mortal coil”. The unmitigated cynicism of Jaques in As You Like It is a constant reminder that grief, sorrow, suffering and death provide the inevitable counterpoint to all human joy and success. Through this melancholy mouthpiece, Shakespeare tells us more about the pre-eminence of the life-course approach to social protection than the majority of learned papers and policy documents on the subject. With that in mind, and in deference to those that persist with the alternate neo-liberal approach, let’s leave the last wise word to another splendid character in the same play, the court jester or “fool”, Touchstone:

The fool doth think he is wise, but the wise man knows himself to be a fool.

MCTs: Mislabelled Cash Transfers

The original version of this blog appeared as a Pathways Perspective on Development Pathways

A new sleight of prestidigitation has occurred in the arcane world of the wizards of conditionality!

As readers may be aware, these conditionistas have promoted four major experiments over the last few years, which they hoped would demonstrate, once and for all, the added value of imposing conditions on the recipients of social transfers. After a series of contested, controversial, but ultimately rather disappointing results from the first three experiments (in Malawi[i], Burkina Faso and Kenya), all hopes were pinned on the fourth experiment, the Tayssir programme in Morocco. This was a two-year pilot designed to increase student participation in primary school, in which independent researchers partnered with the Government of Morocco to evaluate whether imposing conditions increased the educational impact of the cash transfer. [It also had a secondary research purpose, to establish whether it made any difference if the transfer was made to the father or to the mother.]

As with the other three experiments, the methodology used represents what the World Bank’s magnum opus on CCTs[ii] describes as the gold standard:

Ideally, to disentangle the effect of conditions from the income effect inherent in the transfer, an experiment would be designed whereby a first group of households or villages receives a UCT, a second group receives a CCT, and a third group serves as a control group.

This is exactly what happened in Morocco, and the results have just been reported in a paper dated July 13, 2013[iii]. There were essentially two versions of the programme. In its unconditional arm, families with children of primary school age could receive transfers whether or not their children attended school, while, in its conditional arm, cash transfers were disbursed to parents of primary school-age children only on condition that their child did not miss school more than four times each month. [These two groups were further subdivided, to determine if the effectiveness of the transfers depended on the gender of the parent who received the transfer (the child's mother or father).]

Each school sector sampled for the study was randomly assigned to one of five groups:

·       UCT issued to fathers: 80 communities from 40 school sectors.

·       UCT issued to mothers: 80 communities from 40 school sectors.

·       CCT issued to fathers: 180 communities from 90 school sectors.

·       CCT issued to mothers: 178 communities from 89 school sectors.

·       Comparison group: 118 communities from 59 school sectors received no transfers.

Researchers collected information on student attendance and enrolment status for over 47,000 primary school aged children through unannounced visits to all schools. Comprehensive baseline and endline surveys gathered data on around 4,400 households. At endline, a basic test was administered to measure the arithmetic performance of one child per household.

And the result? Overall the impact of providing the cash transfer was impressive: “The Tayssir cash transfers greatly increased school participation under all versions of the program, with the UCT having slightly larger impacts. After two years, the dropout rate among students enrolled in school at the start of the program in UCT schools was about 7 percentage points lower than the dropout rate in comparison schools (at 10 percent), a 70 percent decrease. Re-enrollment of those who had dropped out of school before the program almost doubled in UCT schools as compared to comparison schools, and the share of students who never enrolled in school fell by 43 percent. Performance on a basic arithmetic test improved but not significantly.” [iv]

All well and good. But what about the key question the experiment was meant to be answering, to which there is the tantalising reference underlined above? Yes, indeed. The findings continue: “Making cash transfers conditional did not improve the effectiveness of the program, but may actually have somewhat reduced it: relative to UCT schools, CCT schools had a slightly higher drop-out rate.” Furthermore, “While the CCTs also had a large positive effect on school participation, explicitly conditioning transfers on attendance significantly decreased their impact in the context of this program. In particular, relative to UCTs, CCTs lowered the impact on re-enrollment of children who had dropped out, perhaps because conditionality discouraged some households (or some teachers) from enrolling weaker children in the program. Correspondingly, CCTs also had a significantly lower impact than UCTs on math scores (CCTs had no impact whatsoever, with negative point estimates).”[v]

Oh dear! Not at all what the conditionistas would have wanted to hear! But because the research had been conducted in partnership with an independent institution, it appears that it wasn’t possible, on this occasion, to revisit the data or redo the analysis (as had happened in the case of Malawi). So the conditionistas did the only thing that was available to them: they invented a new category of cash transfer, a Labelled Cash Transfer (or LCT). This is essentially a UCT that has a message attached, a UCT which tells you that education is good for your child, a UCT for which you need to sign up on school premises. The report suggests that this makes an LCT different from a UCT, and implies that an LCT lies somewhere midway along the spectrum between a UCT and a CCT.

What this means, conveniently for the conditionistas, is that all those uncomfortable findings about a UCT outperforming a CCT can be made to disappear like a rabbit in a hat. The report is explicit about this:

“Note that the comparison between LCTs and CCTs tells us little about the question that other papers in the literature have addressed, namely how an unconditional and unlabeled cash transfer program would compare to a CCT.”

Gobbledygook!! This argument requires that an LCT is not the same as a UCT. Is this tenable? Of course not! A UCT does not become “more conditional” (and thus turn into a new beast, an LCT) just because it has a message attached. A condition is a condition: it is, by definition, a requirement that must be fulfilled in order to receive the transfer, where you are punished if you do not fulfil that condition. If a program does not have that (and the LCT arm in Morocco does not), then it is “unconditional”, pure and simple. Think of this in different context: if a driver has a license, he or she is a licensed driver; if not, he or she is an unlicensed driver. Or another: if a researcher adheres to a set of principles, he or she is a principled researched; if not, he or she is an unprincipled researcher. Ergo, an LCT is a UCT, but just one that places greater emphasis on its messaging. And nobody has ever suggested that such messaging is not a very important component of any cash transfer programme.

What is interesting is where the impetus came from, to mis-label UCTs in this manner. One would assume that it did not come internally, from the authors of the research paper. This is because, intriguingly, there exists an earlier version of their report, dated December 4, 2012. And in this version of the report there is no mention whatsoever of an LCT, the words “label”/“labelled” do not appear one single time, the treatment design is unequivocally defined as “conditional vs. unconditional”, and the respective target groups in the experiment are explicitly described as “UCT to mothers” and “UCT to fathers”. Mysterious indeed that LCTs have reared their ugly heads in the space of just seven months. At whose behest? I think we should be told!

Whatever the answer, there are some key messages that emerge from this important study, and they have nothing to do with Labelled Cash Transfers. These are that, in the case of Morocco:

·       Even small cash transfers have a hugely beneficial impact on schooling

·       Applying conditions has a negative impact on programme performance, whilst at the same time increasing both cost and complexity

·       It makes no difference whether the transfer is given to the father or the mother

·       The existence of an effective information and education campaign increases the impact of the transfers.

These findings have serious implications for policy, and should not be obscured by the chicanery of an invented and unnecessary new category of cash transfer. Please may the authors recognise this, and rewrite their report along the lines of their earlier draft, thus proving that they are principled researchers rather than “labelled researchers”! Otherwise there is a danger of not seeing the Bretton Woods for the LCTs.

---

[i] For a critique of the Malawi experiment, see: Kidd and Calder (2012) The Zomba Conditional Cash Transfer Experiment: an assessment of its methodology. Pathways Perspective No. 6, September 2012.

[ii] Fiszbein, A., and Schady, N., “Conditional Cash Transfers: reducing Present and Future Poverty”, World Bank Policy Research Report, 2009.

[iii] Turning a Shove into a Nudge? A “Labeled Cash Transfer” for Education by Najy Benhassine, Florencia Devoto, Esther Duflo, Pascaline Dupas and Victor Pouliquen.

[iv] NB In these quotations from the report, I have replaced the misleading term “LCT” with the correct term “UCT” – for the reasons explained in the subsequent paragraphs.

[v] Ditto.

The Seven Deadly Myths of Social Protection

The original version of this blog appeared as a Pathways Perspective on Development Pathways

In a stimulating blog, Stephen Kidd discussed the issue of myths in social protection. In this article, I would like to take this further and try to dispel a number of the more common myths surrounding social security systems in developing countries.

Coincidentally, I have just returned from a trip to Madrid. Naturally I visited the Prado museum, where one of the highlights was a splendid table attributed to Hieronymus Bosch, called The Seven Deadly Sins and the Four Last Things.  Here it is:

It occurred to me that there are some interesting parallels between the myths I want to dispel, and the Seven Deadly Sins depicted.

But first, by way of introduction, let us start by clarifying the two very distinct ideologies around social protection that Stephen has delineated.

On the one hand, we have a neo-liberal, small state approach – let us call it Tea Party social protection in memory of Boston, and the resultant philosophy of low taxation and low government expenditure. This approach is firmly rooted in a nineteenth-century “Anglo-Saxon” worldview of “poor relief” and “workhouses”. It is tightly poverty-targeted, based on the argument that scant resources should be focused on the poorest, often identified through (highly inaccurate) mechanisms such as proxy means testing, community targeting, or self-selection.  It is also premised on a belief that “beneficiaries” should do something in exchange for receiving their benefits. So it imposes conditions: either work (as on a workfare programme), or a set of behaviours (such as visiting a health clinic or sending a child to school). There is an emphasis on – not to say obsession with – “graduation” and “exit strategies”; and there is no recognition of entitlement or rights. Examples abound: Latin America’s much-lauded Conditional Cash Transfers (CCT); Ethiopia’s Productive Safety Net Programme (PSNP); Indonesia’s Program Keluarga Harapan (PKH); Pakistan’s Benazir Income Support Programme (BISP); etc. The major proponents of this ideology are the Bretton Woods institutions.

On the other hand, we have a more “universalist” approach, akin to a Nordic view of social security. This places the emphasis on tackling inequality as a means of combating poverty. It tends to provide support to much broader “vulnerable groups” such as people with disabilities, the elderly and children. And it provides benefits to all (or almost all) in these identified groups, even those that are not poor. It represents a more inclusive, so more expensive approach; but it is also more popular, with stronger political appeal, so ultimately better funded and more fiscally sustainable. The argument is that the poor get a more valuable (and more sustainable) transfer than if a much smaller programme had been targeted exclusively at them. These programmes tend not to be conditional, but entitlement-based; and there is much less emphasis on graduation, since exiting happens naturally. Again there are many examples: South Africa’s suite of social grants; old age pensions in Lesotho, Nepal and Thailand; India’s employment guarantee scheme (MGNREGA). The major global proponents of this approach are the United Nations, embodied in the concept of a social protection floor.

As Stephen argues so cogently in his blog, it is – regrettably – the former ideology that is currently predominant in the development discourse. But it is predominant largely because it is based on a set of myths. And it is these myths that I think can be linked to Hieronymus Bosch’s Seven Deadly Sins. They might thus be termed the Seven Deadly Myths of Social Protection.

The first is the Sin of Sloth (accidia in the Latin of the Catholic Catechism), which is linked to the myth propagated by protagonists of the Tea Party approach that providing people with social transfers will result in laziness and dependency. Whilst this may indeed be a concern in developed countries, as current debates in the UK show, it is demonstrably not the case in developing countries where benefit levels tend to be low. In fact all the evidence points to the opposite conclusion: that social transfers, by mitigating risk and allowing investment, actually reduce dependency and improve productivity. Recipients of social transfers in Mexico and South Africa look for work more intensively and extensively, and find employment more successfully, than do workers in comparable households that do not receive social grants; an evaluation of the Mchinji cash transfer programme in Malawi found that 50% of recipients were more likely to produce crops since receiving the cash transfer; and beneficiaries of the Mexican agricultural support programme, Procampo, raised their income by 1.5 to 2.6 times the value of the actual transfer.

The second is the old and much-repeated myth that social transfers encourage irresponsible spending – in other words that they encourage the Sin of Lust (luxuria). Quite apart from being patronising and hypocritical, this assertion is also demonstrably wrong. Poor households are by far the best judges of how to use effectively any resources they are given; and a number of studies have confirmed that they spend their transfers wisely: on food, on health, on education, on productive investments … not on sex, alcohol and cigarettes. Recipients of the old age pension in Lesotho spent less on all such “luxuries” combined than they did in contributions to the church collection-plate! And a study in South Africa showed that households that receive social pensions have higher expenditure shares on food and education and lower expenditure shares on alcohol, tobacco and entertainment, than other households do. As the t-shirts distributed to recipients of the Kalomo social cash transfer in Zambia proudly proclaim: “the poor are not irresponsible”!

The third myth of the Tea Partiers is linked to the Sin of Envy (invidia). Described by Aquinas as “sorrow for another's good”, this is manifested in social protection as a reluctance to “give something for nothing”. This argument, linked to the ones above, is as commonly heard among politicians and commentators in OECD countries as it is in developing countries. It is often used to justify “productive safety nets” (such as workfare) or conditional schemes as a basis for social transfers. But the “productivity” of such schemes, sometimes imposed with minimal consultation by external agents such as governments and donors, is often far less productive than the alternative of providing individuals and households with the means to make their own consumption and investment decisions, without obliging them to waste valuable time and energy on misguided and frequently “unproductive” initiatives.

The fourth argument used to support tightly poverty-targeted social protection programmes may be associated with the Sin of Greed (avaricia). This is the argument that any broader, more universal approach is “unaffordable”. But this is arrant nonsense. Affordability is much more closely associated with political will than with fiscal resources. First, the OECD countries which now rely heavily on social security were themselves much poorer when they introduced social security. Arguably this is indeed part of the reason why they have now become so much richer: there is a strong correlation between national wealth and the amount countries spend on social protection. Another example is Mauritius, where even the IMF cites its early adoption of comprehensive social transfers as a key reason for its dramatic improvements in living standards. Finally, ILO studies of selected African countries calculate that the cost of an $18 per month pension for all over-65s and all people with disabilities would typically represent no more than 0.3% to 1.0% of GDP. Lesotho, for example, introduced a universal social pension which was denounced by the Bretton Woods Institutes as unaffordable…but has gone on increasing its value ever since.

Fifth, and closely linked, is the Sin of Gluttony (gula).  Interestingly, gluttony is defined both as “a misplaced desire of food or its withholding from the needy”, with the latter being most relevant in this context. What those who argue that social protection is unaffordable are actually saying is that they do not accept a greater degree of redistribution: they do not wish current government expenditure from which they may benefit to be redirected to others. The typical example here is encapsulated in consumer subsidy programmes, which by definition benefit those who consume more, to a much greater extent than those who consume less. So, for example, 55% of the benefit of Indonesia’s fuel subsidy goes to the richest 20%, and less than 5% of the benefit to the poorest 20%. And this fuel subsidy costs the government 2.5% of GDP, compared with the 0.5% of GDP that it spends on social assistance programmes. As Galbraith put it so succinctly, quoted in one of the links on Stephen’s blog: “The modern conservative is engaged in one of man's oldest exercises in moral philosophy: that is, the search for a superior moral justification for selfishness”.

Sixth is the myth that grievance systems can compensate for flawed programme design and substandard implementation. This is linked to the Sin of Wrath (ira) and the mistaken assumption made by Tea Partiers that beneficiaries will demonstrate their anger and frustration through appeals procedures. But there is little evidence to support this myth. Grievance systems are often non-existent. Where they do exist, beneficiaries do not use them because they do not understand the basis on which they were, or were not, selected for a programme; they are not fully aware of their entitlement; or they see the whole process as a lottery where the benefit may be taken away as inexplicably as it came…especially if they are so brazen as to complain about any aspect of it. Interestingly, in one or two cases where appeals procedures were properly introduced, such as Kenya’s OVC programme, they had to be retracted because so many angry citizens protested that the system was overwhelmed.

The last, and most egregious, sin of the Tea Party movement is the hubris inherent in the myth that it is possible to target the poorest in any but the most arbitrary fashion. It is not. This represents the Sin of Pride (superbia). Often considered the most serious of the Seven Deadly Sins, Dante defined pride as “love of self, perverted to hatred and contempt for one's neighbour”. So we see advocates of the poverty-targeted approach establishing a range of inherently inaccurate systems to “identify” their poorer brethren; and then passing these targeting approaches off as being transparent, reliable and authoritative. Yet there are serious, well-documented flaws in all the main poverty-targeting methods: proxy means testing exhibits error rates of 60%-70%, especially in programmes with low coverage (the norm in developing countries); community-based targeting is as likely to perpetuate entrenched inequalities as to overturn them; and self-targeting is often either demeaning (for example when it relies on the provision of inferior goods) or distorted (for example when it excludes those without labour capacity, often the very poorest, from workfare).

So, we are faced with a stark choice, represented on Bosch’s table by the “Four Last Things”. In the first of these we see angels and devils weighing a man’s soul: think of this as being an allegory for the decision we have to take between perpetuating the Seven Deadly Myths of social protection, or refuting them.

We need to get the right answer, because the next panel shows the Last Judgement:

This choice we make will determine whether we end up with social protection systems in developing countries that are worthy of hell (one of Bosch’s specialisms!)…

…or of heaven.

And, be warned: as the good Hieronymus is keen to point out in the very centre of his table, Cave Cave Deus Videt ("Beware, Beware, God Sees"). So let’s refute those Tea Party myths!

Re: silly-ence, or the reinvention of vulnerability

The original version of this blog appeared as a Pathways Perspective on Development Pathways

“Resilience” is decidedly the development flavour of the month! Everybody’s doing it: it’s one of the three pillars in the World Bank’s “Social Protection and Labor Strategy 2012-2022”; the European Commission has put out a whole new Communication devoted to it; DFID has committed “to embed resilience-building in all DFID country programmes by 2015”; WFP (never one to miss a bandwagon) has a “Resilience Project” with the Swiss; and USAID celebrated a “Resilience Week” in April 2012. Academia is responding with delight: trenchantly erudite papers have appeared in the last few months with titles such as “The Relevance of ‘Resilience’?” (from the Humanitarian Policy Group), “Resilience: New Utopia or New Tyranny?” (from the Institute of Development Studies), “The Resilience Renaissance?” (also from IDS), “Resilience, a Risk Management Approach” (from the Overseas Development Institute), and “The Characteristics of Resilience Building” (from the Interagency Resilience Working Group, no less).

But what does it all signify? Does it mean anything? Does “resilience” advance the cause of development in any significant way? Probably not.

Resilience is no more and no less than the developmental antonym of “vulnerability”, a concept which has been around for a number of decades. Resilience, like vulnerability, has two dimensions: one internal and one external. You can increase resilience either by enhancing the inherent ability of an individual, a household, a community, a system or a country to withstand a shock, or you can act externally to reduce the potential for damage from that shock. Resilience can be increased, in just the same way as vulnerability can be reduced, in either of these two dimensions: you can strengthen an entity’s capacity to resist or you can reduce the likelihood of exogenous damage.

Think of driving a car on a busy motorway. There is always the risk of an accident, which could result in death or serious injury. There are two ways to protect against the damage that this might cause. In one dimension you can strengthen the capacity of the car’s occupants to withstand an accident: you can surround all passengers with airbags; you can legislate that all of them wear seatbelts; you can install revolutionary technology such as anti-lock braking systems (ABS) and traction control systems (TCS); you can capacitate the driver to drive better; and you can enforce adherence to the highway code. In the other dimension you can set up systems to implement variable speed limits according to weather conditions or traffic flow; you can position police and emergency services at strategic positions along the motorway; you can train ambulance crews to deal better with accident victims; you can educate the general population how best to react if involved in a collision. Through addressing both dimensions, you are significantly reducing vulnerability…and increasing resilience…to serious harm through a traffic accident.

One of the recent arguments in favour of resilience as a guiding development paradigm proposes that it assembles a broader church of practitioners than vulnerability: from climate change, ecology, disaster management and social protection. But much the same was said of vulnerability a couple of decades back. In practice (sadly) it may be true that different communities of practice still work in technical silos, but in theory both vulnerability and resilience are as broad or as narrow as you choose to make them: both are equally able to encompass climate-related factors, social dynamics, ecosystems, and so on…or not. Simply shifting the standpoint from vulnerability to resilience changes nothing.

Another paper sets up an assessment matrix to show how a commonly-used framework of social protection interventions to reduce vulnerability (with objectives of provision, prevention, promotion and transformation) maps to a suggested framework of resilience (with outcomes of coping/rehabilitating, adapting and transforming). The paper seems pleasantly surprised that the mapping works well. But if you accept that the two concepts are antonyms, then this is less surprising: it is like saying that a framework of increasing brightness maps well to a framework of decreasing darkness.

What is perhaps more useful, in linking social protection to the excitingly fashionable resilience model, is to consider how these four commonly accepted functions of social protection address different aspects of vulnerability to increase resilience. If we start from a definition of vulnerability as being represented by an equation: vulnerability = poverty + risk – agency – voice, then we can see how the four functions of social protection[i] act on each of the constants in the equation:

·       Poverty is reduced through the provision function of social protection, represented by social assistance (such as social pensions, child grants or disability benefits) – this may be likened to the provision of airbags and seatbelts in the motorway example above.

·       Risk is reduced through the preventive function of social protection, encompassing social insurance mechanisms (such as unemployment pay, burial societies and health insurance) – these are represented by the availability of ABS and TCS.

·       Agency is increased through the promotive function of social protection, comprising the kind of social empowerment embodied in asset transfer programmes, cash-for-training and (well-designed) employment schemes – equipping the driver to drive better.

·       Voice is increased through the transformative function of social protection, including interventions in the area of social justice, aimed at reducing discrimination, fulfilling rights, and so on – in our example, the standard application of the highway code, driving tests and driver education efforts.

At the same time, the motorway analogy also serves to underline that social protection is only one of many inter-dependent strategies that should be used to reduce vulnerability (and to increase resilience). Others might include interventions around:

·       Risk assessment, to anticipate the dangers and have plans in place to react to them – such as the ability to communicate weather and traffic information in our motorway scenario.

·       Early warning, such as tsunami sensors, earthquake detectors or drought monitoring systems – or the updating of weather/traffic warnings alongside the motorway in this scenario.

·       Pre-provisioning, in the form of emergency stockpiles or strategic grain reserves – or the pre-positioning of police and ambulance crews in our example.

·       Health services, so that they strengthen people’s ability to withstand shocks – or, in our case, to correctly treat accident victims.

·       Education and training to raise awareness and increase people’s agency – or, in this instance, to persuade people to drive more responsibly and to respond effectively in case of an accident.

But the point is that to the same degree as these coordinated interventions reduce vulnerability, so they also increase resilience. Indeed they would suggest that a correspondingly accurate (and useful) equation for resilience would be: resilience = assets – risk + agency + voice.

The danger here is that resilience does not really advance the conceptual debate. It’s as if development partners are searching for a new buzzword around which to rally, which the poverty academics are all too ready to supply. The literature of resilience is thus self-inflating, and much of it self-defeating as it revolves not around the legitimate application of the word itself but around the baggage that other writers have loaded onto it.

As a parting concession to the resilience lobby, however, there is a valid argument that framing objectives as positives instead of negatives may be psychologically preferable: on that basis alone, it may be better to rally the development community around “increasing resilience” rather than around “reducing vulnerability”. To that extent, resilience may be a useful hanger; but to pretend it is anything more than that is, well, just plain silly!

---

[i] Not everyone accepts this P-P-P-T framework of the functions of social protection, nor that the different types of social protection intervention can be mapped directly to a single individual function (eg a pension can be used for provision, prevention and promotion). But it is widely used, and serves as a useful structure for grouping these different interventions.