MCTs: Mislabelled Cash Transfers

The original version of this blog appeared as a Pathways Perspective on Development Pathways

A new sleight of prestidigitation has occurred in the arcane world of the wizards of conditionality!

As readers may be aware, these conditionistas have promoted four major experiments over the last few years, which they hoped would demonstrate, once and for all, the added value of imposing conditions on the recipients of social transfers. After a series of contested, controversial, but ultimately rather disappointing results from the first three experiments (in Malawi[i], Burkina Faso and Kenya), all hopes were pinned on the fourth experiment, the Tayssir programme in Morocco. This was a two-year pilot designed to increase student participation in primary school, in which independent researchers partnered with the Government of Morocco to evaluate whether imposing conditions increased the educational impact of the cash transfer. [It also had a secondary research purpose, to establish whether it made any difference if the transfer was made to the father or to the mother.]

As with the other three experiments, the methodology used represents what the World Bank’s magnum opus on CCTs[ii] describes as the gold standard:

Ideally, to disentangle the effect of conditions from the income effect inherent in the transfer, an experiment would be designed whereby a first group of households or villages receives a UCT, a second group receives a CCT, and a third group serves as a control group.

This is exactly what happened in Morocco, and the results have just been reported in a paper dated July 13, 2013[iii]. There were essentially two versions of the programme. In its unconditional arm, families with children of primary school age could receive transfers whether or not their children attended school, while, in its conditional arm, cash transfers were disbursed to parents of primary school-age children only on condition that their child did not miss school more than four times each month. [These two groups were further subdivided, to determine if the effectiveness of the transfers depended on the gender of the parent who received the transfer (the child's mother or father).]

Each school sector sampled for the study was randomly assigned to one of five groups:

·       UCT issued to fathers: 80 communities from 40 school sectors.

·       UCT issued to mothers: 80 communities from 40 school sectors.

·       CCT issued to fathers: 180 communities from 90 school sectors.

·       CCT issued to mothers: 178 communities from 89 school sectors.

·       Comparison group: 118 communities from 59 school sectors received no transfers.

Researchers collected information on student attendance and enrolment status for over 47,000 primary school aged children through unannounced visits to all schools. Comprehensive baseline and endline surveys gathered data on around 4,400 households. At endline, a basic test was administered to measure the arithmetic performance of one child per household.

And the result? Overall the impact of providing the cash transfer was impressive: “The Tayssir cash transfers greatly increased school participation under all versions of the program, with the UCT having slightly larger impacts. After two years, the dropout rate among students enrolled in school at the start of the program in UCT schools was about 7 percentage points lower than the dropout rate in comparison schools (at 10 percent), a 70 percent decrease. Re-enrollment of those who had dropped out of school before the program almost doubled in UCT schools as compared to comparison schools, and the share of students who never enrolled in school fell by 43 percent. Performance on a basic arithmetic test improved but not significantly.” [iv]

All well and good. But what about the key question the experiment was meant to be answering, to which there is the tantalising reference underlined above? Yes, indeed. The findings continue: “Making cash transfers conditional did not improve the effectiveness of the program, but may actually have somewhat reduced it: relative to UCT schools, CCT schools had a slightly higher drop-out rate.” Furthermore, “While the CCTs also had a large positive effect on school participation, explicitly conditioning transfers on attendance significantly decreased their impact in the context of this program. In particular, relative to UCTs, CCTs lowered the impact on re-enrollment of children who had dropped out, perhaps because conditionality discouraged some households (or some teachers) from enrolling weaker children in the program. Correspondingly, CCTs also had a significantly lower impact than UCTs on math scores (CCTs had no impact whatsoever, with negative point estimates).”[v]

Oh dear! Not at all what the conditionistas would have wanted to hear! But because the research had been conducted in partnership with an independent institution, it appears that it wasn’t possible, on this occasion, to revisit the data or redo the analysis (as had happened in the case of Malawi). So the conditionistas did the only thing that was available to them: they invented a new category of cash transfer, a Labelled Cash Transfer (or LCT). This is essentially a UCT that has a message attached, a UCT which tells you that education is good for your child, a UCT for which you need to sign up on school premises. The report suggests that this makes an LCT different from a UCT, and implies that an LCT lies somewhere midway along the spectrum between a UCT and a CCT.

What this means, conveniently for the conditionistas, is that all those uncomfortable findings about a UCT outperforming a CCT can be made to disappear like a rabbit in a hat. The report is explicit about this:

“Note that the comparison between LCTs and CCTs tells us little about the question that other papers in the literature have addressed, namely how an unconditional and unlabeled cash transfer program would compare to a CCT.”

Gobbledygook!! This argument requires that an LCT is not the same as a UCT. Is this tenable? Of course not! A UCT does not become “more conditional” (and thus turn into a new beast, an LCT) just because it has a message attached. A condition is a condition: it is, by definition, a requirement that must be fulfilled in order to receive the transfer, where you are punished if you do not fulfil that condition. If a program does not have that (and the LCT arm in Morocco does not), then it is “unconditional”, pure and simple. Think of this in different context: if a driver has a license, he or she is a licensed driver; if not, he or she is an unlicensed driver. Or another: if a researcher adheres to a set of principles, he or she is a principled researched; if not, he or she is an unprincipled researcher. Ergo, an LCT is a UCT, but just one that places greater emphasis on its messaging. And nobody has ever suggested that such messaging is not a very important component of any cash transfer programme.

What is interesting is where the impetus came from, to mis-label UCTs in this manner. One would assume that it did not come internally, from the authors of the research paper. This is because, intriguingly, there exists an earlier version of their report, dated December 4, 2012. And in this version of the report there is no mention whatsoever of an LCT, the words “label”/“labelled” do not appear one single time, the treatment design is unequivocally defined as “conditional vs. unconditional”, and the respective target groups in the experiment are explicitly described as “UCT to mothers” and “UCT to fathers”. Mysterious indeed that LCTs have reared their ugly heads in the space of just seven months. At whose behest? I think we should be told!

Whatever the answer, there are some key messages that emerge from this important study, and they have nothing to do with Labelled Cash Transfers. These are that, in the case of Morocco:

·       Even small cash transfers have a hugely beneficial impact on schooling

·       Applying conditions has a negative impact on programme performance, whilst at the same time increasing both cost and complexity

·       It makes no difference whether the transfer is given to the father or the mother

·       The existence of an effective information and education campaign increases the impact of the transfers.

These findings have serious implications for policy, and should not be obscured by the chicanery of an invented and unnecessary new category of cash transfer. Please may the authors recognise this, and rewrite their report along the lines of their earlier draft, thus proving that they are principled researchers rather than “labelled researchers”! Otherwise there is a danger of not seeing the Bretton Woods for the LCTs.

---

[i] For a critique of the Malawi experiment, see: Kidd and Calder (2012) The Zomba Conditional Cash Transfer Experiment: an assessment of its methodology. Pathways Perspective No. 6, September 2012.

[ii] Fiszbein, A., and Schady, N., “Conditional Cash Transfers: reducing Present and Future Poverty”, World Bank Policy Research Report, 2009.

[iii] Turning a Shove into a Nudge? A “Labeled Cash Transfer” for Education by Najy Benhassine, Florencia Devoto, Esther Duflo, Pascaline Dupas and Victor Pouliquen.

[iv] NB In these quotations from the report, I have replaced the misleading term “LCT” with the correct term “UCT” – for the reasons explained in the subsequent paragraphs.

[v] Ditto.

Conspirational Cash Transfers

This blog first appeared on the Wahenga website, under the pseudonym of Sissy Teese

Is there a conspiracy afoot? Practitioners of social protection have long debated the relative merits of conditional and unconditional cash transfers. Now the World Bank appears to have introduced a third category. We label these conspirational cash transfers: cash transfer programmes about which the evidence is either suppressed or massaged in a conspiracy to support the case for conditional cash transfers!

Before looking at two examples of such schemes (or schemings), let us take stock. Conditional cash transfers (CCT) require beneficiaries to meet one or more conditions before they receive their transfer: for example, to ensure their children are enrolled in, or attend, school, or to have their children inoculated or regularly visit health clinics. Unconditional cash transfers (UCT) do not: they provide the transfer to everyone eligible, regardless of their behaviour.

CCTs fit much better with the World Bank’s philosophy of seeing social protection in dispassionate, purely economic terms. As a recent publication by the Brooks World Poverty Institute puts it: 

The World Bank conceptualizes social protection as social risk management and proposes policies that seek ‘to assist individuals, households and communities in better managing income risks’ (Holzmann and Jorgensen, 1999: 4). It moves beyond what it sees as ‘traditional’ social protection by adding the goals of macroeconomic stability and financial market development. The emphasis on risk assumes that vulnerability to hazards is a significant constraint on economic and human development, and that efforts to reduce the likelihood of hazards, or to ameliorate their effects on living standards, are essential for economic growth and development [emphasis added]

So the World Bank has been promoting CCTs all over the world, and has recently published a 383-page eulogy on CCTs. There can be no doubt in the reader’s mind after perusing this (or even its 28-page “Overview”) that CCTs have been remarkably successful in achieving their objectives of better education, nutrition and health outcomes for their recipients.

But - and this is the nub of the problem - UCTs have done exactly the same. Unconditional programmes, for example in South Africa, Namibia and Malawi, have been every bit as successful in improving health and education indicators among their beneficiaries as CCTs. So there is a legitimate question to be asked of the World Bank: why bother with the moral hazard, additional cost and complexity of imposing, monitoring and enforcing conditionalities, when unconditional programmes appear to have the same effects?

Answering this question is difficult, because it is almost impossible to unpick the reasons why a CCT programme works: is it because of the conditionality, because of the attendant awareness-raising, or simply because of the cash transfer itself. As Samson et al put it when discussing Mexico’s Oportunidades programme: 

[CCTs] combine three key mechanisms: grants that increase the income of poor households, awareness promotion that emphasises the importance of human capital, and conditionalities that link the two – making the grants conditional on behaviours that reinforce human capital development. The evaluations have successfully demonstrated that all three of these ingredients together can generate very positive results. However, the studies so far have been unable to identify which is most important – the income, the awareness or the conditionality[i]

But, just occasionally, the opportunity arises for the Bank to try to demonstrate that it is the impact of conditionality that determines the beneficial impact. And we document here two example of the lengths to which it will go to take full advantage of these opportunities - two Conspirational Cash Transfer schemes in Ecuador and Malawi.

The first, less flagrant but still revealing, comes from an evaluation of the impact of Ecuador’s Bono de Desarrollo Humano (BDH) cash transfer program on school enrolment and child work among poor children. The unusual thing about this programme was that (unlike most other such programmes in Latin America) it was not in fact conditional … though a number of its beneficiaries were under the impression that it was. This allowed a comparison of impact between recipients who were “conditioned” (i.e. who thought there were conditions attached) and those who were “unconditioned” (i.e. who thought there were no such conditions), which is discussed in a paper by Schady and Araujo[ii].

What is interesting here is the marked difference in the conclusions between a first version of the report dated November 27, 2005 (now virtually unobtainable), and the version published officially as World Bank Policy Research Working Paper 3930 in June 2006. The respective abstracts are already instructive. The earlier version reads as follows: 

We conclude that the program had positive effects on enrolment, and negative effects on child work. Enrollment effects are concentrated among the poorest children in our sample. We also show that the impact of the BDH program was largest among households who believed that transfers were conditional on school enrolment, although the effect of the (unenforced) condition appears to have interacted in important ways with baseline socioeconomic status [emphasis added]

The published version reads very differently:

The main results in the paper are two. First, the BDH program had a large, positive impact on school enrollment, about 10 percentage points, and a large, negative impact on child work, about 17 percentage points. Second, the fact that some households believed that there was a school enrollment requirement attached to the transfers, even though such a requirement was never enforced or monitored in Ecuador, helps explain the magnitude of program effects.

What has happened to the caveat about “interaction with baseline socio-economic status”? In the body of the first report, it is clear that - even at baseline - there are marked, observable differences between the “conditioned” and “unconditioned” samples: 

Conditioned households had significantly higher levels of both paternal and maternal education, children in these households were significantly more likely to be enrolled in school, and conditioned households were significantly more likely to have a television.

Such differences may well explain any differences in subsequent impact between the two groups. [There is also the obvious consideration, that, since the evaluation was based on self-reported school attendance and not on objective evidence such as school records, it is far more likely that respondents who believed the transfer to be conditional upon school enrolment would lie about it, compared with those who knew that telling the truth would have no impact on their entitlement. This would greatly inflate the apparent impact on enrolment for “conditioned” households.]

Overall, the findings in the earlier version are hedged around with provisos, recognising that: 

… enrolment regressions at baseline suggest that observable rather than unobservable differences between conditioned and unconditioned households explain differences in their enrolment decisions. We do not have the data to conclusively rule out this possibility [emphasis added] 

… we believe that it is most likely that the perceived, although unenforced conditions played some causal role explaining the large program effects among better-off, conditioned households [emphasis added]

And the conclusion is correspondingly circumspect: 

It would not be wise to conclude from our results that simply announcing that households have to comply with certain conditions, without enforcing them, will substantially affect household behavior in the long run, in Ecuador or elsewhere

In the published Working Paper (only six months later) such doubts are swept away: 

Although the comparison of lottery effects for conditioned and unconditioned is not experimental, we conclude that the general pattern of results is most consistent with the (unenforced) BDH schooling requirement having a causal effect on outcomes

 and the same paper ends now with a ringing endorsement of conditionality: 

In Ecuador, significant program effects on enrollment are only found among households who believed that there was an enrollment requirement associated with the program; this suggests that this unenforced condition was important

Interestingly, another contemporary study of the same programme in Ecuador[iii] finds “no significant impact of the program on school enrolment” and concludes that “the enforcement of conditionality has not any significant effect on school enrolment among the less poor”. This further underlines the need to retain the important caveats of the first version of the World Bank report.

The second egregious example of a Conspirational Cash Transfer is more recent, and comes from a World Bank experiment in Malawi. This was an evaluation of the impact of a randomized conditional/unconditional cash transfer intervention targeting young women in Zomba district that provides incentives (in the form of school fees and cash transfers) to current schoolgirls and young women who have recently dropped out of school to stay in or return to school. Splitting the sample - between some that received a transfer conditionally, some that received it unconditionally and some that received no transfer at all - allowed a comparison of different impacts between the three different groups.

Here again we can observe an unexpected evolution in the published results. The first set of findings is drawn from a publicly available PowerPoint presentation[iv] of the study. This concludes unequivocally that: 

We don’t find any evidence that the conditional transfers are more effective in improving schooling

And it suggests that the best way to design a cost-effective program, based on the lessons learned, would be “possibly foregoing the conditionality”!

Indeed, looking at the data presented (which unfortunately appears on slides in the PowerPoint that cannot be cited), it is clear that there is no significant difference at all between the conditioned and unconditioned groups in terms of school enrolment or literacy in English; and that it is the unconditioned group that performs better in terms of avoiding early marriage. Altogether, on this basis, the study would not appear to present a strong argument in favour of CCTs!

Yet what do we find when the same study is published as official World Bank Policy Research Working Paper 5089[v]? Miraculously the selfsame findings are used to proclaim: 

Overall, these results suggest that conditional (sic) cash transfer programs not only serve as useful tools for improving school attendance, but may also reduce sexual activity, teen pregnancy, and early marriage

 How has this come about? First of all by ignoring entirely the set of data from the unconditional sample. Tucked away in an unobtrusive footnote on page 11 is the following: 

283 of these girls resided in EAs where the offers for baseline schoolgirls were not conditional on school attendance, and, as such, are not part of the analysis for this paper

How convenient! Secondly by arrogating to conditional cash transfers all of the benefits that should rightly be ascribed to cash transfers. Another footnote on page 12 brushes this aside:

Finally, baseline schoolgirls in a randomly selected small percentage of the EAs received unconditional offers, meaning that the transfers were not conditional on school attendance, or any other behavior other than showing up to collect monthly payments, for these beneficiaries in those EAs. The analysis of the heterogeneity of the impacts with respect to each of these design features is beyond the scope of this paper. Here, we aim to establish the average effect of the conditional treatment arms, which may not equal the treatment effect of the average treatment if these impacts are nonlinear [emphasis added]

Quite so! The result is that an experiment which has actually demonstrated that unconditional transfers are every bit as effective as conditional transfers has now been manipulated to support a claim that: 

schooling CCTs (sic) for young women in the context of poor Sub-Saharan countries with high HIV rates seem like “win-win” programs, as they may not only increase schooling for young women, but also significantly reduce their risk of HIV infection.

Yes, the Zomba programme does appear to have extremely promising impacts: large increases in self-reported school enrolment, and declines in early marriage, teenage pregnancy, sexual activity, and risky sexual behaviour. But to claim that this has anything to do with conditionality per se is highly disingenuous. It is self-evidently the cash transfer and not the conditionality that is the cause.

Here we have an important debate, but it is one that should be aired openly and honestly. Policy-makers should be aware of both sides of the argument, and should not allow themselves to be pushed into Conditional - and still less into Conspirational - Cash Transfer programmes!

---

[i] Samson, M., van Niekerk, I., and MacQuene, K. (2006) ‘Designing and Implementing Social Transfer Programmes’, Economic and Policy Research Institute Press, Cape Town: EPRI

[ii] Schady, N., and Araujo, M. ‘Cash transfers, conditions, school enrollment, and child work: Evidence from a randomized experiment in Ecuador’

[iii] Ponce, J., The Impact of a Conditional Cash Transfer on School Enrollment: the Bono de Desarrollo Humano of Ecuador”, FLACSO Documento de Trabajo 06/302, April 2006

[iv] Available at http://www.fundp.ac.be/eco/recherche/cred/SUMMERSCHOOL/amid/ozler2.pdf, June 2009

[v] Baird, S., Chirwa, E., McIntosh, C. and Özler, B., The Short-Term Impacts of a Schooling Conditional Cash Transfer Program on the Sexual Behavior of Young Women, World Bank, Oct 2009