Bean impact: making sense of 16%

12 July, 2013 by (comments)

Having had the chance to witness the spread of better beans in East Africa it was great to hear that the, well, bean counters had been sent in to do a formal impact assessment. The results will form part a series of studies known as the Diffusion and Impact of Improved Varieties in Africa (DIIVA).

NP Rwanda 4

But at first glance, it seems this particular DIIVA isn’t exactly dancing. Like all impact reports, the bean study provides a lot of cold, hard statistics – some of them excruciatingly precise – but the gist is essentially this: years of research, development and dissemination of improved bean varieties have resulted in negligible reductions in poverty, down 0.1% in Uganda and 0.4% in Rwanda, and only a 2% reduction in food insecurity in Uganda, and 16% in Rwanda.

What happened? The beans had all manner of useful characteristics: pest and disease resistant, able to produce high yields, and culturally acceptable in terms of colour, grain size and cooking time. Is 16% the best we can show?

Wait a second

If the impacts seem low, that’s because we’re not comparing like-with-like. Bean impact is not a maths test, where a score of 16% might rightly result in your prompt ejection from class. Quite the opposite. When it comes to tackling food insecurity, no-one is under any illusion there’s a magic bullet solution; it’s not that simple. The Green Revolution was more than just semi-dwarf rice, after all.

So when you can pin a 16% reduction in food insecurity to one single variable, that’s actually a sign of a pretty successful intervention, isn’t it? In fact, the report itself describes this particular impact as “substantial.” It means that for about 500,000 people, the hunger months – those where food supplies are lowest – have completely vanished thanks to improved beans. When it comes specifically to poverty reduction, the team’s findings were as expected because “the influence of improved varieties…is likely to occur through channels other than profitability.”

Nevertheless, the report’s pages of methodological minutiae leave the positive impacts somewhat short of a human face. That’s part-and-parcel of economics as the oft-maligned dismal science: it’s by nature a little bit joyless and impersonal.

But I know that in those statistics somewhere is the ear-to-ear grin of farmer Jean Damascene Bizimana, when we met him and his star-performing climbing beans in Rwanda’s Northern Province. I know it captures the gentle optimism of mild-mannered Stanslas Binyavanga – who, now in his sixties – has lived through poverty and genocide to end up growing improved beans, earning enough money to send his children to university and buy cows for his neighbours. And I know it includes the industrious resilience of Olive Nakure, who, with the money from her beans, invested in a big sewing machine and trained in Uganda for two months to learn how to use it. She makes about ten children’s jumpers a week, selling them to local schools. In the field, the stats spring to life.

NP Rwanda 27

Margins for understatement

There are several other things to bear in mind. When undertaking an impact assessment, economists inevitably have to take the plunge and stop the clock at an arbitrary point in time to take stock, while beneficial ripple effects continue to spread out unquantified. Adoption of the new beans in neither country has reached saturation point yet, so expect more and more farmers to get involved, and the benefits to gradually – we all hope – snowball.

It also takes a long time to crunch the numbers, so when the clock was stopped in 2010 and assessment officers visited bean farmers in East Africa the following year, they didn’t include the several improved varieties released post-2010. Also, the assessment only includes beans that are considered “modern” varieties – those released since 1998, even though CIAT has been involved in bean improvement in the region since the 1980s. Some of these pre-modern varieties are still the top-performing beans in some areas, but they’re excluded from the report.

Then there’s another “X-factor”: in both countries researchers found that some beans could not be reliably identified as originating from work conducted by CIAT and its partners – up to 20.61% in the case of Uganda and 12.49% in the case of Rwanda, almost certainly leading to an understatement in the headline figures to some degree. “It’s better to be cautious,” says CIAT impact assessment officer Ricardo Labarta, one of two CIAT scientists who compiled the report, with researchers from Virginia Tech in the USA, and he’s right.

Separate impact assessment work in Bolivia is attempting to close this kind of uncertainty gap. Rice samples of uncertain origin will be grown at CIAT’s headquarters in Colombia and genetically sequenced to see if they contain DNA traceable to samples its gene bank. Eventually, Ricardo hopes to do the same with beans in Africa. He says it’s a technique that the Standing Panel on Impact Assessment (SPIA) – part of the Independent Science and Partnership Council – strongly recommends.

So really, the bean report should be commended for erring on the side of caution, and readers should try and contain knee-jerk reactions to the apparently less-than-spectacular results, until digging a little deeper.

And at the very least, 16% means improved beans could be one possible magic bullet in a barrel of six. Or rather, 6.25.


The DIIVA report will be made available online soon. In the meantime you can request a copy from r dot labarta at cgiar dot org

CIAT will be discussing its bean improvement work at the 6th Africa Agriculture Science Week in Accra, Ghana, next week. For the latest on the event, follow the AASW Blog and Twitter feed #AASW6.


See also:

Tagged With: , , , , , , , , , , , , , , , , , , , , ,
Filed Under: Africa @en, Beans @en