Protocol for a Systematic Review: Effects of Certification Systems for Agricultural Commodity Production on Socio‐economic Outcomes in Low and Middle‐Income Countries

Campbell Systematic ReviewsVolume 11, Issue 1 p. 1-71 PROTOCOLOpen Access Protocol for a Systematic Review: Effects of Certification Systems for Agricultural Commodity Production on Socio-economic Outcomes in Low and Middle-Income Countries Carlos Oya, Carlos OyaSearch for more papers by this authorDeborah Johnston, Deborah JohnstonSearch for more papers by this authorEvans Muchiri, Evans MuchiriSearch for more papers by this authorFlorian Schaefer, Florian SchaeferSearch for more papers by this authorDafni Skalidou, Dafni SkalidouSearch for more papers by this authorKelly Dickson, Kelly DicksonSearch for more papers by this authorClaire Stansfield, Claire StansfieldSearch for more papers by this author Carlos Oya, Carlos OyaSearch for more papers by this authorDeborah Johnston, Deborah JohnstonSearch for more papers by this authorEvans Muchiri, Evans MuchiriSearch for more papers by this authorFlorian Schaefer, Florian SchaeferSearch for more papers by this authorDafni Skalidou, Dafni SkalidouSearch for more papers by this authorKelly Dickson, Kelly DicksonSearch for more papers by this authorClaire Stansfield, Claire StansfieldSearch for more papers by this author First published: 01 July 2015 https://doi.org/10.1002/CL2.144Citations: 2 Linked article: Systematic review. AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat BACKGROUND Introduction The role of international trade in reducing poverty and increasing welfare in low- and middle-income countries (LMICs) remains an issue of controversy and debate (Winters, 2003; McCulloch et al., 2001). Open economies may perform better in the long term, Winters (2002) argues, but in the short term trade liberalisation can have adverse effects on the most vulnerable actors in the economy and some risk getting trapped in poverty. This is likely to happen to agricultural producers in developing countries, as Nicholls and Opal (2004) highlight, where deficient microeconomic conditions (poor market information, limited access to markets and credit, lack of ability to adapt rapidly to market changes, among others) are coupled with chronic macroeconomic failures, such as the lack of infrastructure and investment, heavy dependence on only few primary commodities and corruption. Primary commodity producers are often particularly vulnerable to price volatility and inadequate and asymmetric price transmission mechanisms. In addition, international markets for agricultural commodities are increasingly demanding in terms of quality and production conditions, whether related to social or environmental sustainability (Henson & Humphrey, 2010). For example, demand trends in consuming countries have included the emergence of the 'specialty coffee market', which is becoming ever more important as traceability and other specific features become valued by consumers (Daviron & Ponte, 2005). Partly as a result of these broad world demand trends in agricultural trade, a wide range of voluntary private standards (or codes of conduct) have emerged in the past few decades to complement public standards to deal with the trade in agricultural commodities, typically monitored through private audits and third-party certification (Barrientos et al., 2003; Schuster & Maertens, 2015). Such voluntary private standards can be classified either as own company standards, which affect only the workings and supply chain of a single company, or collective standards at both national and international levels, which are available to any number of actors as long as they can fulfil the requirements set by the standard (Henson & Humphrey, 2010). We focus in particular on certification schemes for agricultural commodity production, by which we mean collective standards, subject to third-party certification and auditing processes. Usually these standards should or tend to conform to internationally recognised guidelines such as ISO/IEC 17065:20121. A broad definition provided by ISO/IEC states that 'the overall aim of certifying products, processes or services is to give confidence to all interested parties that a product, process or service fulfils specified requirements'2. Since such standards increasingly determine the terms of integration of agricultural producers in LMICs into global supply chains (Gibbon & Ponte, 2005), an important debate has emerged about the effectiveness of certification in raising the welfare of direct producers and workers. Given the wide range of certification schemes, certified products and countries involved, it is perhaps not surprising that impact evaluations have found different results. Many studies tend to report mixed findings with some positive and other negative elements, or cases where effects are only marginal (Nelson & Martin, 2013). Some have even found that certification schemes may actually undermine the incomes of the poorest farmers (Henson & Jaffee, 2008), some reported positive impacts for some certification types, but not others (Chiputwa, Spielman & Qaim, 2014), others found effects only for richer farmers (Hansen & Trifković, 2014), while still others showed how certification schemes can help raise rural incomes and reduce poverty (Maertens & Swinnon, 2009). In the case of Fair Trade3 standards the evidence from primary studies is not conclusive either and the quality of studies measuring effectiveness is uneven and uncertain, as a number of studies and metareviews show (Ruben, 2013; FTEPR, 2014; Valkila & Nygren, 2009, Terstappen et al., 2012, International Trade Centre, 2011; Nelson & Pound, 2009; Nelson & Martin, 2013). Many of these studies and existing meta-reviews acknowledge the fact that mixed results may be consistent with the inherent complexities within these different types of interventions, the presence of many factors outside the control of interventions, such as the specifics of commodities and value chains considered, the different standards in question, as well as the diversity of implementation contexts, even within the same certification scheme. This debate is likely to become increasingly relevant as the sales of agricultural commodities through market channels that require these kinds of certification expand rapidly. For example, in the case of Fairtrade, there are now more than 30,000 certified products worldwide (Fairtrade International, 2014) and the UK market has grown to over 4,500 Fairtrade certified products (http://20years.fairtrade.org.uk/). In the UK in 2013 alone 'sales of Fairtrade products exceeded an estimated value of £1.7bn, a 12% increase on 2012' (Fairtrade Foundation, 2014, p. 11). Part of the growth of certified products like Fairtrade has to do with the fact that large suppliers and retailers have embraced the branding opportunities involved, as in the case of Nestlé, which launched its own Fair Trade coffee in 2005, and Sara Lee/Douwe Egberts' growing association with UTZ Certified coffee for the European market (Tropical Commodity Coalition, 2012). Description of Certification Schemes and their Interventions Most certification schemes (CS hereafter) for agricultural commodity production have their roots in ideas about ethical trading in Europe and the US going back at least to the 1980s (Blowfield, 1999; Barratt-Brown, 1993). With supply chains lengthening as a result of the spread of global value chains, consumers – and some firms –began to question the pay and working conditions of the workers and producers in LMICs. Ethical trade seemed to offer an alternative and by the late 1990s voluntary private standards were firmly established in a number of sectors (Barrientos, 2000). More recently, food standards, aimed primarily at quality assurance, have become vital to food exports from LMICs (Hansen & Trifković, 2014). Generally, certification schemes aim to improve on the effects of free trade by offering better trading conditions, supporting smallholder producer organisations to gain better market access, assisting to enhance product quality, designing specific interventions or incentives to raise productivity, or a combination of these aspects. A challenge for any study of certification of agricultural commodities is that standards tend 'to vary in terms of their reach and objectives' and 'there are also major differences regarding the scope of the offering of certified commodities and products' (von Hagen et al., 2010: 1). They encompass a wide range of different goals and of different methods of achieving those goals. An important differentiation has to be made between the act of licensing itself and direct interventions that precede or follow the licensing process. While the act of certification itself is not a development intervention per se, the introduction of codified standards, often, but not always, in the form of a consumer label following an auditing process, may induce behavioural changes in farmers, resulting for example in specific investments that benefit production conditions and open access to better market opportunities, without any direct intervention at farm level by the certifying body. But most certification schemes do require direct interventions at the level of the farm, the producer group or the workers' group. In short, different certification schemes are best understood as bundles of interventions, guided by a variety of theories of change. As a result, certification schemes differ greatly in the populations they target, in the outcomes they seek to certify, in the implementation models, and in the audit and certification process itself. Besides, levels of compliance and requirements for improvement over time will also vary between certification schemes. There are certification schemes which operate primarily to enhance the quality and sustainable farming practices achieved by agricultural producers to ensure their products qualify for better market niches, more amenable to sustainable income generation, as is the case of some MPS certificates for flowers, or GlobalG.A.P. (previously EurepG.A.P.) for horticultural produce, which, for example, do not result in consumer labels. Other schemes more directly seek to establish ethical trading conditions by offering alternative markets with higher prices and/or floor prices that cover the costs of production. Amongst these certification schemes, one influential set are Ethical Trading Initiative (ETI) schemes and particularly Fairtrade, which aim to address the adverse effects of international trade by offering better trading conditions to, and securing the rights of, small agricultural producers, workers and their communities and helping them to organise to achieve these goals (Dragusanu et al., 2014). Fair trade-type schemes, unlike other CS more concerned about the quality and characteristics of the product, were primarily designed to directly affect socio-economic outcomes and the empowerment of agricultural producers and workers through different direct interventions, originally as an alternative to perceived 'unfair' free market channels (Barratt-Brown, 1993). Fairtrade, for instance, operates through a set of standardised and audited interventions (floor prices, provision of a premium, credit-availability, assistance to access the market, support to small producer organisations –SPOs – such as small farmers' cooperatives and/or workers' organisations, and so forth) which are conditional on a number of requirements related to democratic processes, participation, transparency and the adherence to environmental and labour standards (see Fairtrade ToCs for a more detailed account of the various pathways to impact considered in relation to their different aims and interventions)4. It is worth mentioning that sometimes CS coexist alongside additional interventions by NGOs that adhere to the CS social and environmental sustainability standards, as is the case of OXFAM and the Fairtrade certification or TechnoServe (see section on study design for a more detailed discussion). Therefore, as well as the direct interventions being implemented by CS themselves, there is often some form of external support (by NGOs, donor agencies, buyers) which may have been leveraged because the producers or groups of producers have obtained a certification. This can affect the interpretation of findings, so the review will have to consider these additional factors as part of the moderator analysis, as long as these instances are actually reported by included studies. The range of mediating factors is obviously wide and variegated. However, an important aspect, given that the focus is on agricultural commodities, are the market conditions and value chain characteristics for each particular commodity that may be subject to a range of standards set and monitored by various CS. Therefore, some CS may be more or less effective in reaching some socio-economic outcomes depending on the nature of the value chain and also on the dynamics of agricultural commodity markets, particularly for interventions that focus on prices and premium for producers. Commodity market cycles are likely to have an impact on producers and their workers thereby affecting the interpretation of findings about effects of CS depending on the period of time considered, even in the case of longitudinal studies. Overall, in any case, by implementing the various bundles of interventions, CS are expected to produce positive outcomes that improve the wellbeing of beneficiaries in terms of higher and more stable incomes, better services to improve businesses, as well as education, health and other aspects of human welfare, and decent working conditions for wage workers. These interventions are expected to directly and indirectly empower marginalised agricultural producers, workers and their communities. A further complication is that CS increasingly expand their set of standards to qualify for a wider range of markets, products and consumers, and to compete with other certification schemes. A quick scoping survey of CS shows that overlaps may be significant and the wording of standards and codes of conduct are often strikingly similar despite very different histories and modus operandi5. Each certifying organisation may operate different certifications at the same time, depending on the standards applied and the target group (whether small or large farmers, workers, individual producers or organisations). Therefore it is not possible to assign a particular type of certification to a single particular scheme. Many of these schemes, for example apply conventional decent work ILO labour standards as part of their commitment to ethical trade, or share emphasis on 'sustainable farming methods'. In other words, while CS may be very different in some respects they may also overlap substantially on some of their standards. It is therefore necessary to distinguish between a certification scheme (Fairtrade, MPS, Utz Certified, Rainforest Alliance, and so forth) and a standard (more broadly social or environmental standards, and, more specifically, a living wage, the prohibition of certain chemicals, democracy in producer organisations, and so forth). Moreover, overlaps may also happen at the beneficiary level, when producer organisations or individual producers/employers may receive more than one certification making attribution particularly difficult especially when studies do not report the timing of different certifications and the extent of compliance for each of the certifications (see for instance Woubie et al., 2015 on the implications of double certification). A related challenge in any review of studies of the effects of certification on socio-economic outcomes is that a particular type of certification can be provided by a variety of certifying bodies/organisations, which may fall under the broad category of voluntary 'social sustainability standards' and conform to broad internationally recognised guidelines such as ISO/IEC 17065:2012, which replaced ISO/IEC Guide 65:19966. For instance, Fair Trade certification may be provided by the Fairtrade International (FLO), or alternative trade organisations within the WFTO, such as CTM Altromercato. Indeed, the Fair Trade network has evolved significantly in the past three decades and has given rise to a variety of organisations that may share a similar ethos and objectives but may differ in terms of focus, outreach, interventions and auditing processes (Jaffee & Henson, 2004; ProForest, 2005; Muradian & Pelupessy, 2005; Kolk, 2005). There can also be various levels of certification by the same certifying body as in the case of MPS, depending on what particular standards are applied. There is therefore a multiplicity of standards and certifications that often overlap and compete with one another (von Hagen et al., 2010). The types of CS that are the focus of this review are described in detail in section 3.1.2. A systematic review could in theory be conducted on every single intervention, which could happen under different CS, as in the case of labour standards interventions that are common to most schemes subscribing to ethical trade standards. However, the reality is that most CS operate with bundles of interventions and most studies will report on the certifications and not on single interventions. Also, seemingly similar interventions may be structured and implemented differently in different places and at different times, encompassing different intervention components. For example, technical assistance and capacity building for better farming practices or to improve organisational performance, may be implemented with a variety of intervention components. This makes the analysis of the causal chain particularly complicated because endpoint outcomes may be attributable to a bundle of interventions without sufficient evidence on which particular intervention component is more effective. For example, in the case of MPS-SQ is the certification more effective because of the enforcement of labour standards or because of the quality standards imposed and their spillover effects on other intermediate outcomes? In the case of Fair Trade-type interventions, if effects on endpoint outcomes are considered positive, can they be attributed to the setting of a price premium or to the adequate investment of a premium or simply to a balanced combination of both in addition to other direct support to producers' organisations? Most impact evaluations will find it difficult to disentangle the specific effects of these different interventions under the same scheme. At the same time, most CS will consider that what matters is the specific mix of interventions, for instance including various forms of capacity building for producers, and not any one intervention in particular. However, studies may report relevant information that may give insights into the key causal mechanisms either through quantitative or qualitative evidence and this will be used to assess Review Question 2 (see the section on objectives for questions addressed in this review). HOW THE INTERVENTIONS MIGHT WORK A major challenge for this review will be that different certification schemes that aim to improve the welfare of agricultural producers and workers in agriculture differ in their model of intervention and in their theory of change (ToC). For example, Fair Trade schemes focus on prices, market access and organisational empowerment, while MPS is mainly about sustainable quality and social standards, and UTZ Certified, while similar to Fair Trade schemes in terms of the broad aims, works in terms of improvements in farming practices and quality rather than price mechanisms. Moreover, each certification scheme may also incorporate different grades of certification, as in the case of MPS for flowers, or the different standards applied by Fairtrade to SPOs (Small Producer Organisations) or HLOs (Hired Labour Organisations, that is, large-scale plantations). Given the wide variety of certification schemes, their intended outcomes and methods of intervention there is no single theory of change that is valid for all types of certification schemes. There have been attempts by researchers to develop a ToC valid for more than one CS (Nelson & Martin, 2011; Nelson & Martin, 2013). Indeed in 2009, as reported by these authors, 'sustainability standards had yet to articulate their own theories of change […] although this situation has now changed as a result of the ISEAL Impacts Code and with contributions from this research project', including subsequent studies that drew on NRI reports to develop ToC for impact assessments. Some certification schemes have also recently produced an explicit theory of change, but readers may benefit from consulting the ToC developed, for example, by Utz Certified and Fairtrade as indicative examples7. ISEAL, as an umbrella organisation for a range of sustainability standards have also produced a ToC and routinely publish drafts and discussions of ToC for individual member organisations8. Drawing from these initial attempts at developing ToC to evaluate impact evidence on sustainable standards, as well as on the recent ToC developed by CS themselves, we have produced a simplified synthetic ToC that summarises the key linkages in the causal chain between types of interventions, intermediate outcomes and endpoint outcomes, bearing in mind the focus of this review on socio-economic outcomes, and in line with some earlier attempts such as Nelson and Martin (2011). Some of the organisational ToC mentioned above, particularly the one developed in 2013 by Fairtrade, may be more complex and multifaceted than the synthetic ToC we propose here. This is because CS like Fairtrade also focus on actions and advocacy among consumers to expand the market for Fairtrade certified products and generally the values of Fair Trade. They also include environmental standards as part of the broad canvass of sustainable outcomes. The focus of this review is, however, on the role of standards and interventions that more directly affect the wellbeing of producers and workers involved in the production of certified commodities. The aim is not to evaluate the work of all these different CS, rather to evaluate and synthesise the existing evidence on socio-economic outcomes associated with interventions under CS as defined in this review. This is necessary to keep the review manageable and allow a consistent framework that can be applied to a wider range of CS. Below are illustrations of how different types of interventions, which are used by different certification schemes, may affect intended outcomes, and therefore the assumed causal chains, which will be analysed in this review. 1. Price and contract interventions. Interventions to offer price premium or floor prices are expected to → contribute to higher and more stable producer prices, which can → result in higher net profits for agricultural producers, assuming they are not offset by high certification costs. In addition, both pre-payment, credit and longer-term contracts can → improve income stability and reduce vulnerability to shocks. These effects can → result in higher income and consumption at household level. 2. Market access interventions. Access to alternative and/or additional market certification schemes can → contribute to: higher prices; better contracts; strengthened market power and negotiation capacities of producer organisations and ultimately → to their members' empowerment. 3. Product quality. Better farming practices (including in some cases environmental standards developed through organic farming practices that may lead to better remunerated markets) and associated technical assistance (capacity building) are expected to → lead to higher agricultural incomes and strengthened market power of beneficiaries thereby → raising their capacity to invest in their own production. 4. Monitoring of producer organisation practices and technical assistance (capacity building) to producer organisations and individual agricultural producers. Democracy standards (Fairtrade) and other organisational improvements can → result in strengthened organisations in terms of their legitimacy, participation and capacity to negotiate, which → can lead to members' empowerment and access to better services. 5. Premium. The premium, as an additional sum of money paid on top of the minimum price → can be invested (by farmers and workers' organisations) in a variety of assets/infrastructure (for example, social, environmental and economic developmental projects), leading to possible positive outcomes → better education and health access/outcomes; incomes if economic infrastructure/assets improve; empowerment via strengthened beneficiary organisations; better working conditions. 6. Labour standards. Their implementation can directly → impact workers' wellbeing through of living/better wages, and better working conditions, especially when health and safety conditions improve and affect workers' health. Outcomes should be reviewed for workers employed by all types of agricultural producers from smallholder to large scale organisations. In Figure 1 we present a simplified synthetic theory of change, which captures the overall logic of interventions under certification schemes. This is synthesised from multiple theories of change from some of the most prominent CS types. The synthetic theory of change was developed to be broad enough to be able to capture all intervention methods we are going to encounter under various certification schemes. It summarises potential causal pathways to impact and key assumptions for four different broad categories of intervention, namely interventions around farm practices, on prices, markets and purchasing agreements, on labour standards and through premiums. Some schemes focus on one or more of the interventions mentioned, while others focus only on one. This synthetic theory of change illustrates the difficulties inherent in aggregating results on effectiveness over a heterogeneous body of schemes and interventions. A key aspect of any theory of change is a listing of the assumptions that must hold at each step along the causal chain for interventions to have their desired effect. If assumptions do not hold effects may be diminished, skewed, or entirely absent. In the worst case there may even be unintended adverse effects on producers or workers. However, assumptions also differ in their importance for different interventions and thereby certification types. For instance in some cases farmers' pre-existing capacities and therefore self-selection into the scheme (as in quality-oriented schemes) are more important than others (such as Fair Trade schemes for example, see section on study design for more details). In other cases assumptions about distribution of benefits among members of a group matter more when beneficiaries are targeted in groups (as with the premium in Fairtrade certification of SPOs for instance) than when they are targeted individually. The distribution of benefits may also not be equal between workers and employers, where large employers are targeted, or there may be differences between different types of workers. The distribution of personal protective gear may for instance benefit the most at risk workers (for example, sprayers), but may have very little impact on workers who are less at right (for example, those in packaging). Finally, it is important to note that this review does not take the ToC shown below as the definitive ToC that will guide the final analysis of findings. This simplified ToC may omit some other possible pathways to impact that may not conform to the chains illustrated in the diagram. However, it provides what we think are the most important ones, based on knowledge of literature and other ToC that have been developed so far. In any case, the results of the integrated synthesis for Review questions 1 and 2 will be used to update and reconsider some of the linkages, assumptions and pathways to impact anticipated in the ToC developed for this protocol. It is also hoped that the resulting ToC will be of use to organisations in the process of revising or developing their own ToC, given that a ToC of certification standards is still work in progress. Figure 1. Simplified synthetic theory of changeOpen in figure viewerPowerPoint WHY IS IT IMPORTANT TO DO THIS REVIEW? This systematic review will address the extent to which, and under what conditions, interventions under various certification schemes for agricultural commodity production result in higher socio-economic welfare for agricultural producers and workers in low- and middle-income countries (LMICs) – questions about which there is an ongoing and as yet unsettled debate. As briefly noted above, the current evidence base for the overall impact of interventions resulting from certification schemes for agricultural commodity production on agricultural producers and workers is generally mixed in terms of the reported results, including a range of studies that report either quite positive or negative results. There have, however, been some attempts to systematically review the evidence. A study by the International Trade Centre (2011), one of a four part review series on certification schemes, for instance seeks to present the overall findings of the relevant literature using systematic review methods. Unfortunately, the study uses vote counti