In last week’s GCC Journal Club meeting we discussed an article by Jeremy et al. (2012, see reference below), a paper that uses a nice framework and an impressive “multilevel design and model-fitting approach” to evaluate to what extent Community-Based Conservation (CBC) projects have been successful around the world. The study tests hypotheses about different features of the national context, project design, and community-level characteristics, an approach that provides for many possible links and explanations and a wealth of data as well as multiple options of bivariate and multivariate data analysis explaining the success CBC case studies. 
The reason I had selected this particular paper for discussion was because, after 15 or so years of working with CBC-related issues, I was excited to finally come across such a comprehensive review of CBC (n=136 case studies reviewed). In addition, despite being a CBC promoter myself, I admittedly carry many years of hesitance and doubt as to whether or not most CBC projects really do work in the long run. Based both on what I’ve witnessed on the ground, as well as what I’ve read in the literature, my sense has long been that if any aspect of CBC is going to work, it’s going to be on the economic front, but less likely in terms of ecological or even social (equity-based) outcomes. This was just the hunch I had.
Hence, after reading this paper, I wanted to share my pleasant surprise with my peers: the authors found that “projects reported more successes than failures across all four domains, with ecological outcomes having, proportionally, the highest frequency of success”. There are two equally surprising points here that jumped out at me: 1) that CBC project do work; and 2) that particularly the ecological outcomes (as opposed to economic or social outcomes) were reported as being most successful in the reviewed CBC cases. I was even more surprised when I read further along the paper that:

“we also found no evidence that market integration was associated with any measure of success, underscoring how generalizations should not be made about the effects of market access on project outcomes without considering the nature of the resources, the size and make-up of the community, and the type of market activity in question.”

This comes across as a striking contrast to what has (worryingly) become the rhetoric in environmental sciences and politics today: that giving monetary value to the environment will help save it. It seems today that everyone from climate scientists to ecosystem services researchers are talking a language that revolves around monetary valuation, markets, financial mechanisms (eg. conservation payments, REDD+, etc), and payments. Even the leading journals in these fields are taking a strong stance to support this line of thought (one only has to look at the publication trends of journals like Ecological Economics to see this shift in stance).
But, coming back to the article by Brooks et al. (2012), the questions and discussion that arose in our journal club were more scientific than philosophical. Firstly, how and why were these four indicators – namely attitudes, behavior, ecological and economic “outcomes” – for measuring success selected? The authors do not provide any clear or convincing explanation or justification for this selection, nor do they deal with the question of causality between these outcomes. At least I was left to wonder how much difference there really is in a project outcome between attitude and behavior. Are the two not linked facets of the same side – the social dimension – of any project? And as for causality, are these not all causal factors of outcomes as much as they are outcomes per se? For example, surely attitudes will influence behavior, and possibly also ecological and economic performance, just as economic outcomes will no doubt influence attitudes and behavior. There is a lot of dubious interplay and overlap between these, and the paper fails to go into more depth and detail to examine these fine but important dynamics.
The study is also weak in its analysis of each specific outcome. As a group of predominantly ecologists, we found that the methods they has used for analyzing the ecological outcome are so broad that the case study results are almost made non-comparable across sites. Successes in projects should in general be measured against project objectives, something this article did not look into. So, for instance, it would be interesting to see whether very specifically-targeted CBC projects (e.g. for the protection of turtles, chimpanzees, or lemurs) have different outcomes across domains than the more widely conceptualized projects (such as general community forests or CBC more wide in scope). Similarly, the study did not look at pro-conservation mechanisms, such as possible regeneration initiatives and what role these might play, nor did it question the role, pressures and impacts of possible encroachers on community-managed lands. After all, the success of these projects is not only based on the local people, but also on the surrounding socio-political and economic environment, including the presence of external pressures such as encroachers.
We also found the analysis to be too rigid, categorizing cases as either “successes” or “failures”, when in reality we know that no project is an absolute success or failure, and it may be successful in one domain but a failure in another. This simplified analytical framework easily waters down the results to the point where we might be losing findings on intricate links and dynamics that could work to better inform CBC project managers. The paper would have been much more helpful and useful for future research and application if it had looked in more detail at the fine nuances of “successes” or “failures” of CBC, rather than just broadly throwing the results into two or three “success categories”. Another major limitation in the paper is that the supposed “successes” were measured in any way in relation to the original starting point of the project, leading us to question whether these results are even comparable without taking baseline data into account.
All these caveats and limitations of the study should have been stated more clearly and explicitly in the article itself. For instance, the authors should have pointed out the biases of the original dataset (e.g. that it mostly consists of “success story” case studies, as opposed failed CBC projects, largely also due to the tendency in science to only publish positive results: see “Why Science Goes Wrong”). The case studies are also not globally representative, with no case studies whatsoever from Europe. These and other caveats are currently hidden in small text in the extra supporting material annexes to the article, whereas they should be raised in the article itself so that readers are made aware of them.

I did find it refreshing, however, to read that “ecological success is most likely when the project engages positively with cultural traditions and governance institutions,… builds capacity in communities…, and when communities participate in project initiation, establishment, and daily management…” These are findings that have long been written about in the social science literature (especially in the literature on institutions and participation), but having quantitative results from a comprehensive, systematic literature review like this one really helps to further stress the importance of each these points – and particularly the ecological importance of them (as usually the argument has been an ethical or socio-political one). Similarly, tenure was (surprise surprise) once again highlighted as a key factor influencing CBC results – something that keeps being “all talk, little action” in our world where lands are still to a large extent state-owned.
Yet, one surprising result in the paper was that “the national context does not play an important role in any domain of project success”, and that “well-designed projects can be successful even in national contexts often viewed as nonconductive to success (such as rampant corruption)”. This is a strong statement to make, and I wonder whether the authors have looked at this variable in relation to a) the running-time length of the project (especially projects spanning more than 10 or 20 years in highly unequal, political unstable, corrupt nations), and b) who is really benefitting from these projects in cases where CBC supposedly “works”, but the national context is highly unconducive or unsupportive. It would also be interesting to look closer at only the case studies involving National Parks, and see whether in those cases the national context plays a more significant role.
The authors also found that projects that have been running for a longer period of time are more likely to succeed economically than more recently started projects, implying that CBC requires time for economic development (e.g. for accrued income generation, etc) to be felt. Good point, but I would challenge the authors that this might hold true also for other outcomes: e.g. that ecological benefits might in some ecosystems also accrue only over long periods of time (especially if there is a conservation targeted species or habitat with long regeneration/reproductive cycles).
I appreciated the second-last paragraph where the authors discuss qualitative vs. quantitative approaches to studying CBC. Here the authors do finally acknowledge that studying conservation outcomes involves “a complexity of causal processes”, which at times can be better illuminated using detailed qualitative research. However, as the authors state, quantitative correlative studies (where correlation does not equal causation) are also critical for testing hypothesis derived from qualitative work and for conducting systematic comparisons that are “indispensable for guiding a broader understanding of the challenges and opportunities of CBC.”
Over all, and to sum up, the authors have set a good framework for a review and analysis of CBC projects, one that is good to elaborate on in future research. However, now we should go back to the data and look more carefully at that: at how it was collected, on what is comparable and what is not, and then carry out an even more quantitatively rigorous analysis. This article claims to be quantitative in approach, yet it is still too qualitative in its outcome data and comparison. The discussion would also benefit from incorporating other similar studies that have emerged since its publication (e.g. by Porter-Bolland et al, 2012 and Nolte et al, 2013).
Paper discussed:
Brooks, Jeremy S., Waylen, Kerry A. and Borgerhoff Mulder, Monique (2012) How national context, project design, and local community characteristics influence success in community-based conservation projects. PNAS, 109(52): 21265-21270
Other references cited above:
Nolte, C., Agrawal, A., Silvius, K.M., and Soares-Filho, B.S. 2013. Governance regime and location influence avoided deforestation success of protected areas in the Brazilian Amazon. PNAS
Paneque-Gálvez J., Mas J.F., Guèze M., Luz A.C., Macía M.J., Orta-Martínez M., Pino J., Reyes-García V. Land tenure and forest cover change. The case of southwestern Beni, Bolivian Amazon, 1986-2009. Applied Geography 43: 113-126.
Porter-Bolland, L., Ellis, E.A., Guariguata, M.R., Ruiz-Mallén, I., Negrete-Yankelevich, S., Reyes-García, V. Community managed forest and forest protected areas: An assessment of their conservation effectiveness across the tropics. Forest Ecology and Management. 268(SI):6–17. 2012.
The Economist (2013) Problems with scientific research: How Science Goes Wrong? October 13th, 2013. Availabe online at: http://www.economist.com/news/leaders/21588069-scientific-research-has-changed-world-now-it-needs-change-itself-how-science-goes-wrong