CLA Evidence Collection
Building the Case for Collaborating, Learning and Adapting
For knowledge and learning specialists in international development, it’s very common to get questions about what works within this practice area and what evidence might exist to support claims of effectiveness. To answer these questions for ourselves and others, the Office of Learning, Evaluation and Research in USAID’s Bureau for Policy, Planning and Learning conducted a learning agenda to systematically address: what do we know about how CLA affects organizational effectiveness and development results? The CLA Evidence Collection in this section starts to fill the evidence gaps and synthesize existing evidence from a wide range of disciplines around the practices and approaches associated with collaborating, learning and adapting.
The formal evidence gathering and analysis effort started in 2015, and although the structured learning agenda concluded in 2019, this space continues to act as a living collection of evidence and analysis that sheds light on CLA and its contribution to organizational effectiveness and development results. Some key learning questions that still remain include (among others): what do we understand about resourcing collaborating, learning and adapting?, and how can we measure CLA’s contributions, both at the level of individual activities and especially in the aggregate? If you or your organization are doing work in this area, please contribute your findings! Find out how (link to Call to Action below).
Selected Findings
Research on the impact of strategic collaboration, organizational learning and adaptive management approaches to international development is relatively scarce, scattered and disparate. While methodological challenges have made it difficult to identify solid evidence, and in many cases have prevented researchers and practitioners from even attempting to collect it, two studies provided strong evidence of CLA’s contribution to better development outcomes. Both of these studies were deep dives into cases that initially came through the CLA Case Competition, for which we gathered significant additional evidence to explore whether our initial assessment that such evidence might be available was valid. They are:
- CLA Case Analysis Deep Dive: Zambia's Community-Led Total Sanitation Program
- Global Communities’ Ebola Response in Liberia / CLA Case Analysis: Deep Dive
- More methods
Most of the strongest evidence developed through this Learning Agenda relates to CLA’s contributions to organizational effectiveness; and the evidence of contribution to better development outcomes, aside from that obtained through the deep dive cases, is weaker and more difficult to get at. A review of academic and gray literature revealed a variety of relevant studies on specific aspects of CLA that begin to build an evidence base.
Browse Full Evidence Collection
Click on one of the six components of the CLA Framework below to view a summary of the evidence and a list of all of the articles and studies we’ve compiled on that topic. There, you can filter by more specific CLA approaches.
Methods
In this effort, USAID examined evidence in order to either identify direct impacts or establish plausible contributions of CLA efforts to organizational effectiveness and development results - given that identifying direct contributions is not always possible. As such, a variety of methods have been used to conduct this research, including literature reviews, internal and external learning networks, case analyses and additional studies.
For example, USAID’s LEARN Project undertook a qualitative analysis of case studies submitted through the 2015, 2016, and 2017 CLA Case Competitions to answer two questions:
- What patterns of contribution emerge from an analysis of results chains across strong CLA cases?
- Under what conditions does CLA contribute to improved organizational effectiveness and/or better development outcomes?
To answer the first question, researchers looked at a sample of 83 cases to uncover patterns of contribution emerging from an analysis of results across these cases by using a blended deductive and inductive analysis approach to code the sequence of events and underlying assumptions about how change happened in each case. To determine under what conditions CLA contributes to improved organizational effectiveness and/or better development outcomes, the researchers thematically coded the enablers and barriers that contributed to the specific outcomes of each case. For both questions, they then developed more general results chains by synthesizing the evidence around these contribution patterns.
In a second example, researchers from the LEARN Project conducted an analysis of secondary data from the Federal Employee Viewpoint Survey (FEVS) to answer three questions:
- What are the relationships among FEVS items relevant to collaborating, learning and adapting?
- What is the relationship between CLA and indicators of organizational effectiveness in the FEVS?
- How have mission scores on CLA-related items changed over time?
One method included identifying FEVS items that best represented elements of the CLA Framework and then using Confirmatory Factor Analyses (CFA) and Structural Equation Modeling (SEM) to assess whether the hypothesized relationships among CLA variables held up and therefore whether the hypothesized model of CLA fit the observable data in FEVS. SEM was also used to analyze the factors associated with organizational effectiveness, such as staff Empowerment, Engagement, and Satisfaction, and to what degree they are associated with CLA.
Development practitioners may be interested in adapting these research methods to conduct further analysis.
Contribute to the Evidence Collection
Maintaining and expanding the value and relevance of this evidence collection is a community-wide effort! If you have evidence or analysis that surfaces learning on how collaborating, learning and adapting contributes to organizational effectiveness and/or development results, please submit your content Learning Lab!