Performance Results Are Not the Whole Story: Using Non-Traditional MEL and Targeting Cohesion in Collective Action Programs

Development practitioners have come to realize that the challenges facing communities where we work are anything but straightforward. When we consider the often dynamic and complex environments in which these challenges are situated, it’s clear that simple solutions and approaches do not adequately address them. These development challenges typically exceed the mandates of any individual organization or institution involved, and non-aligned efforts by different stakeholders can drain limited resources and produce detrimental effects. 

To account for this, USAID and its implementing partners increasingly use collaborative approaches like Collective Action to bring together diverse and interdependent actors to address shared challenges and achieve shared objectives. Collective Action is an application of systems thinking since it uses a lens that focuses on the interrelationships of actors connected to a development challenge. This approach, while recognized as effective, is not easily aligned with traditional MEL frameworks, which are built around known or hypothesized cause and effect relationships. 

Recently released guidance on Collective Action in USAID programs highlights that in order to achieve progress toward shared development issues, a collective action coalition needs to intentionally focus as much on cohesion among the members as on progress toward resolving the issue. This, for example, is an opportunity to build in indicators and questions around cohesion and collective capacity rather than only focusing on short-term technical results. Case in point — technical results alone do not tell us the full story. 

So if traditional MEL approaches are not the answer, what could help?  Complexity-aware monitoring offers approaches on how to conduct M&E in complex and dynamic environments and is intended to  complement and enhance  performance  monitoring, USAID’s standard monitoring approach. Complexity aware monitoring is most useful when results are difficult to predict and take into account the inherently uncertain and changing nature of complex situations, where agreement on the development problem is low, and certainty about how to solve the problem is also low. 

In these instances, we need to supplement the story told by performance monitoring to support adaptive management.  To aid adaptive management, complexity-aware monitoring strives to understand interrelationships, engage with multiple perspectives, and reflect on boundary judgments and adjust monitoring to synchronize with the pace  of change.

But what approaches do we use? Well that depends on the needs! There are several complexity-aware monitoring methodologies to choose from and they should be chosen based on your theory of change, the evidence you are trying to capture at what stage (e.g., design and planning, implementation, post and final evaluation, etc.) and the data sources (i.e., qualitative, quantitative), how evidence will be used, and any other operational and contextual considerations to keep in mind. You may even use a combination of approaches. 

Module Nine of the Practical Guide for USAID Missions examines the application and use of complexity-aware monitoring methodologies to understand and improve the effective implementation of Collective Action. In addition, the guide links to resources that provide further details on complexity-aware tools and methods. 

Now that we know a little more about complexity-aware monitoring, let’s talk through some practical tips: 

• At first, monitoring unpredictable change pathways while keeping tabs on a broad range of actors and factors that influence the program may seem impossible. But try visualizing and mapping the actors and factors that affect the system. Using causal-loop diagrams, social network analyses and other similar tools can help to articulate the interrelationships and direction of influence the actors and factors have. Understanding the relationships and directionality can help us better identify points of leverage and opportunities and key barriers for entry. 
• Second, you will want to create boundaries and prioritize based on your key areas of focus in the theory of change. But be aware that priorities may change over time, so staying up to date and adjusting as necessary are critical. 
• Next, remember to include a diverse range of perspectives as you analyze and interpret the data because you never know what you may have missed and you don’t want to miss an opportunity to gain a new understanding. 
• Lastly, be prepared to add flexibility to your MEL timetable. In complex situations, plans to monitor, evaluate and learn may need to take place very frequently or less often than expected depending on the context. It is likely that plans will need to include both short- and long-term time horizons and that frequencies will be decided as appropriate. 

If the Collective Action approach is new to you and you want to learn more, check out the Collective Action Guides on Learning Lab. Resources include answers to common questions about Collective Action, case studies from USAID activities as well as practical guides to Collective Action which illustrate the processes, know-how, and considerations for embarking on a Collective Action journey.

----

The Collective Action Blog Series features posts from guest authors highlighting how Collective Action relates to their work, content of the guides, findings from the research, and connections to other USAID frameworks and approaches. Please send your thoughts and feedback [email protected] with "Collective Action Feedback" in the subject line. We would love to hear from you.