Measuring Organizational Performance Improvement: Perspectives from USAID Mission Staff

Measuring the effectiveness of local capacity strengthening (LCS) efforts is key to tracking USAID’s progress towards reaching long-term development outcomes. Capacity is a form of potential and is not visible until exercised. It is through performance, or the exercise of capacity, that local actors demonstrate the achievement of their own development priorities. Therefore, any programmatic considerations regarding LCS must plan for and measure improved performance—not latent capacity. Performance improvement is a key consideration when measuring the effectiveness of capacity strengthening activities.

To learn how USAID Missions are measuring performance improvement—that is, the results of their capacity strengthening activities—we interviewed USAID development practitioners across six Missions representing a diversity of regions, sectors, and capacity strengthening approaches. 

From our interviews, we distilled three key lessons for assessing performance improvement.

Lesson 1: Leverage tools already in use and valued by citizens for meaningful measurement

Local Capacity Strengthening (LCS) is a strategic and intentional investment in the process of partnering with local actors—individuals, organizations, and networks—to jointly improve the performance of a local system to produce locally valued and sustainable development outcomes. 

Municipal government effectiveness and citizen civic engagement are long-standing challenges in the Kyrgyz Republic. To improve government functioning related to citizen-determined priorities, one of USAID/Kyrgyz Republic’s implementing partners provided capacity strengthening support in several local priority areas, such as improving municipal garbage removal services. To assess performance improvements, municipalities solicited feedback from citizens. In addition to feedback on specific areas of municipal services, citizens' feedback response rates were the key data point. Mission and IP staff knew, from experience, that their approach for capturing citizen response rates had to be valued by and accessible to the citizens. For instance, if citizens were required to access a survey platform that is not a part of their daily life, they would be less likely to engage due to the inconvenience. To engage citizens to the greatest extent possible, municipalities used WhatsApp, a popular messenger app, to solicit feedback directly from citizens. Citizen feedback response rates increased, and improved citizen civic engagement continued after completion of the USAID activity in part because municipalities utilized WhatsApp as their regular management practice, better positioning citizens to conveniently communicate with municipal government officials. 

Lesson 2: Use the “Keep It Simple And Sweet” approach in performance metric indicator selection. 

USAID uses a Standard Foreign Assistance Indicator, “Percent of USG-assisted organizations with improved performance (CBLD-9 Capacity Building Indicator),” to track improved performance in organizations assisted through USG-funded capacity strengthening efforts. The indicator criteria require that capacity strengthening responds to locally-defined priorities.

At USAID/Peru, the Mission used the CBLD-9 indicator to capture regional governments’ ability to propose and secure funding for public investment projects. CBLD-9 is USAID’s standard indicator for measuring improved organizational performance in line with local priorities. The Mission used CBLD-9 to its advantage: they used the indicator criteria as an impetus for selecting clear metrics that showed performance improvement, instead of counting simple outputs of  persons trained or other capacity strengthening support. The capacity strengthening activity provided targeted technical assistance to regional government bodies, including equipping them to successfully apply for central government funding for public investment projects. USAID/Peru additionally used the metric ‘number of public investment projects funded’ to assess performance improvement. 

This clear, simple metric was publicly available on the national public investment online system. Regional governments were able to interpret the metric with ease, as opposed to a complicated index tool like the Organizational Capacity Assessment, and citizens were able to understand how well their regional governments performed. The number of public investment projects funded from one year to the next served as evidence of regional government performance improvement. 

This lesson exemplifies Principle 3 of the LCS policy–"Plan for and measure performance improvement in collaboration with local partners." USAID/Peru and the regional government bodies collaborated to use a metric that was locally meaningful. Being deliberate about what was measured and ensuring that local stakeholders could relate to the findings equipped supported organizations and their constituents to assess progress toward performance improvement goals. This approach can contribute to local systems strengthening by improving accountability between local organizations and the people they serve. 

Lesson 3: Co-create measurement approaches with local partners to increase development impact.

USAID's LCS Policy focuses on strengthening and measuring the capacity of local actors and local systems to achieve and sustain development results. The LCS Policy is guided by seven principles for effective local capacity strengthening. Guiding Principle 3: "Plan for and measure performance improvement in collaboration with local partners" is of particular importance to development practitioners seeking to measure organizational performance improvement in a manner that captures changes prioritized by local actors.

Co-creation of performance measures with local actors sets the stage for a mutual understanding of a program’s intended outcomes. Collaborating with local partners to select or develop indicators and data collection approaches can boost local partners’ buy-in for monitoring processes. These approaches prioritize local partners’ ability to track their own progress towards their performance objectives –rather than these efforts serving only as a reporting exercise for USAID.

Co-creation is a design approach that brings people together to collectively produce a mutually valued outcome, using a participatory process that assumes some degree of shared power and decision-making.

At USAID/Paraguay, co-creation of measurement approaches with local actors contributed to the development of useful indicators that informed decision-making and local accountability. A local Paraguayan NGO, Fundación Paraguaya (FP), planned to provide capacity strengthening support to committees of low-income women entrepreneurs. FP leveraged the CBLD-9 indicator requirement to obtain input from supported organizations as an impetus for working with the women’s committees to shape the measurement approach.

To meet the CBLD-9 requirement of obtaining input from the supported organizations, FP facilitated focus group discussions with the supported women’s committees, who expressed the need for committee solidarity–namely, meeting together more frequently, developing common goals, and improving teamwork. FP facilitated the design of a “Committee Strengthening Stoplight” rating tool that helped the committees self-assess their performance gaps. For example, in areas for which a committee self-assessed as “green,” they were able to recognize their strengths, and in areas for which they self-assessed as “red” or “yellow,” they were able to identify their performance gaps. This tool aided the action planning process, in which the women’s committees discussed what they needed to do to achieve their self-identified priorities (for example, committee solidarity, formalization through government channels, improved conflict resolution processes). By working with the women’s committees to determine what to measure, and devolving power for initial assessment (and subsequent action planning) to the committees themselves, FP helped position the local committees to manage for their own results.

Read the entire case study here. 

See Program Cycle Discussion Note: Co-Creation Additional Help

Learn More: To find USAID resources on measuring organizational capacity, check out this Capacity Building Indicator Resource page. To explore USAID’s framework and guiding principles for local capacity strengthening, read the Local Capacity Strengthening Policy.

Many thanks to Timur from USAID/Kyrgz Republic, Idriss from USAID/Morocco, Nina and David from USAID/Georgia, Laura from USAID/Paraguay, Radhika, Maneka, Sabita, Jaganath, Gajendra, and Bishwas from USAID/Nepal, and Bea and Miriam from USAID/Peru  for their important insights. Many thanks to USAID’s LFT Hub and USAID/PPL, more specifically Colleen, Amanda, Elliot, Zach, Van, Eskedar, Danielle and Janet, for their guidance, support and feedback.