What Can the Federal Employee Viewpoint Survey Tell Us About Collaborating, Learning and Adapting in USAID Missions?

The concept of collaborating, learning and adapting (CLA), introduced by USAID’s Bureau for Policy, Planning and Learning (PPL) in 2012, has served to operationalize and strengthen the processes of strategic collaboration, continuous learning and adaptive management throughout USAID’s Program Cycle. In its effort to advance understanding of CLA implementation within USAID missions, as well as CLA’s relationship to specific indicators of organizational effectiveness, PPL/LEARN recently examined CLA-related items in the Federal Employee Viewpoint Survey (FEVS) by mission.

The FEVS, a tool that measures employee perceptions of whether, and to what extent, conditions characterizing successful organizations are present within their agencies, provides feedback on key performance metrics that can drive continuous improvement efforts (OPM, 2016). Focused on employee responses within USAID missions, this analysis of CLA in the FEVS can help PPL/LEARN incorporate employee feedback about CLA implementation into relevant strategies, policies, and services to support missions in fostering the environments needed to accomplish their goals.

This analysis was guided by the following questions:

1. What are the relationships among FEVS items relevant to collaborating, learning and adapting?
2. What is the relationship between CLA and indicators of organizational effectiveness in the FEVS?
3. How have mission scores on CLA-related items changed over time?

Using OPM data about the FEVS provided to USAID’s office of Human Capital and Talent Management (HCTM), this analysis examined percent positive scores (i.e., scores of 5 and 4 on a 5-point Likert-type scale), aggregated by mission. The 2016 sample included more than 3,000 employees in 62 missions, with an average response rate of 63 percent.

The analysis was organized into two parts:

Part 1 examined how CLA was represented as a holistic concept within the FEVS. It used structural equation modeling (SEM) to establish a CLA index made up of seven FEVS items related to collaborating, learning and adapting. It then tested relationships between CLA and indicators of organizational effectiveness, including: Employee Empowerment, Engagement, Satisfaction, and perceived Organizational Effectiveness.

Part 2 assessed how well CLA has been integrated within and across USAID missions by highlighting changes in CLA scores between 2013 and 2016.

This secondary analysis of mission staff responses to the FEVS focused on perceptual and internal measures of organizational effectiveness. The study could not control for limitations in the FEVS data (e.g., response rates or biases) or incomplete representation of the CLA construct in existing FEVS questions. However, the analysis yielded some useful preliminary findings that can help build the evidence base for CLA.

Results from the analysis showed:

• Evidence for a holistic approach to CLA: This analysis provided support for the hypothesized links between collaborating, learning and adapting, demonstrating strong relationships among these variables and showing that they ‘move’ or work together within the context of USAID missions. For example, the analysis indicated that according to mission staff, where managers’ support collaboration and communication more, there are also higher rates of staff cooperation and knowledge sharing. Staff have more knowledge and skills necessary to perform their jobs well, and there is stronger support for innovation and adaptation. The analysis also provides evidence for a holistic approach to CLA. It establishes a robust measure of the multidimensional CLA construct, which allowed us to examine CLA in relation to indicators of organizational effectiveness in the FEVS.
• Strong relationships between CLA and indicators of organizational effectiveness: The relationships between CLA and employee empowerment, engagement, satisfaction and perceived organizational effectiveness proved to be strong, positive, and significant. Missions where employees reported high levels of CLA also reported high levels of these indicators. A growing body of evidence from both quantitative and qualitative studies recognizes engagement, empowerment and satisfaction as critical to successful organizational performance (GAO, 2015). CLA’s strong association with these indicators provides an important foundation for further investigation into the direct and indirect effects of CLA on organizational effectiveness.
• Need to strengthen adaptive management and employee empowerment: In the analysis of CLA-related items, across missions, employees rated collaborating and learning items highest and gave the lowest ratings to adapting. In other words, while employees in missions personally believe they collaborate well and have sufficient knowledge to do a good job, they do not feel encouraged by managers or the general organizational culture to innovate or adapt to improve their work. In the analysis of all FEVS items, mission employees gave the highest ratings to intrinsic motivation for their work: they believe their work is important, constantly look for ways to improve their efforts, and are willing to put in extra effort to get the job done. However, their lowest ratings all related to empowerment. This suggests that despite feeling personally motivated, mission employees do not believe their talents are used well in the organization, nor do they feel personally empowered about work processes, or satisfied by their involvement in decisions that directly affect them. Relatedly, while employees indicated that they personally looked for better ways to do their job, overall they did not feel very supported or encouraged to do so by managers or the organizational culture within their missions.
• Overall increases in CLA ratings between 2013–2016: The 37 missions that participated in the FEVS for both years showed an average increase of 9 percent on CLA-related items. Collaboration items had the largest mean percent increase (11.52 percent), followed by learning (9.7 percent) and adapting (5.9 percent). Across all FEVS items, the largest mean increases over this period were in: 1) merit-based pay raises; 2) meaningful recognition of performance differences; 3) assessment of training needs; 4) satisfaction with new trainings received; and 5) satisfaction with senior leaders’ policies and practices.

The findings of this preliminary FEVS analysis have external use in building the evidence base for a holistic approach to CLA as well as confirming strong relationships between CLA and measures of organizational effectiveness. Based on this analysis, in addition to missions using CLA approaches to improve strategy, project, and activity design and implementation, CLA can also be seen as a leadership tool for creating more effective organizations where employees are more satisfied, engaged, and empowered.