Improving Validity and Reliability of Licensure Examination for Health Workers

Jhpiego is implementing the USAID-funded Strengthening Human Resources for Health (HRH) project (2012-2017) to support the efforts of the Government of Ethiopia to improve the status of the health workforce. To this end, the project collaborates with a number of local entities. This case study highlights collaboration with the Occupational Assessment and Certification Agency in Amhara Regional State, Ethiopia to assure competence of the health workforce by improving the quality of the licensure exam. The state licensure exam suffers from important weaknesses that affect its validity and reliability. The collaboration started with a consensus-building meeting. A rapid assessment of the assessment practices corroborated the existence of gaps. In response, the project trained 80 item writers who went on to develop 60 validated assessment tools. The team also prepared 330 qualified assessors to administer the examination. To ensure sustainability, the project helped create a technical expert panel and advisory panel responsible for developing and validating items, respectively. 

The critical success factors were the spirit of partnership, the systems strengthening approach, the technical expertise of the implementing organization, linkage of training to quality improvement and presence of strong follow up. As a result of our collaboration, we have expanded pool of quality items and qualified assessors. The collaboration is in early stage but is believed to improve quality and safety of healthcare by ensuring only competent health workers receive the license to practice. The collaboration will continue until all factors affecting the validity and reliability of the licensure exam are addressed.

This case study was submitted as part of USAID's CLA Case Competition, held in August 2015. Taken together, this collection of submissions illustrates the diversity of ways collaborating, learning, and adapting approaches are being operationalized in the field. Stringent judging criteria was used to determine official CLA Case Competition winners, so not all submissions should be considered an official USAID endorsement of best practice. To view all entries, visit the CLA Case Competition page.