Health Research Capacity Strengthening Snapshots

"Strengthening health research capacity in low- and middle-income countries (LMIC) is a recognized way to advance health and development. However, systematic evidence on the effectiveness of different approaches remains limited, as their complexity and diversity make monitoring and evaluation (M&E) difficult."

To address that challenge, each of these 4-page documents gleans out key points from academic papers (available only by subscription via the journals Health Policy & Planning (Snapshot 1) and Health Research Policy and Systems (Snapshots 2 and 3)) that explore how M&E of health research capacity strengthening (RCS) takes place and what can be done to improve it. The research shared here emerged from a partnership between the Liverpool School of Tropical Medicine (LSTM) and the ESSENCE (Enhancing Support for Strengthening the Effectiveness of National Capacity Efforts) on Health Research Initiative, hosted by the Special Programme for Research and Training in Tropical Diseases (TDR).

Beginning in 2011, ESSENCE identified M&E of health RCS as an area for potential harmonisation and alignment of good practices, in accordance with aid effectiveness principles. Based on a systematic analysis of 18 evaluation reports, representing 12 evaluations undertaken between 2000 and 2013, ESSENCE produced a Planning, Monitoring and Evaluation (PM&E) Framework as a guide for their members and grantees and encouraged the sharing of lessons about health RCS evaluations. In short, researchers found that, "if evaluations are to be meaningful to all stakeholders, the participation of researchers and funding beneficiaries in LMIC is vital. Yet donors often need speedy evaluations to demonstrate accountability and value for money. This raises the question of how to share wider learning about health research capacity strengthening..." Thus, the snapshots include:

1. Snapshot 1: Tensions in Evaluating Health Research Capacity Strengthening [PDF] - Selected communication-related findings/recommendations:
   • Lack of information about the intended use of evaluations can lead to different expectations from the funders, funding recipients, and evaluators. Thus, it is important to define the explicit purpose and intended use of the health RCS evaluations to be conducted.
   • Involve funding recipients and other stakeholders in all stages of the evaluation process to encourage learning, ensure a feeling of ownership over the project, and facilitate implementation of recommendations.
   • If evaluators (found in the analysis to be most of them) do not have enough time to incorporate a theory of change (a description of the relationships between activities, outputs, and outcomes) and/or theory-informed indicators of impact and sustainability, there will be missed opportunities for funders and recipients to learn about how improve planning, monitoring, and evaluation of health RCS initiatives.
   • Research funders should ensure that capacity-building initiatives become self-sustaining, described here as more likely to be achieved via a developmental participatory evaluation than a traditional summative evaluation.
   • Document lessons systematically and share them on an ongoing basis.
2. Snapshot 2: Frameworks for Evaluating Health Research Capacity Strengthening [PDF] - One selected finding is that the ESSENCE PM&E Framework has not been widely used in practice for various reasons (e.g., other guidelines were already in place or were more appropriate for specific projects), yet the issue of harmonising PM&E practices and frameworks between organisations appears to be increasingly recognised. One communication-related lesson: "Ideally, frameworks for planning, monitoring and evaluating health RCS should be easily accessible to stakeholders and facilitate high-quality data collection and analysis....Additional information, combined with training, could be important for funders to provide if they want to encourage the participation of stakeholders in the evaluation process."





3. Snapshot 3: Indicators for Evaluating Health Research Capacity Strengthening [PDF] - The analysis found that funders used a wide range of indicators to track progress and measure impact, yet few reports explicitly defined indicators. Furthermore, "[t]he evaluations did not consider some important aspects of health RCS, particularly ongoing relationships among RCS stakeholders to facilitate, conduct and use research." Recommendations are offered to address these weaknesses, such as investing in science communication, developing indicators to better encompass relationships between researchers and knowledge users, and paying attention to different levels of the research environment, such as research networks.