Compendium on Impact Assessment of ICT-for-Development Projects

Written by personnel from the University of Manchester's Centre for Development Informatics with funding from the International Development Research Centre (IDRC), this compendium presents a set of frameworks that can be used by information for communication technology for development (ICT4D) practitioners, policymakers, and consultants to understand the impact of informatics initiatives - e.g., telecentres, village phone schemes, e-health and e-education projects, and e-government kiosks - in developing countries.

It begins with an overview that explains the basis for understanding impact assessment of ICT4D projects, noting that this assessment can be based around six questions:

1. Why: what is the rationale for impact assessment?
2. For whom: who is the intended audience for the impact assessment?
3. What: what is to be measured?
4. How 1: how are the selected indicators to be measured?
5. When: at what point in the ICT4D project lifecycle are indicators to be measured?
6. How 2: how are impact assessment results to be reported, disseminated and used?

Figure 2 in this overview section of the report shows the guiding model for impact assessment: what the author's term the ICT4D Value Chain. This builds on a standard input-process-output model to create a sequence of linked ICT4D resources and processes. It is divided into 4 main targets for assessment:

1. Readiness: measures the systemic prerequisites for any ICT4D initiative - e.g. presence of ICT infrastructure, ICT skills, and ICT policies. One could also assess the strategy that turns these precursors into project specific inputs, and the presence/absence of those inputs.
2. Availability: assesses the presence and availability of intermediates/deliverables - telecentres, shared telephony, etc.
3. Uptake: measures the extent to which the project's ICT deliverables are being used by its intended population. Broader assessment could look at the sustainability of this use over time and at the potential or actuality of scale-up.
4. Impact: assesses outputs (the micro-level behavioural changes associated with the ICT4D project), outcomes (the specific costs and benefits), and development impacts (the contribution of the ICT4D project to broader development goals).

Next, the authors provide an overview of the different impact assessment frameworks that can be used, which they classify into 6 categories: generic (e.g., cost-benefit analysis, project goals); discipline-specific (e.g., communication studies, development studies, information science, sociology); issue-specific (e.g., gender, enterprise - growth; application-specific (e.g., telecentres); method-specific, and sector-specific. The compendium focuses on the first 4 categories.

Table 2 on page 11 of the compendium offers a methodological overview of the ICT4D impact assessment frameworks.

The second part of the compendium summarises a series of impact assessment frameworks. For each, the compendium entry covers:

• Summary: a one-paragraph overview of the framework.
• The Framework: an explanation of the origins and content of the particular approach, explaining how it would organise ICT4D impact assessment data and knowledge.
• SW Analysis: a summary of the perceived strengths and weaknesses of this approach to impact assessment.
• Methodological Summary: an overview of the nature and requirements of data-gathering using this framework.
• Method Recommendations: good practice notes on applying the framework.
• References: literature sources referred to in the entry.
• Bibliography: additional key literature sources, where found.
• Variants: variations on the main framework that may be used in ICT4D impact assessment.
• Examples of Use: summarised examples of applying the framework to ICT4D project assessment.

The frameworks are:

1. Cost-Benefit Analysis (CBA)
2. Project Goals
3. Communications-for-Development
4. Capabilites (Sen) Framework
5. Livelihoods Framework
6. Information Economics
7. Information Needs/Mapping
8. Cultural-Institutional Framework
9. Enterprise: Variables, Relations, Value Chain
10. Gender
11. Telecentres

The final part of the compendium is an ICT4D impact assessment bibliography - a summary of literature, including many real-world case studies. Each entry summarises 5 things: framework type, reference information, value chain stage the document mainly focuses on, the depth of information provided in the document about the actual methods used, and a short summary/opinion about the document.

Also available is an educator's guide to student questions for the compendium.

The compendium was a feed-in to the joint IDRC/Gates Foundation Institute of Public and International Affairs (IPAI) project - a five-year project on impact assessment of public access to ICT.