The Drum Beat 490 - Virtual Change, Part I

Since the early 1990s, information and communication technology (ICT) has increasingly been used - in increasingly diverse ways - to accelerate and improve development action around the world. Members of the international social change community have proceeded with great hope to integrate ICTs into their efforts to enable the exchange, processing, and management of information and knowledge. Numerous case studies have explored myriad examples of these intersections, such as fishermen and women obtaining up-to-the-minute information on market prices for their catches and women entrepreneurs setting up rural internet kiosks powered by local energy sources such as wind and water mills.

Thirty years later, the question examined in a February 2009 paper, written by Warren Feek of The Communication Initiative and published by FAO's Research and Extension Division, Natural Resources and Environment Department, is this: how do we measure the impact of ICT-based interventions and within which framework would those indicators work? "Virtual Change: Indicators for Assessing the Impact of ICTs in Development Research and Extension Division" extends a series of core communication indicators for assessing the impact of activities and projects using ICTs for development and suggests appropriate evaluation methodologies for each indicator.

This issue of the Drum Beat is the first part of a two-part series. It draws out some of the key elements of "Virtual Change: Indicators for Assessing the Impact of ICTs in Development Research and Extension Division". To access the full document [PDF], click here.

A summary of the document may be accessed within our Strategic Thinking section. 

We ask for your help in creating Part II, to be published next week. Please send us examples of ICT for development (ICT4D) impact, which illustrate one or more of the indicators you will read about below. Relevant submissions will make an appearance in Part II (as space allows), along with other examples from your peers and from The CI knowledge sections. Send details to drumbeat@comminit.com

The Drum Beat 490 contains:

•  CONTEXT: ICT indicators. 
•  All of The CI's ICT4D knowledge in one place - the ICT4D THEME SITE. 
•  Vote in a POLL on e-health research and development. 
•  Understanding THE INDICATORS. 
•  Become a CI ASSOCIATE! 
•  Intersections and CONCLUSIONS. 
•  INVITATION: Send us your ICT impact examples!

BACKGROUND AND CONTEXT: INDICATORS FOR ICT IMPACT

1. The Challenge

"...substantial resources have flowed from the international development community to support expansion of access to new technologies in financially poor countries and more extensive use of those technologies for improved development impact. But how...can we demonstrate that the efforts of a farmer or any of their peers around the world developing equally creative initiatives, have not been for nought? Accurate measurement requires 'astute' indicators. Which indicators (the small bits from the big picture) will produce the sharpest information regarding the overall contribution (positive, negative or neutral) of ICTs on development status?...The paper is based on a short desk study; there is no original research. It draws from the practice and thinking already in circulation, looks at the information and analysis and produces a set of indicators that can be justified by experience and argument." ["Virtual Change," pages 1-2]

2. The Data

"The task of defining core indicators to demonstrate the impact of ICT interventions on overall development status would be greatly eased by the availability of quality data - quantitative and qualitative. Such data is difficult to find even in the most advanced technology countries....This study shows that, even in the multi-billion dollar world of eHealth in the most sophisticated technology country in the world, there is scant data and what data there is, is considered unreliable. It is therefore not surprising that there is little reliable data evaluating the application of ICTs to thorny development issues such as rural poverty, gender equity and HIV/AIDS....Most of the documents reviewed for this paper could best be characterized as opinion pieces or personal perspectives and testimonies..." ["Virtual Change," pages 2-3]

3. The Rationale

"...ICTs...are simply tools. There is nothing inherent in the new technologies that prompts positive development....Even the one-to-one and many-to-many interactive capacities of the new technologies, which many cite as providing an essential positive dynamic for development, can be misused for other purposes. It is insufficient to measure, for example, household or village access to the Internet as a predictor of positive future social and economic progress. What if the headman in the village or the senior male in the family monopolizes use and information flow?" ["Virtual Change," page 3]

4. Schools of Thought

Warren Feek lays the groundwork for his series of indicators, exploring 2 schools of thought: FAO's Participatory Rural Communication Appraisal approach, and the communication for social change strategic thinking facilitated by The Rockefeller Foundation. He provides 3 main reasons for choosing these models:

a. Both the change models and their strategic thinking derive from a combination of experiences of communication for development practitioners and of research and evaluation data.  

b. Both approaches share an emphasis on community engagement and management, participation, empowerment, local capacities, ownership, and negotiation between vested interest groups.  

c. Communication for Social Change is a comparatively recent theoretical and strategic approach to communication for development, while the Participatory Rural Communication Appraisal approach is drawn from the long-standing participatory rural appraisal strategies. "This combination of new and established thinking provides a credible lens." ["Virtual Change," page 4]

•  Related summaries from The CI website:

Participatory Rural Communication Appraisal (PRCA)

Evaluating Social Change and Communication For Social Change: New Perspectives

Monitoring and Indicators for Communication for Development

Measuring and Evaluating Communication for Social Change

Can New Technology Promote Dialogue?

[top]

You will find a rich array of ICT for Development materials on our dedicated theme site -  where you may also contribute to various debates through our poll.

If you'd like to receive our bimonthly ICT4D mailing, please email it4d@comminit.com with your request.

[top]

Please VOTE in our current ICT4D Poll:

In what direction should current e-Health research and technical development go?

Direction:

•  Diagnosing through mobile phones. 
•  Certifying phone services as coming from authentic health providers. 
•  Building a menu of types of health providers (e.g., MD, traditional, clinic, pharmacy) into mobiles.
•  Linking remote clinics with specialists. 
•  Linking communities in "the last mile" with hospital or clinic diagnosis and care centres.

VOTE and COMMENT click here.

[top]

THE INDICATORS

5. Criteria for Indicator Selection

"The basic requirement is to establish a connection between a given development intervention (e.g. increased internet access) and changes in the area of development in question (e.g. improved governance)....Secondly, there is a need to link a specific strategy to specific outcomes within the overall issue being addressed." With these demands in mind, Feek has identified a series of indicators based on the 2 aforementioned overall social change development strategies, which he states have proven impact. He stresses that they are informed (and reinforced) by the perspectives emerging from the ICT evaluation literature. They also provide short-term measurements that predict long-term change, are simple and practical, and are applicable across the full range of development contexts. ["Virtual Change," page 8]

6. The 18 indicators may be grouped as follows:

I. Holistic dialogue: "If there is no dialogue there is no development....From a development perspective, holistic refers to a full and varied assessment of both the situation and the options for action, from a range of perspectives..." ["Virtual Change," pages 9-10] Feek's review of the literature reveals 3 ways in which ICT has the potential to greatly expand dialogue: a) ICTs can serve as a community meeting space for feedback and permanent learning, one where two-way and horizontal communication can take place with few intermediaries. b) ICTs can function as an efficient means of knowledge sharing. c) ICTs ensure that anyone can access the information and contribute to the dialogue from anywhere.

Indicators:  

i. The ICTs are increasingly used for dialogue and debate.

ii. Policy and programme knowledge is increasingly communicated through the ICTs.

iii. There are increased levels of access to the ICT processes.

II. Community and individual voice: "When major development successes are assessed, there is a clear correlation between the prominence of the voices of those most affected by the issues in question and the effectiveness of the action....Specific groups of people, those most often marginalized both in communities and in the development process, are highlighted as being vital to engage for effective use of ICTs." ["Virtual Change," pages 13-14]

Indicators:

iv. The opinions and ideas expressed through ICT channels are increasingly those of the people most affected by development issues in any given context.

v. The people most affected by development issues in any given context increasingly dominate the physical use of the ICTs.

vi. Technical experts on ICT for development increasingly respond to and implement the technical requirements voiced by those most affected.

III. Participatory decision-making: "Lack of involvement of the people most affected is a consistent failing in development programming....The added value of ICTs for enhancing the engagement of the people most affected in decision-making about action on the issues that most concern them is reflected in the ICT literature at two levels: decision-making about the priority use and development of the ICTs themselves; and using the ICTs to engage more people centrally affected by development issues in overall decision-making processes." ["Virtual Change," pages 15-17]

Indicators:

vii. A minimum of 40% of the people involved in the management are directly affected by the development issues that the ICTs being mobilised are designed to address.

viii. There are x (the number inserted here depends on the scale and nature of the programme being evaluated) examples in the last 12 months of the use of the ICTs for engaging people directly affected by development issues in overall programme management and/or policy development.

IV. Building communication platforms: "The approach to development projects by donors and international agencies is predominantly narrow; projects are designed to address particular issues....The platform approach takes an entirely different communication tack. Rather than attempting to directly and discretely address an issue, a communication platform seeks...to: Provide a general communication base that allows for consideration of and action on a full range of development issues...[and] Provide, communicate and facilitate the essential information that people and organizations require to be more effective in their work..." Feek indicates that researchers, commentators, and policy analysts agree that "the new ICTs provide an excellent means to operationalize" these platforms when it comes to: sharing knowledge, developing communication processes within the institutions, establishing the "spaces" to communicate, and recognising the strongest characteristics of each communication medium." ["Virtual Change," pages 19-20]

Indicators:

ix. The ICTs are increasingly used to draw relationships between different development issues.

x. The ICTs are increasingly used as a communication platform to identify and negotiate the specific strategic and technical support that development organisations require.

xi. The ICTs are increasingly used as the source for the core information needed to better inform individual development activities.

xii. The ICTs are increasingly used as the gathering point for like-focused organisations and groups.

V. Change symbols: "symbols and images that serve the very powerful purposes of highlighting the 'cause', embodying the case and provoking the necessary dialogue and debate....ICTs provide some distinct advantages for highlighting, multiplying and conveying meaning related to the symbols and images that are core components of any social change strategy..." ["Virtual Change," pages 22-23]

Indicators:

xiii. The ICTs are increasingly used to highlight emerging symbols and images related to action on the development issue(s) in question.

xiv. The ICTs are increasingly used to multiply strong symbols and/or images that are emerging from the struggle.

xv. The ICTs are increasingly used to both convey meaning and deepen debate and dialogue through the symbols and images presented.

VI. Working alliances: "Effective and sustainable change requires an alliance of interests rather than a single, centralized programme of action....The writers reviewed expressed a strong belief that ICTs add substantively to the process of partnership building, as well as contributing to attaining higher levels of influence, to enhancing social support, to networking and to building consensus..." ["Virtual Change," page 24]

Indicators:

xvi. The ICTs are increasingly used to build working strategic and/or operational partnerships with other organisations that have similar vested interests.

xvii. The ICTs are increasingly used to participate in networks of like-focused organisations.

xviii. The ICTs are increasingly used to both provide support to others involved in compatible action and to receive support from such organisations.

[top]

BECOME A CI ASSOCIATE!

Please consider the possibility of supporting The CI's work through the CI Associates process - details and sign-up click here.  Also see Warren Feek's note in The Drum Beat 485. Thank you.

[top]

INTERSECTIONS and CONCLUSION

7. Relationship between the Indicators

Feek explains - and illustrates, through a diagram on page 26 - that little will be gained from a massive increase in one element of the process - for example, just focusing on getting as many people directly affected as possible involved in the management of the programme or initiative. Attention to that element of the process needs to be matched by corresponding improvements in action in the other 5 areas. That is, there must be a balanced and harmonious relationship between all the elements above.

8. Methodologies for Indicator Measurement

Feek uses the first 4 categories of indicators to highlight the possible ways in which evaluation methodologies can be used to collect the relevant information. He examines the following methods:

a. content analysis

b. structured and unstructured interviews

c. surveys and questionnaires, and

d. participant and non-participant observation ["Virtual Change," pages 29-34]

9. Conclusion

Feek concludes with 2 core observations. First, "[t]here is a chicken-and-egg quality to the perspective presented above. Which should come first when developing indicators to measure the impact of ICTs on development issues: the evidence from research or the assertions from experience and thinking? He explains that, though there is no compelling data at this time, the assertions are nonetheless important because they provide one framework through which the research can be funnelled. Also, Feek recognises that "[w]hen people and organizations know the criteria by which they will be assessed, they tend to work to strengthen those factors....If the dominant thinking assesses ICT developments according to these evaluation indicators, this will drive programming towards those elements." ["Virtual Change," pages 34-35]

[top]

To access "Virtual Change: Indicators for Assessing the Impact of ICTs in Development Research and Extension Division" [PDF], please click here.

For further information about "Virtual Change: Indicators for Assessing the Impact of ICTs in Development Research and Extension Division", please contact: Clare O'Farrell, Communication for Development Officer, FAO at comdev@fao.org

SEND US ICT4D IMPACT EXAMPLES

We want to hear from you! Do you have reports or examples to share of how your organisation has evaluated the impact of ICT in a social change context? Please send information about the indicator(s) you have drawn on in your work to drumbeat@comminit.com as soon as possible for potential inclusion in Part II of this Drum Beat!

[top]

This issue of The Drum Beat was written by Kier Olsen DeVries.

The Drum Beat seeks to cover the full range of communication for development activities. Inclusion of an item does not imply endorsement or support by The Partners.

Please send material for The Drum Beat to the Editor - Deborah Heimann dheimann@comminit.com

To reproduce any portion of The Drum Beat, click here for our policy.

To subscribe, click here.