The Drum Beat 447 - Trends in Impact Evaluation of C4D [Part 3]

This is the third and final issue of The Drum Beat focusing on World Bank Working Paper (#120), published in June 2007, "Communicating the Impact of Communication for Development: Recent Trends in Empirical Research" by Nobuya Inagaki. The study is a survey of empirical research on communication for development based on a sample of peer-reviewed English-language articles from academic journals published between 2000 and 2005. The purpose of the analysis was to collect evidence from peer-reviewed academic research to highlight the impacts of communication on development initiatives and to present current trends in theoretical underpinnings and communication approaches.

Below we present the final excerpts from the study, with a focus on Chapters 5 and 6, looking at discussion of the trends and evidence and presenting conclusions. Part 2 of this 3-part Drum Beat series focused on Chapters 1-4, looking at theory, trends, and evidence – click here for that issue online. Part 1 of this series reviewed the studies that served as background evidence for Mr. Inagaki's analysis – click here for that issue online.

===

Please note: The Communication Initiative (CI) worked with Mr. Inagaki in designing the research approach and supporting the research that was the background for the paper. The conclusions within the paper are Mr. Inagaki's and are not necessarily representative of The CI or the Partners of The CI.

You may download a PDF version of the full study - click here.

You may download a plain text version of the full study - click here.

Also note: footnotes and internal references have been removed from the below excerpts. For the full text, including references, please see the full study.

===

DEBATE!

This issue of The Drum Beat will serve as food-for-thought for the continuing discussion among the Drum Beat Chat network of this World Bank Working Paper (#120), "Communicating the Impact of Communication for Development: Recent Trends in Empirical Research" by Nobuya Inagaki.

Please register (if you are not already registered) and engage in dialogue, which began on June 10th, through the DrumBeatChat forum: click here for the archives. Register, and either participate online or send your contributions via email to drumbeatchat@comminit.com (you must be registered to participate). If connectivity is an issue for you, you may also send your contact information via email to the moderator, Deborah Heimann - dheimann@comminit.com - who can assist you with the registration process.

===

Communicating the Impact of Communication for Development: Recent Trends in Empirical Research World Bank Working Paper #120 by Nobuya Inagaki

EXCERPTS [Chapters 5 and 6]:

Chapter 5 - Discussion

Evidence at Hand

Interpersonal communication is the single most researched communication approach, and constitutes the core strategy in diffusion and participatory projects. In projects informed by diffusion theory, interpersonal communication is often co-employed along with some form of vertical communication channel. In the diffusion theory framework, the expected function of interpersonal communication is primarily to amplify and relay the messages to peers, spouses and clients. In participatory projects, interpersonal communication (including horizontal group communication) has a generative function, in addition to being a vehicle for transferring information and knowledge. Some of the reviewed studies demonstrated that horizontal communication among participants allowed people to identify and acknowledge common community problems, to become self-reflexive, to produce bottom-up solutions to identified problems, and to rejuvenate their commitment to social causes to improve their well-being.

Vertical transmission of information through mass media is still a popular strategy. This communication strategy, despite its affinity with the widely criticized modernization paradigm, not only prevails in many recent communication interventions, but also proves itself a viable approach as demonstrated by many studies showing positive outcomes using such interventions. Several studies suggested that repeated exposure to media messages is important to produce direct impacts. Within this mode of communication, entertainment-education and social marketing are two of the most systematically investigated strategic communications among published empirical research. Research in mass media-based development communications seems to indicate that parallel use of vertical and horizontal modes of communication are widespread, as seen in the many diffusion projects.

Traditional development goals - induced changes in behavior, attitude and knowledge - are pursued in a number of development projects and actively investigated in recent empirical research. The underlying assumptions of these strategic goals can reflect a form of paternalism; projects implement externally defined ideals in development country settings that may not share historical-cultural conditions in which international aid discourse originally emerged (industrialized North). This interpretation poses a challenge when evaluating participatory projects, which, like modernization and diffusion projects, often set the parameters of communication as well as the desired outcome. In principle, participatory projects are open-ended and rely on organic emergence of development vectors and substance through participatory engagement of diverse voices. In reality, many participatory projects are goal-oriented, framing participation as a strategic instrument to achieve pre-determined outcomes. The problem of assessing participatory processes is compounded by the relative lack of clear empirical documentation of the extent of participation that actually took place.

Partially clearing up our skepticism for participatory projects, programmatic outcomes that are more generative than inductive - empowerment, coalition building and resource generation - are the domains in which participatory projects made much greater contributions than modernization or diffusion projects. Traditionally, projects based on the modernization and diffusion theories are designed in a way to relay messages working through the individual psychology of the receivers of information; less attention is given to types of effects that occur in collective and social levels. Papa and others wrote:

"Most past communication research on media effects consists of surveys or experiments in which unconnected individuals are the units of response and the units of analysis. Such research designs necessarily make it difficult to investigate the social context in which media effects occur, and concepts like collective efficacy and community action cannot easily be measured."

This said, these goals aimed at collective and generative effects were not entirely absent in the projects that principally employed vertically orientated communication. For example, the Soul City project in South Africa and a Nepalese entertainment-education project for better village life both promoted community mobilizations through mass media messages. These examples may indicate a new direction of communication intervention toward integrated strategic and theoretical approaches. In her review of recent diffusion and participation projects, Nancy Morris concludes that communication experts and researchers have started to ponder and experiment with the crosspollination between the diffusion and participatory approaches.

Integrated communication strategies are widely used in communication initiatives. Evaluations of integrated communication strategies showed varying impacts attributable to not only different modes of communication (for example, vertical and horizontal communications), but also particular channels of communication within each mode (for example, broadcast media versus printed materials, one-to-one communication versus group communication). Concomitant use of multiple communication channels may be implemented to reduce the risk of failure. In addition, empirical studies suggest the evidence of interaction among different communication channels and modes. The overall communication effects thus may reflect a synergetic process of direct and indirect effects of individual communication components. Unfortunately, the mechanism of integrated communication effects have not been fully explored in published evaluations. A little over half of the empirical studies in our sample evaluated integrated communication projects, but very few of them actually offered systematic investigation of the mechanisms of inter-modal and inter-channel interactions. [pps. 37-39]

Blind Spots in Recent Research

Blind Spot 1: Evidence of Failure

[I]t is probably erroneous to assume that published evidence actually corresponds to the performance of other, unreported communication interventions. It is often pointed out that published studies tend to deal with projects that had significant outcomes, with an effect of under-reporting failed projects. Ironically, tracking the source of project failures is a fruitful exercise because it allows practitioners and researchers to empirically identify sources of failure. Such knowledge can feed back into more effective planning and implementation of projects in the future.... [p. 39]

Blind Spot 2: Structural Blockages to Communication Interventions and Research Methodologies

[T]he issue of power is a common cause of unsuccessful outcomes in these interventions; power imbalances in political, economic, occupational and gender domains created blockages to communication across social boundaries. Another commonality is that these failed experiences were identified through qualitative evaluation methods. There were no quantitatively oriented studies that deemed projects overwhelmingly unsuccessful, although some quantitative studies identified less successful components within the overall project implementation.... [p. 40]

Blind Spot 3: Sustainability of Communication's Impacts

Two factors seem to be associated with the lack of investigations into sustainable communication interventions. First, most of the project implementation schedules are too short if one tries to gauge long-term impacts during or within the timeframe of the projects.... Second, recalling the issue raised in the methodological notes for the present work, many of the researchers authoring academic evaluations also play the role of communication consultants within the projects they subsequently evaluate.... [p. 41]

===

VOTE in our Democracy and Governance POLL!

Which of the following contributions on the part of media is the most supportive of substantive democratic governance processes?:

• Watchdog on government actions
• Supplier of public information
• Forum for public debate
• Provider of political analysis
• Protector of citizens' rights
• Interface between people and politicians
• Mobiliser of political involvement
• Alternative forum to formal processes
• There is no definitive contribution from media
• Other [If registered, please comment in the form provided. You must be a registered member to comment. Please register here]

VOTE - Top Right Side of the website.

===

Chapter 6 - Conclusion

[T]he following areas are awaiting greater attention by development communication researchers.

Communicating the Impact of Communication for Development

Efforts must be made to emphasize and to improve the operational function of communication in the field of international development assistance.... [There are] a number of empirical evaluations of project performance toward substantive development objectives (health, sustainable development, education, poverty); many of them acknowledge the importance of communication in the project design but stopped short of engaging in systematic analysis of how communication affected or did not affect the overall project performance.

For the researchers of these studies, communication is an abstract concept not easily rendered operational. The perceived conceptual fuzziness of communication among development practitioners and researchers calls for a greater effort in communicating communication's impacts.... [pps. 43-44]

Attention to Structural Problems

Academic research in development communication needs to engage more fully with larger structural issues that may not be adequately addressed in project evaluations conducted as part of individual assistance projects.... Many of the reviewed studies do indeed acknowledge deep power issues afflicting the quality of lives of people, but a number of them - particularly those investigating vertical, top-down media campaigns - ultimately treated such structural issues as exogenous factors to which very little can be done through communication interventions. [T]here seems to still exist a peculiar disjuncture between empirical research and critical theoretical discourses, at least among the empirical studies published in the mainstream development studies journals.

Such a disjuncture may put the legitimacy of the scholarship into question. If reduced to purely instrumental research interests, academic research will lose its authenticity as a voice questioning fundamental development problems such as racism, structural poverty, political economy of malnutrition and diseases, commercial exploitation of indigenous and natural resources, and international conflicts. For example, communication campaigns for condom use may lessen the threats of sexually transmitted diseases; but these campaigns alone cannot address the root causes of STD in poor countries - namely, gender inequality and sexual predation, poor health care resources, and male seasonal workers in urban areas. Academic research is called for in order to put forward empirical discourses linking larger structural issues to the immediate concerns of the life circumstances of people.... [p. 44]

Methodological Integration

Finally, development communication research needs to address the gaps among different methodological paradigms in order to advance more holistic understanding of communication processes in international development settings. We earlier discussed the presence of intertwined methodological and theoretical biases in the recent empirical literature. The evaluations using quantitative methods and those investigating vertical communication strategies did not, or failed to, capture project failures caused by power inequality. We also discussed that quantitative research methods were employed extensively in the evaluations of vertical communication strategies (such as social marketing) but very little in those investigating horizontal communication strategies (such as group communication).

There are two underlying issues possibly driving these observed patterns. On one hand, there may be a tendency for the research focus to be dictated by researchers' choice of methodological paradigm... Research methods are often chosen according to subjective criteria, such as researcher's worldview, his or her training and experience, psychological attributes, and the intended audience for the study... On the other hand, the level at which one determines the development issues problematic may be influencing the choice of methodological paradigm. Among the reviewed studies, those studies dealing with the most immediate aspects (for example, family planning promotion, AIDS awareness campaigns, pest management education) tended to use quantitative methods, while those empirically questioning structural challenges (for example, indigenous vs. modern knowledge, gender relations, representations of marginal voices) were more likely to use the qualitative paradigm.

Together, these two factors seem to be creating an undesirable division between qualitative and quantitative approaches in development communication research...

There is no reason to avoid quantitative research methods to analyze convoluted structural problems. If employed correctly, quantitative methods can establish causal links between immediate issues at hand with root structural causes of development problems. Strengths in quantitative research - the ability to generalize findings for future applications, established techniques to handle voluminous data, operationalization of complex factors into measurable variables - can make invaluable contributions to the formulation of much needed holistic development approaches. The existing division between methodological paradigms must be replaced by constructive dialogues between different approaches so that the empirical evidence generated in the scholarship will achieve greater legitimacy and substance.... [p. 45]

Throughout the trajectory of international assistance practices in the last 60 years, communication has always played important roles, albeit mostly in the background. Some of the evidence in this paper shows that communication can make real changes, even under most challenging circumstances. The elevated international development efforts motivated by the UN Millennium Development Goals are encouraging and should be sustained, but we should be reminded that communication is one of the few remaining resources - both viable and universal - that can be mobilized by, and for, disadvantaged people and communities in developing countries. [p. 46]

===

The above text was excerpted from "Communicating the Impact of Communication for Development: Recent Trends in Empirical Research" - World Bank Working Paper #120 - by Nobuya Inagaki. This issue of The Drum Beat focused on Chapters 5 and 6, looking at discussion of the trends and evidence and presenting conclusions. Part 2 of this 3-part series focused on Chapters 1-4, looking at theory, trends, and evidence. Part 1 reviewed the studies that served as background evidence for Mr. Inagaki's analysis.

You may download a PDF version of the full study - click here.

You may download a plain text version of the full study - click here.

===

DISCUSS!

This issue of The Drum Beat will serve as food-for-thought for the continuing discussion among the Drum Beat Chat network of this World Bank Working Paper (#120), "Communicating the Impact of Communication for Development: Recent Trends in Empirical Research" by Nobuya Inagaki.

Please register (if you are not already registered) and engage in dialogue, which began on June 10th, through the DrumBeatChat forum: click here for the archives. Register, and either participate online or send your contributions via email to drumbeatchat@comminit.com (you must be registered to participate). If connectivity is an issue for you, you may also send your contact information via email to the moderator, Deborah Heimann - dheimann@comminit.com - who can assist you with the registration process.

===

This issue of The Drum Beat includes excerpts of a paper published by The World Bank. The views expressed herein are the perspective of the writer and are not necessarily reflective of the views or opinions of The Communication Initiative or any of The Communication Initiative Partners.

===

The Drum Beat seeks to cover the full range of communication for development activities. Inclusion of an item does not imply endorsement or support by The Partners.

Please send material for The Drum Beat to the Editor - Deborah Heimann dheimann@comminit.com

To reproduce any portion of The Drum Beat, see our policy.