Impact: A Guide to Evaluating Community Information Projects

This guide aims to help organisations collect useful information about the effectiveness and impact of their community information projects by highlighting aspects of the evaluation process that are unique, challenging, or critical in a community information context. It also describes and includes what are meant to be relevant and meaningful tools that can be used to assess community information projects. Produced in partnership with FSG Social Impact Advisors, the guide synthesises John S. and James L. Knight Foundation internal insights from evaluating the Knight Community Information Challenge (KCIC) and journalism projects. It is designed to help grantees and others manage and strengthen their projects by answering questions such as:

• What questions should I ask when evaluating my information and media project?
• What outcomes should I track for my project?
• How can I best utilise web analytics and social media tools to understand reach and online engagement?

The guide includes a set of assessment tools, tips, and examples from individual projects, such as:

1. Describe your project and identify your intended audience - This starts with articulating a theory of change (TOC). A TOC is a graphic representation of your assumptions and beliefs about how your project will effect the changes that you hope to see - for example, satisfying the hunger of community members to contribute to local news and information and engage in community issues but who lack the tools to do so. Next, create a logic model; a template for creating your own logic model and additional examples are included in the Appendix. Figure 3 on page 8 includes an Outcomes Framework for Community Information Projects. "Projects pursuing different types of activities to address community information needs (e.g., news, voice, action, awareness and capacity) may find that certain indicators are more relevant for them than others, depending on the approach of your project. Figure 4 highlights some indicators that might be particularly relevant for projects based on their primary project type."
2. Identify the evaluation's purpose and key questions - One tip: "By identifying and engaging stakeholders early in the evaluation process, you can begin to determine how the evaluation findings could and will be used. It is important to acknowledge that stakeholders may have different needs for the evaluation. For example, program staff may want to evaluate at the project level, whereas funders or external stakeholders may want to evaluate at the community level (refer to Figure 3)."
3. Design the evaluation using effective methods - Common methods for information projects are described, such as website analytics, social media analysis, online polls, post-event surveys, and interviews.
4. Communicate and report the evaluation findings to make decisions and take action - "The communication and reporting format you decide to use for your evaluation may be contingent on your desire for interaction with stakeholders and/or whether you wish to encourage individual or group learning. Figure 8 shows a diversity of communication formats along a spectrum of interaction..."

Links to the resources described in the guide are located in the Resource section (page 30). The Appendix contains templates that can be applied in any evaluation, as well as examples from KCIC grantees. The last page of the guide provides a one-page worksheet that you can use to begin planning your evaluation.