Introduction to Impact Evaluation of Health Communication Programs

Source - © 2016 Sarah Hoibak/VectorWorks, Courtesy of Photoshare. Addo and Festus, students at Amanhyia Catholic School in Ghana, receive long-lasting insecticide-treated bed nets (LLINs) from their teacher Justine Ayivor during the 2016 Ghana Primary School based distribution campaign.

Impact evaluation, which measures the extent to which a program's goals and objectives were attained and helps explain those effects, is vital to strategic and cost-effective investment and decision-making.

This course provides an introduction for learners to apply theories of social and behavior change communication (SBCC) and advanced statistical methods to improve the precision of the measurement of health communication impact. Learners will be able to develop a theory of change to help guide the design of impact analysis, describe the types of research designs and the strengths and weaknesses of each, and identify threats to the validity of causal inferences and how to overcome them.

The course is designed for people with previous graduate study in public health or one of the social sciences and familiarity with basic statistical analysis, including multivariate regression.

This course can be taken alone or as the first in a three-part certificate course:

  • Part I: Introduction to Impact Evaluation of Health Communication Programs
    • ​This course provides an introduction to basic concepts of communication and evaluation, identifies challenges inherent in evaluation and causal attribution, reviews common theories of communication and behavior change, guides learners through the process of identifying a theory of change that can be tested through impact evaluation, and introduces basic types of research design.
  • Part II: Preparing Data for Impact Analysis
    • ​This course provides an introduction to the types and characteristics of variables commonly used in program evaluation; reviews basic commands in the statistical package, STATA, used to manipulate data in preparation for analysis; introduces the logic and process of creating key variables for impact analysis such as measures of exposure, recall, knowledge, and attitudes; and guides learners through the steps of creating indices and scales that are commonly used in impact analysis of health communication.
  • Part III: Basic and Advanced Impact Analysis
    • ​This course reviews basic and more advanced statistical concepts that are central to understanding the measurement of change, including concepts of frequency distribution, central tendency, and statistical significance; guides learners through techniques used to compare group differences and to test bivariate and multivariate causal models; and introduces learners to newer advanced multivariate techniques, including biprobit regression and propensity score matching that improve the rigor of causal attribution.
 

Objective

By the end of this course, you will be able to:

  1. Identify main constructs from common theories of communication and behavior change.
  2. Describe a Theory of Change (or hypothesis) of communication effects for a specific project that can be tested empirically.
  3. Identify an appropriate evaluation design matched to the characteristics of a specific communication intervention.

Credits

The course authors would like to thank Joan Kraft and Maria Carrasco of USAID and Paul Hewett and Laura Reichenbach of Population Council for their review and contributions to the course.