Prevention Center of Excellence

Annotated Bibliography: Methods of Evaluation of Coalitions/Partnerships

Methods of Evaluation of Coalitions/Partnerships available in PDF. (Download Acrobat Reader)

Compiled by Clare Desrosiers
March 1, 2006

Begley, C. E., Fourney, A., Elreda, D., Teleki, A. (2002). Evaluating outcomes of HIV prevention programs: Lessons learned from Houston, Texas. AIDS Education and prevention, 14(5), 432-443.

This article provides a case study of an evaluation approach used with an HIV prevention program. The authors report that the approach was not entirely effective. The researchers found that community providers were not willing or able to follow through with evaluation protocol, few clients returned mailed surveys, and an attempt to establish rapport and trust through one meeting with each client was not effective. On a positive note, the researchers report they were able to obtain behavioral change information reflecting program objectives from a brief survey

Berkowitz, B. (2001). Studying the outcomes of community-based coalitions. American Journal of Community Psychology, 29(2), 213-227.

The article outlines the challenges of studying community coalitions and collaborative partnerships and provides a brief summery of current research on the outcomes of coalitions and partnerships. The author identified a need for more research on coalition/collaborative outcomes and describes a three-step approach for doing such research (compilation and interpretation of existing knowledge, creation of new knowledge, and dissemination of information gained).

Birkby, B. (n.d.). Community readiness for strategic planning in substance abuse prevention: Literature review, conceptual framework, and proposed plan. Louisville, KY: REACH of Louisville, Inc.

This article provides a list of tools to evaluate community readiness, in addition to extensive information about strategic planning.

Butterfoss, F.D., & Francisco, V.T. (2004). Evaluating community partnerships and coalitions with practitioners in mind. Health Promotion Practice, 5(2), 108-114.

Literature review summarizes conditions for successful coalition implementation and achievement of desired outcomes. Describes a method for coalition evaluation. Recommends that evaluation of coalitions should include measurement of (1) processes that sustain and renew coalition process and function; (2) programs intended to meet target activities or work towards coalition goals; and (3) changes in health status or the community. Provides a list of sources of tools and tips for evaluating coalitions and partnerships.

Chalmers, M.L., Housemann, R.A., Wiggs, I., Newcomb-Hagood, L., Malone, B., & Brownson, R.C. (2003). Process evaluation of a monitoring log system for community coalition activities: Five-year results and lessons learned. American Journal of Health Promotion, 17(3), 190-196.

The article describes an evaluation of a method for tracking coalition activities. A community coalition to prevent cardiovascular disease was used as the study subject. Authors write that tracking activities is an effective way of observing changes in a community’s health promotion environment and providing an intermediate measure of a coalition’s success. From the findings, authors make several recommendations for implementing a readily usable method to track activities.

Cheadle, A., Beery, W.L., Greenwald, H.P., Nelson, G.D., Pearson, D., Senter, S. (2003). Evaluation of the California Wellness Foundation’s health improvement initiative: A logic model approach. Health Promotion Practice, 4(2), 146-156.

The article briefly describes the approach to coalition activity implementation used in the California health improvement initiative. It also describes the way in which coalitions were evaluated and offers recommendations for improved evaluation methods.

Chinman, M., Imm, P., Wandersman, A., Kaftarian, S., Neal, J., Pendleton, K.T., & Ringwalt, C. (2001). Using the Getting to Outcomes (GTO) Model in a statewide prevention initiative. Health Promotion Practice, 2(4), 302-309.

The article briefly introduces the Getting to Outcomes (GTO) Model for planning, evaluation, and accountability. The GTO uses 10 accountability questions about programming, planning, and evaluation as its primary tool to help programs achieve positive outcomes. The article also presents a case example of the South Carolina Department of Alcohol and Other Drug Abuse Services, which utilized the GTO in a statewide prevention initiative to improve substance abuse prevention services.

Gabriel, R.M. (2000). Methodological challenges in evaluating community partnerships and coalitions: Still crazy after all these years. Journal of Community Psychology, 28(3), 339-352.

Recommends evaluation strategies to improve evaluation of community drug prevention coalitions:

1. Increased interaction with program staff during evaluation yields better understanding of the program itself as well as a better assessment of its processes and outcomes.
2. The use of logic models helps clarify connections between community needs, program activities, and intended outcomes.
3. Identify and systematically eliminate (if possible) alternative/rival explanations to the evidence of positive change in identified outcomes. This is to compensate for not being able to have a control or comparison group.
4. Reporting of results to programs should be frequent and in a simple format.

Goldstein, S.M. (1997). Community coalitions: A self-assessment tool. American Journal of Health Promotion, 11(6), 430-435.

The article presents an assessment tool for community coalitions to use to evaluate effectiveness. The tool examines coalition participants, structure, processes, and stage of development. The article includes a copy of the tool.

Goodman, R.M., Wandersman, A., Chinman, M., Imm, P., & Morrissey, E. (1996). An ecological assessment of community-based interventions for prevention and health promotion: Approaches to measuring community coalitions. American Journal of Community Psychology, 24(1), 33-61.

The article recommends that community-based interventions be directed at multiple social levels (community, political, individual) and take into account a community’s readiness to receive interventions at each stage of coalition functioning. Due to the complexity of these recommended interventions, the authors also recommend the use of triangulation (the use of multiple methods in the study of subjects) as a research strategy. Finally, the article details an example of a case in which their practice and research recommendations were utilized and references 13 measurement tools or methods to assess coalition functioning, impact, and outcomes.

Granner, M.L., & Sharpe, P.A. (2004). Evaluating community coalition characteristics and functioning: A summary of measurement tools. Health Education Research, 19(5), 514-532.

Article summarizes research measurement tools for coalition/partnership characteristics or functioning; summary includes a report of instrument validity and reliability measures. Review of literature lists factors suggested to be important to coalition and partnership effectiveness and describes stages of coalition development. Authors note there is a lack of consensus on how to measure concepts related to coalition/partnership functioning. Authors recommend that, prior to starting a study of coalitions/partnerships, researchers should:

1. Identify a framework or theory for understanding coalition functioning.
2. Identify specific evaluation objectives based on project needs, community context and stakeholders’ expectations for evaluation.
3. Identify measurement tools with adequate validity and reliability, and/or create and validate new tools if time and resources allow.

Green, L.W., & Kreuter, M.W. Fighting back or fighting themselves? Community coalitions against substance abuse and their use of best practices. American Journal of Preventative Medicine, 23(4), 303-306.

The article critiques a research report about a study of coalition effectiveness by Halifors et. al. and warns against expecting too much success from coalitions. The authors also note that there needs to be a balance between expectations that coalitions will use “best practices” and adaptation of those practices to local circumstances. It recommends several factors to consider when planning evaluation of community coalitions:

1. Examine what is feasible for collaborative mechanisms to accomplish and, if necessary, realistically scale back goals and expectations.
2. Connect researchers, practitioners and/or local policy makers in the research, planning, and evaluation cycle.

Hausman, A.J., Brawer, R., Becker, J., Foster-Drain, R., Sudler, C., Wilcox, R., & Terry, B.J. (2005). The value template process: A participatory evaluation method for community health partnerships. Journal of Public Health Management and Practice, 11(1), 65-71.

The article describes the use of the Value Template evaluation process with a community health partnership. In the evaluation process, focus groups are used to obtain information from partnership members about their respective interests and concerns in regards to their involvement in the partnership. Focus group participants are also asked to identify indicators that the partnership is successfully meeting its goals. This information is use to identify similarities and differences in members’ perceptions, presented to partnership members, and used to implement a strategic plan for the partnership. Authors reported the Value Template process was helpful for the partnership with which it was used.

Hausman, A.J. (2002). Implications of evidence-based practice for community health. American Journal of Community Psychology, 30(3), 453-467.

The article summarizes the concept of evidence-based practice in community health and discusses several barriers to using evidence in community health practice. In light of these barriers, the authors make several recommendations: assess practitioners’ information utilization; evaluate planning models; improve access to relevant information; assess inclusion rules; and improve evaluation practice.

Kegler, M.C., Twiss, J.M., & Look, V. (2000). Assessing community change at multiple levels: The genesis of an evaluation framework for the California Healthy Cities Project. Health Education and Behavior, 27(6), 760-779.

The article describes an evaluation framework used to measure the outcomes of Healthy Cities programs in California. The framework assessed outcomes at the individual, organizational, community, civic, and inter-organizational levels through the use of a survey and documentation of activities, civic participation, changes in the community, and changes in Healthy Cities organization. The Healthy Cities program is similar to the Healthy Communities program. The article also includes a brief summary of barriers to evaluation of community initiatives and identifies non-traditional outcome evaluation methods that may be used when evaluating community initiatives.

Kreuter, M.W., Lezin, N.A., Young, L.A. (2000). Evaluating community-based collaborative mechanisms: Implications for practitioners. Health Promotion Practice, 1(1), 49-63.

The article discusses reasons why there is only marginal evidence in the research literature for coalition effectiveness. It reviews the stages of coalition development. Finally, it presents three frameworks for evaluation of coalitions.

Minkler, M. (2000). Using participatory action research to build healthy communities. Public Health Reports, 115, 191-197.

The article reviews the participatory action research approach (PAR) and recommends its use to help build Healthy Communities programs. It provides three examples of situations in which PAR was used. The key feature of PAR is that those affected by the issue being studied are involved in every part of the study process. This process is meant to provide an opportunity for researcher and community members to learn together and to take action to effect social change.

Reinert, B., Carver, V., Range, L.M. (2005). Evaluating community tobacco use prevention coalitions. Evaluation and Program Planning, 28, 201-208.

Article describes an objective evaluation system for tobacco use prevention coalitions in Mississippi. The authors also identify several factors characteristic of successful coalitions. Coalitions that received a high quality rating:

1. were flexible in the types of output they choose to generate and active in ways that fit their members and communities.
2. had a board composed of 9-12 members (more or less created a situation in which the board was less effective).
3. Had the largest number of people having an active job who were able to smoothly manage routine operations (e.g. clerical, scheduling, needs assessment).

Roussos, S.T., & Fawcett, S.B. (2000). A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health, 21, 369-402.

This article reviews 34 empirically based studies of the effects of collaborative partnerships on community and systems change, community-wide behavior change, and more distant population health outcomes. The authors make numerous recommendations for research and practice. They also suggest that use of geographic information systems tools may be helpful.

U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Prevention. (2002). A framework for strengthening state substance abuse prevention systems: Sharing practical experiences. Author. Retrieved 09/2005 from http://www.modelprograms.samhsa.gov/pdsf/FrameworkforStrengthening.pdf

The article provides examples of practices that have been shown to strengthen substance abuse prevention. Relevant examples include

1. Washington State provides common needs assessment data to local prevention planners. This example includes a description of the survey instruments used to collect data.
2. Washington State recently developed a survey to assess substance abuse prevention providers’ fidelity to the prevention programs they utilize and connections between fidelity to programs and program outcomes.
3. Kansas conducts an annual survey of drug use in four grade levels and provides the data to all public and private schools to be used for prevention planning and outcome assessments. The state arranges for regional prevention centers to provide training on how to use the data.
4. Illinois recently developed guidelines for prevention programs’ adoption of evidence-based interventions.

Wallerstein, N. (2000). A participatory evaluation model for healthier communities: Developing indicators for New Mexico. Public Health Reports, 115, 199-204.

The article describes the development of an evaluation process to assess the development of Healthy Communities coalitions in New Mexico. The process extended over three years and involved a community profile of population health statistics and existing collaborations and leadership; a process evaluation of leadership and coalition changes; an impact evaluation of programs, systems, or policy changes; and an outcome evaluation of economic, health, or social welfare indicators. The Participatory Evaluation Workbook for Community Initiatives was compiled from the work on this project. The authors conclude that this workbook is a useful tool for professional evaluators and planners but not for communities. The model detailed in the book can be used to identify and evaluate indicators of success in community initiatives.

| PCoE Home | PCoE Research Materials | PCoE Additional Resources |