Summative evaluation results and lessons learned from the Aligning Forces for Quality program

OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015.

METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee "alliances" from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities.

RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.