Using Frameworks to Diagram Value in Complex Policy and Environmental Interventions to Prevent Childhood Obesity Melissa Farrell Swank, MPH; Laura K. Brennan, PhD, MPH; Daniel Gentry, PhD, MHA; Allison L. Kemner, MPH rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Background: To date, few tools assist policy makers and practitioners in understanding and conveying the implementation costs, potential impacts, and value of policy and environmental changes to address healthy eating, active living, and childhood obesity. For the Evaluation of Healthy Kids, Healthy Communities (HKHC), evaluators considered inputs (resources and investments) that generate costs and savings as well as benefits and harms related to social, economic, environmental, and health-related outcomes in their assessment of 49 HKHC community partnerships funded from 2009 to 2014. Methods: Using data collected through individual and group interviews and an online performance monitoring system, evaluators created a socioecological framework to assess investments, resources, costs, savings, benefits, and harms at the individual, organizational, community, and societal levels. Evaluators customized frameworks for 6 focal strategies: active transportation, parks and play spaces, child care physical activity standards, corner stores, farmers’ markets, and child care nutrition standards. Results: To illustrate the Value Frameworks, this brief highlights the 38 HKHC communities implementing at least 1 active transportation strategy. Evaluators populated this conceptual Value Framework with themes from the strategyspecific inputs and outputs. The range of factors corresponding to the implementation and impact of the HKHC community partnerships are highlighted along with the inputs and outputs. Conclusions: The Value Frameworks helped evaluators identify gaps in current analysis models (ie, benefit-cost analysis, cost-effectiveness analysis) as well as paint a more complete picture of value for potential obesity prevention strategies. These frameworks provide a comprehensive understanding of J Public Health Management Practice, 2015, 21(3 Supp), S116–S120 C 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright
investments needed, proposed costs and savings, and potential benefits and harms associated with economic, social, environmental, and health outcomes. This framing also allowed evaluators to demonstrate the interdependence of each socioecological level on the others in these multicomponent interventions. This model can be used by practitioners and
Author Affiliations: Transtria LLC, St Louis, Missouri (Mss Swank and Kemner and Dr Brennan); and University of Memphis School of Public Health, Memphis, Tennessee (Dr Gentry). Support for this evaluation was provided by a grant from the Robert Wood Johnson Foundation (#67099). The authors are grateful for the collaboration with and support from the Robert Wood Johnson Foundation (Laura Leviton, PhD; and Tina Kauh, PhD), the Washington University Institute for Public Health (Ross Brownson, PhD), the Healthy Kids, Healthy Communities (HKHC) National Program Office (Casey Allred; Rich Bell, MCP; Phil Bors, MPH; Mark Dessauer, MA; Fay Gibson, MSW; Joanne Lee, LDN, RD, MPH; Mary Beth Powell, MPH; Tim Schwantes, MPH, MSW; Sarah Strunk, MHA; and Risa Wilkerson, MA), the HKHC Evaluation Advisory Group (Geni Eng, DrPH, MPH; Leah Ersoylu, PhD; Laura Kettel Khan, PhD; Vikki Lassiter, MS; Barbara Leonard, MPH; Amelie Ramirez, DrPH, MPH; James Sallis, PhD; and Mary Story, PhD), the Social System Design Lab at Washington University in St Louis (Peter Hovmand, PhD), and participating representatives from all 49 community partnerships. The authors give special thanks to the many individuals who have contributed to these efforts from Transtria LLC, including past and present evaluation officers (Tammy Behlmann, MPH; Kate Donaldson, MPH; Cheryl Carnoske, MPH; Carl Filler, MSW; Peter Holtgrave, MPH, MA; Christy Hoehner, PhD, MPH; and Jessica Stachecki, MSW, MBA), project assistants (James Bernhardt; Rebecca Bradley; Ashley Crain, MPH; Emily Herrington, MPH; Ashley Farell, MPH; Amy Krieg; Brandye Mazdra, MPH; Kathy Mora, PhD; Jason Roche, MPH; Carrie Rogers, MPH; Shaina Sowles, MPH; Muniru Sumbeida, MPH, MSW; Caroline Swift, MPH; Gauri Wadhwa, MPH; and Jocelyn Wagman, MPH), additional staff (Michele Bildner, MPH, CHES, and Daedra Lohr, MS), interns (Christine Beam, MPH; Skye Buckner-Petty, MPH; Maggie Fairchild, MPH; Mackenzie Ray, MPH; and Lauren Spaeth, MS), and transcriptionists (Sheri Joyce; Chad Lyles; Robert Morales; and Vanisa Verma, MPH). The authors declare no conflicts of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (http://www.JPHMP.com). Correspondence: Melissa Farrell Swank, MPH, Transtria LLC, 6514 Lansdowne Ave, St Louis, MO 63109 ([email protected]). DOI: 10.1097/PHH.0000000000000210
S116 Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Using Frameworks to Diagram Value of Prevent Childhood Obesity
community leaders to assess realistic and sustainable strategies to combat childhood obesity.
❘ S117
● Methods Data collection
KEY WORDS: active living, active transportation, childhood
obesity prevention, costs, economic analysis, healthy eating, investments, physical activity, resources, socioecological model, value
The determination of the value of community-based public health approaches to prevent childhood obesity has proven difficult to measure and quantify.1-3 Strategies can be evaluated through cost-benefit or costeffectiveness analyses, which provide value-based assessments within the context of economic and health outcomes, respectively.4-6 While such evaluations consider important factors related to policy, systems, and environmental strategies, confining the discussion about value to economic impact or quantifiable health outcomes may misrepresent the full picture. In turn, the discussion may miss other key elements of value such as social, educational, and environmental benefits and harms that are more qualitative in nature (eg, volunteer time, in-kind donations, saved or preserved resources, life-years added, and adverse social and environmental consequences). In 2012, the Committee on Public Health Strategies to Improve Health recommended addressing the need for understanding and comparing costs and benefits within and across strategies to improve population health. Specifically, the committee proposed the development of methods to understand value and the impact of investments.7 Evaluators for Healthy Kids, Healthy Communities (HKHC), a national program of the Robert Wood Johnson Foundation (www.healthykidshealthy communities.org),8 identified the need to provide the 49 grant-funded HKHC communities with methods to translate the value and potential impact of their work to advocates, decision makers, practitioners, and community residents to help sustain their initiatives. With obesity rates in children and adolescents continuing to rise (over the past 30 years, rates in children aged 6-11 years increased by >250% and adolescents rates increased by >400%),9-11 the communities worked to reverse this trend and support healthier communities for children and families through implementation of healthy eating and active living policy, systems, and environmental changes. Yet, they lacked a tool to show the value of such efforts. The Value Frameworks were created to illustrate the range of inputs and outputs documented in the evaluation of HKHC communities to increase the understanding of the financial and ethical merits of policy, system, and environmental interventions to prevent childhood obesity.
Evaluators coordinated several interviews with project directors and project coordinators in each HKHC community, including staff and partners from a broad range of sectors (eg, transportation, education, health, business) and disciplines (eg, planners, nutritionists, engineers), as well as community residents. Individual and group interviews were conducted before, during, and after site visits with the HKHC grantees between September 2011 and April 2013. An average of 3 interviews were conducted per site with individuals or groups of key informants, including elected or appointed city officials, government agencies, community-based organizations, civic organizations, and community residents. In addition, existing data sources were inventoried to identify themes for the Value Frameworks, including an Institute of Medicine report,1,2 Transtria’s Evidence Review project,12,13 and expert review systems such as The Community Guide.14
Selection of focal strategies At the outset, the evaluation team selected the 6 crosssite strategies (Corner Stores, Farmers’ Markets, Nutrition Standards in Childcare Settings, Active Transportation, Parks and Play Spaces, and Physical Activity Standards in Childcare Settings) by conducting a content analysis of HKHC grantees proposals and work plans to track policy, systems, and environmental strategies across the 49 communities; planned evaluation methods and measures were similarly recorded. See the methods overview in this supplement for more background on the mixed-methods approach.15
Data analysis Interviews with the HKHC community partnerships were transcribed and qualitatively analyzed using focused coding procedures to identify indigenous themes. Key themes derived from interviews were then extracted and included in the Value Frameworks.
Identifying implementation and impact factors Within the context of each strategy, evaluators consulted community representatives and experts within the field to identify elements of implementation (eg, advocacy and organizing, policy development and/or policy implementation and enforcement activities) and impact (eg, influence on policies, environments and services, and/or populations) associated with policy, systems, and environmental changes.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
S118 ❘ Journal of Public Health Management and Practice
Identifying elements of value across socioecological levels
Validating the frameworks with the HKHC sites
Since the understanding and quantification of value changes on the basis of perspective, evaluators developed a socioecological framework to break down inputs and outputs at the individual, organizational, community, and societal levels. In consultation with the community partnerships, HKHC project advisors, and experts in relevant fields (eg, childhood obesity, public health, economics), evaluators identified financial investments and nonmonetary resources (inputs) available to support planning, implementation, or maintenance of strategies as well as the use of the products of these strategies (eg, changes in clientele for local businesses). Similarly, evaluators collaborated with these advisors to isolate potential monetized expenditures and expenditures averted (costs and savings, respectively), in addition to actual or potential favorable or adverse consequences (benefits and harms, respectively) that may result from strategies. Therefore, potential outcomes included in the Value Frameworks reflect both research-based evidence and practice-based evidence shared by HKHC sites. Thus, a Value Framework was created to demonstrate the range of implementation efforts, the range of potential impacts (harms and benefits), the resources used for implementation, and the costs associated with implementation. While 6 strategies were addressed in this fashion, active transportation is highlighted as an example in this article.
Throughout the process, evaluators requested input from potential end users of the tool to ensure the creation of a useful final product. A draft of the Value Frameworks was shared with the HKHC grantees through round table discussions to gain feedback. This feedback was incorporated into the final versions. The final Value Frameworks were shared with a subset of HKHC grantees in the fall of 2013 for some additional comments or suggestions. Throughout the process, iterations were shared with national advisors and partners from the HKHC National Program Office.
● Results Thirty-eight HKHC sites implemented active transportation strategies, with some sites implementing more than 1 strategy, such as public transportation efforts and bicycle infrastructure changes. Within these 38 sites, 48 active transportation interviews were conducted with a total of 158 participants. Components and examples related to the resulting active transportation Value Framework are illustrated in the Figure. For detailed analysis of the Value Framework and its application to active transportation, see Supplemental Digital Content Figures 1-5 (available at: http:// links.lww.com/JPHMP/A122, http://links.lww.com/ JPHMP/A123, http://links.lww.com/JPHMP/A124,
FIGURE ● Examples Illustrating Value Framework Components
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Abbreviation: DOT, Department of Transportation.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Using Frameworks to Diagram Value of Prevent Childhood Obesity
http://links.lww.com/JPHMP/A125, and http:// links.lww.com/JPHMP/A125, respectively). Implementation efforts for active transportation strategies included assessment of street environments, policy analysis, and cross-sector collaboration (see Supplemental Digital Content Figure 1, available at: http://links.lww.com/JPHMP/A122). In turn, these efforts led to the development and adoption of active transportation policies (short-term impact), changes to pedestrian or bicycling infrastructure (intermediate impact), or changes to population health (long-term impact). Furthermore, inputs and outcomes associated with these elements are grouped within the context of each socioecological level (see the Figure for examples and Supplemental Digital Content Figures 1-5, available at: http://links.lww.com/JPHMP/A122, http:// links.lww.com/JPHMP/A123, http://links.lww.com/ JPHMP/A124, http://links.lww.com/JPHMP/A125, and http://links.lww.com/JPHMP/A126, respectively, for detailed analysis). At the individual level, investments and resources may be in the form of salary (money available) or volunteerism (time not otherwise obligated). Costs and savings for an individual may include money spent or saved on transportation fees/tolls. Whereas benefits or harms may be related to traffic exposure, impact on social engagement, or impact on physical activity behaviors (see Supplemental Digital Content Figure 2, available at: http://links.lww.com/JPHMP/A123). Agencies or organizations may see investments/ resources come from federal, state, or local tax credits (for qualifying projects) or as agency/organization donation of meeting space or equipment. Outputs at the agency or organizational level may involve staff time and benefits for active transportation projects (costs/savings) or as transportation access to agencies/organizations (benefits/harms; see Supplemental Digital Content Figure 3, available at: http://links.lww .com/JPHMP/A124). Investments and resources at the community level may take shape as funds from federal or state departments of transportation (monetary) or support from city or county elected officials (in-kind). Active transportation policy, systems, or environmental changes may incur costs or generate savings at the community level related to infrastructure (eg, highways) or create benefits or harms related to mobility and access to destinations (see Supplemental Digital Content Figure 4, available at: http://links.lww.com/JPHMP/A125). Finally, societal investments may come from state or national fund raising initiatives or federal/state land designated for active transportation initiatives. In turn, costs or savings may be experienced at the societal level as state or federal taxes earmarked for active transportation. Benefits or harms at the societal
❘ S119
level may include considerations for transportation equity or state and national rates of chronic disease (see Supplemental Digital Content Figure 5, available at: http://links.lww.com/JPHMP/A126). Individual level inputs and outcomes are sometimes aggregated at the organization/agency, community, or societal levels. Importantly, inputs and outputs are not confined within each socioecological level. Rather, while investments, resources, costs, savings, benefits, and harms are all perceived at the individual, organizational, community, and societal levels, the inputs at one level may be outputs at another level, and vice versa. For instance, the taxes paid by an individual (as an output or cost) may, in turn, provide the tax revenue (input or investment) allocated to support healthy eating and active living policy initiatives, environmental changes, or programs and services through communities (local tax base) or at the societal level (state or federal tax base).
● Discussion Efficient use of resources to achieve the greatest value for effort invested is increasingly receiving attention in public health.16,17 The current economic climate has decreased funding opportunities,17,18 placing even greater emphasis on creating, implementing, and sustaining changes with the potential for short-term and longterm impact across sectors. These Value Frameworks represent the evaluation team’s effort to respond to the demand for resources that can be used by community residents, practitioners, and researchers to express the value of their childhood obesity prevention efforts and illustrate the potential impact on economic, social, environmental, and health-related outcomes. The Value Frameworks are intended to be adapted to a particular setting and set of circumstances. Using the guidelines provided in the Value Framework manual, planners or community representatives would identify a potential strategy and then insert all relevant inputs and outputs specific to their community (using the tool as a guide; see www.transtria.com/hkhc for online access to the Value Frameworks manual). Once customized, these frameworks enable community representatives to plan and prepare for advocacy initiatives (eg, through presentation and discussion of a comprehensive plan with elected or appointed officials); policy adoption, implementation, enforcement, and sustainability efforts (eg, incorporating the voices of community members and practitioners into the plan); and changes to the built environment to prevent childhood obesity (eg, by planning to allocate resources for maintenance of environmental changes). Yet, use of the Value Frameworks is not limited to these 6 strategies.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
S120 ❘ Journal of Public Health Management and Practice The intention of the evaluation team was to create a tool that balanced the accessibility of the document with the inclusion of sufficient detail and examples relevant to diverse sectors and disciplines in communities. With knowledge that strategies overlap (particularly those across the chronic disease prevention portfolio) in both inputs and outputs, these 6 frameworks may be used and adapted to a myriad of policy, system, or environmental changes within the field of childhood obesity prevention and beyond. Regardless of the strategy, a tailored framework can provide a structure for presenting or proposing public health strategies to decision makers or elected officials, using available measures and cost estimates to support projected inputs and outputs. Customized frameworks can also continue to be used during strategy implementation (as a form of process evaluation) and retrospectively to evaluate the consistency of projections with actualized impact (impact and outcome evaluation). In turn, these analyses may enable planners and decision makers to critique and adjust their approach to increase efficient use of resources in the future. Using feedback from community representatives, researchers, and advisors, the authors anticipate continuous refinement and improvement of the Value Frameworks, both in content and in format (ie, potential interactive online interface), as they are shared, applied, tested, critiqued, and revised.
6.
7.
8.
9.
10.
11.
12.
13.
14.
REFERENCES 1. Pronk NP, Hernandez LM, Lawrence RS. An integrated framework for assessing the value of community-based prevention: a report of the Institute of Medicine. Prev Chronic Dis. 2013;10:120323. doi:http://dx.doi.org/10.5888/pcd10 .120323. 2. Committee on Childhood Obesity Prevention Actions for Local Governments, Institute of Medicine, National Research Council. Local Government Actions to Prevent Childhood Obesity. Washington, DC: The National Academies Press; 2009. 3. Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1309-1311. 4. National Research Council. Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making. Washington, DC: The National Academies Press; 2010. 5. Carlson D, Haveman R, Kaplan T, Wolfe B. The benefits and costs of the Section 8 housing subsidy program: a frame-
15.
16.
17.
18.
work and estimates of first year effects. J Policy Anal Manage. 2011;30(2):233-255. Weisbrod B. A guide to benefit-cost analysis, as seen through a controlled experiment in treating the mentally ill. J Health Polit Policy Law. 1983;7(4):808-845. Institute of Medicine. For the Public’s Health: Investing in a Healthier Future. Washington, DC: The National Academies Press; 2012. About Healthy Kids, Healthy Communities. Healthy Kids, Healthy Communities Web site. http://www.healthykids healthycommunities.org/about. Revised. Accessed March 15, 2014. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014;311(8):806-814. National Center for Health Statistics. Health, United States, 2011: With Special Features on Socioeconomic Status and Health. Hyattsville, MD: US Department of Health and Human Services; 2012. Centers for Disease Control and Prevention. Adolescent and school health: childhood obesity facts. http://www. cdc.gov/healthyyouth/obesity/facts.htm. Accessed May 13, 2014. Brennan L, Brownson R, Orleans T. Childhood obesity policy research and practice evidence for policy and environmental strategies. Am J Prev Med. 2014;46(1):e1-e16. Brennan L, Castro S, Brownson R, Claus J, Orleans T. Accelerating evidence reviews and broadening evidence standards to identify effective, promising, and emerging policy and environmental strategies for prevention of childhood obesity. Annu Rev Public Health. 2011;32:199-223. Centers for Disease Control and Prevention. The guide to community preventative services. Obesity prevention and control. http://thecommunityguide.org/library/pubstopic .htm#obesity. Updated April 21, 2014. Accessed May 13, 2014. Brownson R, Kemner A, Brennan L. Applying a mixedmethods evaluation to Healthy Kids, Healthy Communities. J Public Health Manage Pract. 2015;21(3 Suppl):S16-S26. US Department of Health and Human Services. Public health quality. http://www.hhs.gov/ash/initiatives/ quality/quality. Accessed June 2014. North Carolina Association of Local Health Directors Public Health Task Force. A Blueprint of the Future for Local Public Health Departments in North Carolina. 2013 Statewide Public Health Incubator Summary Report & Recommendations. Raleigh, NC: North Carolina Association of Local Health Directors Public Health Task Force; 2013. Abramson S. Holes in the net: surveying the impact of the current economic recession on the health care safety net. Issue Brief. 2009. http://www.apha.org/NR/rdonlyres/ DC0A82C1-606B-4F67-B3DD-3E14A9FB3C9B/0/1SafetyNet .pdf. Accessed June 11, 2014.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Background: To date, few tools assist policy makers and practitioners in understanding and conveying the implementation costs, potential impacts, and value of policy and environmental changes to address healthy eating, active living, and childhood obesity. For the Evaluation of Healthy Kids, Healthy Communities (HKHC), evaluators considered inputs (resources and investments) that generate costs and savings as well as benefits and harms related to social, economic, environmental, and health-related outcomes in their assessment of 49 HKHC community partnerships funded from 2009 to 2014. Methods: Using data collected through individual and group interviews and an online performance monitoring system, evaluators created a socioecological framework to assess investments, resources, costs, savings, benefits, and harms at the individual, organizational, community, and societal levels. Evaluators customized frameworks for 6 focal strategies: active transportation, parks and play spaces, child care physical activity standards, corner stores, farmers’ markets, and child care nutrition standards. Results: To illustrate the Value Frameworks, this brief highlights the 38 HKHC communities implementing at least 1 active transportation strategy. Evaluators populated this conceptual Value Framework with themes from the strategyspecific inputs and outputs. The range of factors corresponding to the implementation and impact of the HKHC community partnerships are highlighted along with the inputs and outputs. Conclusions: The Value Frameworks helped evaluators identify gaps in current analysis models (ie, benefit-cost analysis, cost-effectiveness analysis) as well as paint a more complete picture of value for potential obesity prevention strategies. These frameworks provide a comprehensive understanding of J Public Health Management Practice, 2015, 21(3 Supp), S116–S120 C 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright
investments needed, proposed costs and savings, and potential benefits and harms associated with economic, social, environmental, and health outcomes. This framing also allowed evaluators to demonstrate the interdependence of each socioecological level on the others in these multicomponent interventions. This model can be used by practitioners and
Author Affiliations: Transtria LLC, St Louis, Missouri (Mss Swank and Kemner and Dr Brennan); and University of Memphis School of Public Health, Memphis, Tennessee (Dr Gentry). Support for this evaluation was provided by a grant from the Robert Wood Johnson Foundation (#67099). The authors are grateful for the collaboration with and support from the Robert Wood Johnson Foundation (Laura Leviton, PhD; and Tina Kauh, PhD), the Washington University Institute for Public Health (Ross Brownson, PhD), the Healthy Kids, Healthy Communities (HKHC) National Program Office (Casey Allred; Rich Bell, MCP; Phil Bors, MPH; Mark Dessauer, MA; Fay Gibson, MSW; Joanne Lee, LDN, RD, MPH; Mary Beth Powell, MPH; Tim Schwantes, MPH, MSW; Sarah Strunk, MHA; and Risa Wilkerson, MA), the HKHC Evaluation Advisory Group (Geni Eng, DrPH, MPH; Leah Ersoylu, PhD; Laura Kettel Khan, PhD; Vikki Lassiter, MS; Barbara Leonard, MPH; Amelie Ramirez, DrPH, MPH; James Sallis, PhD; and Mary Story, PhD), the Social System Design Lab at Washington University in St Louis (Peter Hovmand, PhD), and participating representatives from all 49 community partnerships. The authors give special thanks to the many individuals who have contributed to these efforts from Transtria LLC, including past and present evaluation officers (Tammy Behlmann, MPH; Kate Donaldson, MPH; Cheryl Carnoske, MPH; Carl Filler, MSW; Peter Holtgrave, MPH, MA; Christy Hoehner, PhD, MPH; and Jessica Stachecki, MSW, MBA), project assistants (James Bernhardt; Rebecca Bradley; Ashley Crain, MPH; Emily Herrington, MPH; Ashley Farell, MPH; Amy Krieg; Brandye Mazdra, MPH; Kathy Mora, PhD; Jason Roche, MPH; Carrie Rogers, MPH; Shaina Sowles, MPH; Muniru Sumbeida, MPH, MSW; Caroline Swift, MPH; Gauri Wadhwa, MPH; and Jocelyn Wagman, MPH), additional staff (Michele Bildner, MPH, CHES, and Daedra Lohr, MS), interns (Christine Beam, MPH; Skye Buckner-Petty, MPH; Maggie Fairchild, MPH; Mackenzie Ray, MPH; and Lauren Spaeth, MS), and transcriptionists (Sheri Joyce; Chad Lyles; Robert Morales; and Vanisa Verma, MPH). The authors declare no conflicts of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (http://www.JPHMP.com). Correspondence: Melissa Farrell Swank, MPH, Transtria LLC, 6514 Lansdowne Ave, St Louis, MO 63109 ([email protected]). DOI: 10.1097/PHH.0000000000000210
S116 Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Using Frameworks to Diagram Value of Prevent Childhood Obesity
community leaders to assess realistic and sustainable strategies to combat childhood obesity.
❘ S117
● Methods Data collection
KEY WORDS: active living, active transportation, childhood
obesity prevention, costs, economic analysis, healthy eating, investments, physical activity, resources, socioecological model, value
The determination of the value of community-based public health approaches to prevent childhood obesity has proven difficult to measure and quantify.1-3 Strategies can be evaluated through cost-benefit or costeffectiveness analyses, which provide value-based assessments within the context of economic and health outcomes, respectively.4-6 While such evaluations consider important factors related to policy, systems, and environmental strategies, confining the discussion about value to economic impact or quantifiable health outcomes may misrepresent the full picture. In turn, the discussion may miss other key elements of value such as social, educational, and environmental benefits and harms that are more qualitative in nature (eg, volunteer time, in-kind donations, saved or preserved resources, life-years added, and adverse social and environmental consequences). In 2012, the Committee on Public Health Strategies to Improve Health recommended addressing the need for understanding and comparing costs and benefits within and across strategies to improve population health. Specifically, the committee proposed the development of methods to understand value and the impact of investments.7 Evaluators for Healthy Kids, Healthy Communities (HKHC), a national program of the Robert Wood Johnson Foundation (www.healthykidshealthy communities.org),8 identified the need to provide the 49 grant-funded HKHC communities with methods to translate the value and potential impact of their work to advocates, decision makers, practitioners, and community residents to help sustain their initiatives. With obesity rates in children and adolescents continuing to rise (over the past 30 years, rates in children aged 6-11 years increased by >250% and adolescents rates increased by >400%),9-11 the communities worked to reverse this trend and support healthier communities for children and families through implementation of healthy eating and active living policy, systems, and environmental changes. Yet, they lacked a tool to show the value of such efforts. The Value Frameworks were created to illustrate the range of inputs and outputs documented in the evaluation of HKHC communities to increase the understanding of the financial and ethical merits of policy, system, and environmental interventions to prevent childhood obesity.
Evaluators coordinated several interviews with project directors and project coordinators in each HKHC community, including staff and partners from a broad range of sectors (eg, transportation, education, health, business) and disciplines (eg, planners, nutritionists, engineers), as well as community residents. Individual and group interviews were conducted before, during, and after site visits with the HKHC grantees between September 2011 and April 2013. An average of 3 interviews were conducted per site with individuals or groups of key informants, including elected or appointed city officials, government agencies, community-based organizations, civic organizations, and community residents. In addition, existing data sources were inventoried to identify themes for the Value Frameworks, including an Institute of Medicine report,1,2 Transtria’s Evidence Review project,12,13 and expert review systems such as The Community Guide.14
Selection of focal strategies At the outset, the evaluation team selected the 6 crosssite strategies (Corner Stores, Farmers’ Markets, Nutrition Standards in Childcare Settings, Active Transportation, Parks and Play Spaces, and Physical Activity Standards in Childcare Settings) by conducting a content analysis of HKHC grantees proposals and work plans to track policy, systems, and environmental strategies across the 49 communities; planned evaluation methods and measures were similarly recorded. See the methods overview in this supplement for more background on the mixed-methods approach.15
Data analysis Interviews with the HKHC community partnerships were transcribed and qualitatively analyzed using focused coding procedures to identify indigenous themes. Key themes derived from interviews were then extracted and included in the Value Frameworks.
Identifying implementation and impact factors Within the context of each strategy, evaluators consulted community representatives and experts within the field to identify elements of implementation (eg, advocacy and organizing, policy development and/or policy implementation and enforcement activities) and impact (eg, influence on policies, environments and services, and/or populations) associated with policy, systems, and environmental changes.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
S118 ❘ Journal of Public Health Management and Practice
Identifying elements of value across socioecological levels
Validating the frameworks with the HKHC sites
Since the understanding and quantification of value changes on the basis of perspective, evaluators developed a socioecological framework to break down inputs and outputs at the individual, organizational, community, and societal levels. In consultation with the community partnerships, HKHC project advisors, and experts in relevant fields (eg, childhood obesity, public health, economics), evaluators identified financial investments and nonmonetary resources (inputs) available to support planning, implementation, or maintenance of strategies as well as the use of the products of these strategies (eg, changes in clientele for local businesses). Similarly, evaluators collaborated with these advisors to isolate potential monetized expenditures and expenditures averted (costs and savings, respectively), in addition to actual or potential favorable or adverse consequences (benefits and harms, respectively) that may result from strategies. Therefore, potential outcomes included in the Value Frameworks reflect both research-based evidence and practice-based evidence shared by HKHC sites. Thus, a Value Framework was created to demonstrate the range of implementation efforts, the range of potential impacts (harms and benefits), the resources used for implementation, and the costs associated with implementation. While 6 strategies were addressed in this fashion, active transportation is highlighted as an example in this article.
Throughout the process, evaluators requested input from potential end users of the tool to ensure the creation of a useful final product. A draft of the Value Frameworks was shared with the HKHC grantees through round table discussions to gain feedback. This feedback was incorporated into the final versions. The final Value Frameworks were shared with a subset of HKHC grantees in the fall of 2013 for some additional comments or suggestions. Throughout the process, iterations were shared with national advisors and partners from the HKHC National Program Office.
● Results Thirty-eight HKHC sites implemented active transportation strategies, with some sites implementing more than 1 strategy, such as public transportation efforts and bicycle infrastructure changes. Within these 38 sites, 48 active transportation interviews were conducted with a total of 158 participants. Components and examples related to the resulting active transportation Value Framework are illustrated in the Figure. For detailed analysis of the Value Framework and its application to active transportation, see Supplemental Digital Content Figures 1-5 (available at: http:// links.lww.com/JPHMP/A122, http://links.lww.com/ JPHMP/A123, http://links.lww.com/JPHMP/A124,
FIGURE ● Examples Illustrating Value Framework Components
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Abbreviation: DOT, Department of Transportation.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Using Frameworks to Diagram Value of Prevent Childhood Obesity
http://links.lww.com/JPHMP/A125, and http:// links.lww.com/JPHMP/A125, respectively). Implementation efforts for active transportation strategies included assessment of street environments, policy analysis, and cross-sector collaboration (see Supplemental Digital Content Figure 1, available at: http://links.lww.com/JPHMP/A122). In turn, these efforts led to the development and adoption of active transportation policies (short-term impact), changes to pedestrian or bicycling infrastructure (intermediate impact), or changes to population health (long-term impact). Furthermore, inputs and outcomes associated with these elements are grouped within the context of each socioecological level (see the Figure for examples and Supplemental Digital Content Figures 1-5, available at: http://links.lww.com/JPHMP/A122, http:// links.lww.com/JPHMP/A123, http://links.lww.com/ JPHMP/A124, http://links.lww.com/JPHMP/A125, and http://links.lww.com/JPHMP/A126, respectively, for detailed analysis). At the individual level, investments and resources may be in the form of salary (money available) or volunteerism (time not otherwise obligated). Costs and savings for an individual may include money spent or saved on transportation fees/tolls. Whereas benefits or harms may be related to traffic exposure, impact on social engagement, or impact on physical activity behaviors (see Supplemental Digital Content Figure 2, available at: http://links.lww.com/JPHMP/A123). Agencies or organizations may see investments/ resources come from federal, state, or local tax credits (for qualifying projects) or as agency/organization donation of meeting space or equipment. Outputs at the agency or organizational level may involve staff time and benefits for active transportation projects (costs/savings) or as transportation access to agencies/organizations (benefits/harms; see Supplemental Digital Content Figure 3, available at: http://links.lww .com/JPHMP/A124). Investments and resources at the community level may take shape as funds from federal or state departments of transportation (monetary) or support from city or county elected officials (in-kind). Active transportation policy, systems, or environmental changes may incur costs or generate savings at the community level related to infrastructure (eg, highways) or create benefits or harms related to mobility and access to destinations (see Supplemental Digital Content Figure 4, available at: http://links.lww.com/JPHMP/A125). Finally, societal investments may come from state or national fund raising initiatives or federal/state land designated for active transportation initiatives. In turn, costs or savings may be experienced at the societal level as state or federal taxes earmarked for active transportation. Benefits or harms at the societal
❘ S119
level may include considerations for transportation equity or state and national rates of chronic disease (see Supplemental Digital Content Figure 5, available at: http://links.lww.com/JPHMP/A126). Individual level inputs and outcomes are sometimes aggregated at the organization/agency, community, or societal levels. Importantly, inputs and outputs are not confined within each socioecological level. Rather, while investments, resources, costs, savings, benefits, and harms are all perceived at the individual, organizational, community, and societal levels, the inputs at one level may be outputs at another level, and vice versa. For instance, the taxes paid by an individual (as an output or cost) may, in turn, provide the tax revenue (input or investment) allocated to support healthy eating and active living policy initiatives, environmental changes, or programs and services through communities (local tax base) or at the societal level (state or federal tax base).
● Discussion Efficient use of resources to achieve the greatest value for effort invested is increasingly receiving attention in public health.16,17 The current economic climate has decreased funding opportunities,17,18 placing even greater emphasis on creating, implementing, and sustaining changes with the potential for short-term and longterm impact across sectors. These Value Frameworks represent the evaluation team’s effort to respond to the demand for resources that can be used by community residents, practitioners, and researchers to express the value of their childhood obesity prevention efforts and illustrate the potential impact on economic, social, environmental, and health-related outcomes. The Value Frameworks are intended to be adapted to a particular setting and set of circumstances. Using the guidelines provided in the Value Framework manual, planners or community representatives would identify a potential strategy and then insert all relevant inputs and outputs specific to their community (using the tool as a guide; see www.transtria.com/hkhc for online access to the Value Frameworks manual). Once customized, these frameworks enable community representatives to plan and prepare for advocacy initiatives (eg, through presentation and discussion of a comprehensive plan with elected or appointed officials); policy adoption, implementation, enforcement, and sustainability efforts (eg, incorporating the voices of community members and practitioners into the plan); and changes to the built environment to prevent childhood obesity (eg, by planning to allocate resources for maintenance of environmental changes). Yet, use of the Value Frameworks is not limited to these 6 strategies.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
S120 ❘ Journal of Public Health Management and Practice The intention of the evaluation team was to create a tool that balanced the accessibility of the document with the inclusion of sufficient detail and examples relevant to diverse sectors and disciplines in communities. With knowledge that strategies overlap (particularly those across the chronic disease prevention portfolio) in both inputs and outputs, these 6 frameworks may be used and adapted to a myriad of policy, system, or environmental changes within the field of childhood obesity prevention and beyond. Regardless of the strategy, a tailored framework can provide a structure for presenting or proposing public health strategies to decision makers or elected officials, using available measures and cost estimates to support projected inputs and outputs. Customized frameworks can also continue to be used during strategy implementation (as a form of process evaluation) and retrospectively to evaluate the consistency of projections with actualized impact (impact and outcome evaluation). In turn, these analyses may enable planners and decision makers to critique and adjust their approach to increase efficient use of resources in the future. Using feedback from community representatives, researchers, and advisors, the authors anticipate continuous refinement and improvement of the Value Frameworks, both in content and in format (ie, potential interactive online interface), as they are shared, applied, tested, critiqued, and revised.
6.
7.
8.
9.
10.
11.
12.
13.
14.
REFERENCES 1. Pronk NP, Hernandez LM, Lawrence RS. An integrated framework for assessing the value of community-based prevention: a report of the Institute of Medicine. Prev Chronic Dis. 2013;10:120323. doi:http://dx.doi.org/10.5888/pcd10 .120323. 2. Committee on Childhood Obesity Prevention Actions for Local Governments, Institute of Medicine, National Research Council. Local Government Actions to Prevent Childhood Obesity. Washington, DC: The National Academies Press; 2009. 3. Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1309-1311. 4. National Research Council. Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making. Washington, DC: The National Academies Press; 2010. 5. Carlson D, Haveman R, Kaplan T, Wolfe B. The benefits and costs of the Section 8 housing subsidy program: a frame-
15.
16.
17.
18.
work and estimates of first year effects. J Policy Anal Manage. 2011;30(2):233-255. Weisbrod B. A guide to benefit-cost analysis, as seen through a controlled experiment in treating the mentally ill. J Health Polit Policy Law. 1983;7(4):808-845. Institute of Medicine. For the Public’s Health: Investing in a Healthier Future. Washington, DC: The National Academies Press; 2012. About Healthy Kids, Healthy Communities. Healthy Kids, Healthy Communities Web site. http://www.healthykids healthycommunities.org/about. Revised. Accessed March 15, 2014. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014;311(8):806-814. National Center for Health Statistics. Health, United States, 2011: With Special Features on Socioeconomic Status and Health. Hyattsville, MD: US Department of Health and Human Services; 2012. Centers for Disease Control and Prevention. Adolescent and school health: childhood obesity facts. http://www. cdc.gov/healthyyouth/obesity/facts.htm. Accessed May 13, 2014. Brennan L, Brownson R, Orleans T. Childhood obesity policy research and practice evidence for policy and environmental strategies. Am J Prev Med. 2014;46(1):e1-e16. Brennan L, Castro S, Brownson R, Claus J, Orleans T. Accelerating evidence reviews and broadening evidence standards to identify effective, promising, and emerging policy and environmental strategies for prevention of childhood obesity. Annu Rev Public Health. 2011;32:199-223. Centers for Disease Control and Prevention. The guide to community preventative services. Obesity prevention and control. http://thecommunityguide.org/library/pubstopic .htm#obesity. Updated April 21, 2014. Accessed May 13, 2014. Brownson R, Kemner A, Brennan L. Applying a mixedmethods evaluation to Healthy Kids, Healthy Communities. J Public Health Manage Pract. 2015;21(3 Suppl):S16-S26. US Department of Health and Human Services. Public health quality. http://www.hhs.gov/ash/initiatives/ quality/quality. Accessed June 2014. North Carolina Association of Local Health Directors Public Health Task Force. A Blueprint of the Future for Local Public Health Departments in North Carolina. 2013 Statewide Public Health Incubator Summary Report & Recommendations. Raleigh, NC: North Carolina Association of Local Health Directors Public Health Task Force; 2013. Abramson S. Holes in the net: surveying the impact of the current economic recession on the health care safety net. Issue Brief. 2009. http://www.apha.org/NR/rdonlyres/ DC0A82C1-606B-4F67-B3DD-3E14A9FB3C9B/0/1SafetyNet .pdf. Accessed June 11, 2014.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
