RESEARCH AND PRACTICE
Public Health Accreditation: Rubber Stamp or Roadmap for Improvement Angela L. Carman, DrPH, and Lava Timsina, MPH
A variety of events in public health have focused attention on the need to improve public health agency performance. The 1988 Institute of Medicine report, “The Future of Public Health,” characterized public health as a “system in disarray.” In 2002, the Institute of Medicine released a second major report, “The Future of Public Health in the 21st Century,” that recommended exploration of public health accreditation as a means of improving performance and accountability for governmental public health departments. In September 2011, the Public Health Accreditation Board (PHAB) launched the first voluntary accreditation system with consensus standards and measures for state, local, and tribal public health agencies in the United States.1,2 The mission of PHAB is to promote and protect the health of communities by advancing quality and performance of all public health departments in the United States.1,3,4 National public health accreditation domains, standards, and measures are structured around the 10 Essential Public Health Services (EPHSs).5 Each of the 10 EPHSs encompasses a PHAB Domain (1---10) under which standards and measures provide guidance to local health departments (LHDs) for meeting the intent of each essential public health service.1 Additional PHAB domains include Domain 11 (Administrative and Management Capacity) and Domain 12 (Engagement of the Public Health Governing Entity).1 With the release of these standards and the measures grounded in quality improvement (QI),6,7 LHDs, for the first time, have a QI mechanism upon which to benchmark performance nationally.8 Previous studies identified the potential benefits of QI through accreditation, including reduction of costs and increased ability to meet specific customer demands, increased profits, improved efficiency, and productivity and survivability.9,10 However, because difficult economic times (particularly during the recent 2008---2010 economic recession) forced health departments to adjust
Objectives. We identified the characteristics of local health departments (LHDs) that intended to seek accreditation, and also examined the association between that intent and a complete community health assessment (CHA), community health improvement plan, agency strategic plan, or other specific accreditation requirements. Methods. We analyzed data from the 2010 profile survey of LHDs conducted by the National Association of County and City Health Officials (n = 267). Results. Those LHDs that conducted a CHA (adjusted odds ratio [AOR] = 0.62; 95% confidence interval [CI] = 0.38, 1.00; P = .05) and developed a strategic plan (AOR = 0.30; 95% CI = 0.12, 0.74; P = .01) were less likely to have an intent to pursue accreditation in the first 2 years of the program. By contrast, those LHDs that were engaged in quality improvement (QI) activities were approximately 2.6 times more likely to pursue accreditation compared with those LHDs that did not have any QI activities (P < .001). Conclusions. Based on our findings, national public health accreditation might be the vehicle LHDs could use to improve their operating environments, better manage their resources, and reap the rewards associated with meeting national industry standards. (Am J Public Health. 2015;105:S353–S359. doi:10.2105/AJPH. 2015.302568)
services and staffing levels, public health has become strained under the increasing pressures of infectious and chronic diseases, emergency preparedness concerns, and the loss of essential services.11--14 At this time in the history of public health in the United States, the mission of PHAB “to promote and protect the health of communities by advancing the quality and performance of all public health departments in the US.”1,3,4 has become increasingly relevant. Therefore, by providing a national mechanism for QI through benchmarking performance against PHAB standards and measures, public health accreditation provides governmental public health “a roadmap for its QI journey.”8(p57) Combined with investments and training in QI activities, this roadmap can be particularly valuable for those LHDs that have not implemented QI initiatives in the past.15,16 Governmental public health agencies, specifically LHDs seeking accreditation, must satisfy PHAB prerequisites for accreditation by completing a community health assessment (CHA), a community health improvement plan
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
(CHIP), and an agency strategic plan.1 The CHA, addressed in PHAB Domain 1, is a systematic, collaborative method to assess the health needs of a community.17,18 The CHIP is a long-term, systematic plan for addressing those needs,18 and the strategic plan is the result of a deliberative decision-making process that defines the agency’s direction through the identification of a mission and vision with goals and objectives19; both are addressed in PHAB Domain 5.1 Accreditation also requires LHDs to adhere to standards for each of the PHAB domains, such as public health investigations (Domain 2), education and prevention activities (Domain 3), QI (Domain 9), financial management (Domain 11), and governance (Domain 12).1 Beaudry et al.7 outlined opportunities for QI that are evident in the completion of the 3 prerequisites and satisfaction of the standards and measures of each PHAB domain; however, Shah et al.20 also identified perceived LHD barriers to PHAB national voluntary accreditation that included time and effort exceeding benefits.
Carman and Timsina | Peer Reviewed | Research and Practice | S353
RESEARCH AND PRACTICE
We sought to identify the characteristics of LHDs that indicated in the 2010 National Association of County and City Health Officials (NACCHO) survey that they intended to seek accreditation in the first 2 years of voluntary national accreditation, and to determine if an association existed between that intent and the LHD having already completed a CHA, CHIP, agency strategic plan, or other specific accreditation standards mapped to the 10 EPHSs. In this study, we used “intent to pursue accreditation in the first two years” as a proxy for accreditation. We hypothesized that those LHDs that managed activities according to the 10 EPHSs before accreditation and that widely used management functions (e.g., planning, organization of resources, QI, and adherence to goals and objectives)21,22 would be among the first to apply for, and potentially receive, national voluntary public health accreditation quickly because of their previous work.
d
d
d
d
METHODS Our empirical analysis used data from the 2010 profile survey of the LHDs conducted by the NACCHO (response rate = 82%).23 The purpose of the 2010 NACCHO profile survey was to better understand the structure, function, and capacities of LHDs in the United States. Core questionnaires were designed to collect information on core questions and 2 sets of modules collected information based on supplemental questions. Using stratified random sampling (without replacement) with strata defined by the population size of the LHD jurisdiction, LHDs were randomly assigned to receive only the core (1316 LHDs) or the core plus 1 of the 2 modules (core + module 1 = 624 LHDs; core + module 2 = 625).
d
d
Variables We recoded intent to pursue accreditation with a reference time of the next 2 years after the survey23 (“intent”) into 2 categories: intent (agreed or strongly agreed) or no intent (disagreed or strongly disagreed). We assessed 8 of the 12 PHAB standards: d
d
Community health assessment: LHDs that completed a CHA in the previous 5 years. Investigation in public health: we used epidemiological and surveillance activities of communicable or infectious disease and the
laboratory services conducted or contracted out by an LHD as proxies of ensuring investigation. Population-based prevention activities: population-based primary prevention activities in nutrition and physical activity conducted or contracted out by an LHD. Community health improvement plan and strategic plan: we regarded those LHDs that developed CHIPs using the results of a CHA as having developed a CHIP. Similarly, we regarded those LHDs that developed a comprehensive agency-wide strategic plan as having developed a comprehensive strategic plan. Regulation, inspection, or licensing activities: we computed the total number of activities that were regulated, inspected, or licensed directly or contracted out by an LHD as an estimate of the number of services used to enforce public health laws. Quality improvement activities: PHAB defines QI as a formal, systematic approach applied to the processes underlying public health programs and services to achieve measurable improvements.18 We regarded those LHDs that implemented any kind of formal QI activity as having the QI activities. Effective financial management system: we assessed administrative and management capacity by using proxy measures, such as: (1) the ability to distinguish between funds that originated from the state and funds that originated from a federal source and were distributed by a state agency, (2) a current operating budget compared with the previous fiscal year, and (3) the expected budget in the next fiscal year compared with the current fiscal year. Control variables: the control variables of our study were the top executive’s demographic and work-related variables and characteristics of an LHD: age, gender, race, highest level of education, work status, and tenure of top executives; and per capita revenue, per capita expenditure, population size, and governance and jurisdiction classifications of an LHD.
Study Design and Statistical Analyses We included a cross-sectional study of a subsample of LHDs from the NACCHO 2010
S354 | Research and Practice | Peer Reviewed | Carman and Timsina
profile survey that “agreed/strongly agreed” or “disagreed/strongly disagreed” with an intent to pursue accreditation. We dropped LHDs with neutral or missing responses on the intent (n = 181) from the analysis. Our final study sample included 267 LHDs. Less than 13% of the categorical covariates and less than 20% of the continuous covariates were missing across the variables in the data. We used PROC MI in SAS version 9.3 (SAS Institute, Cary, NC) to impute missing data. This approach drew from a multivariate normal distribution of all the variables in the imputation model using a Markov Chain, Monte Carlo (MCMC) algorithm.24,25 We obtained 12 different complete data sets in the imputation process. We visually inspected all parameters in the multiple imputation process using trace plots and autocorrelation plots. The trace plots showed no particular long-term pattern that suggested the convergence of the MCMC algorithm produced the imputed values from the model of a stable distribution.25 Similarly, the autocorrelation plots suggested that there was no autocorrelation between the imputed data sets drawn from successive iterations of the same multiple imputation chain.25 The regression results based on this imputed data were quite stable and close to the results obtained from maximum likelihood.24,26 For each imputed data set, we performed multivariate logistic regression using generalized estimating equations with the REPEATED statement in the GENMOD procedure. We assumed an unstructured correlation for all responses of LHDs within each state. We wrote the model as: logit (lij) = b0+ b1Xij+ b2Zij, where lij = marginal expectation of having intent = E(Yij Xij, Zij); Yij (i = number of individual; j = number of cluster/state) denoted whether the jth LHD in ith state had an intent; Xij and Zij were the vectors of independent and control variables, respectively; b1 and b2 were the corresponding effects; the variance of the response variable was Var(Yij) = lij(1-lij); and the correlation structure between 2 LHDs in a state was assumed to be Corr(Yij, Yik) = ajk. We then used SAS’s PROC MIANALYZE to combine the results from these imputed data sets. Compared with the compound symmetry correlation structure estimates and SEs, the unstructured correlation structure in the
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
generalized estimating equations were found to be robust and had the least SE. We computed the adjusted odds ratio (AOR) and 95% confidence intervals (CIs) for each predictor as measures of their association with having intent. SAS 9.3 was used for all analyses.
TABLE 1—Characteristics of Local Health Departments (n = 267) and Their Intent to Pursue Public Health Accreditation, 2010 Intent to Pursue Accreditation Variables
No. (%)
Yes, % or Mean
No, % or Mean
White
245 (93.16)
52.65
47.35
Other
18 (6.84)
61.11
38.89
77 (30.08)
48.05
51.95
Master’s
114 (44.53)
50.00
50.00
Doctoral
65 (25.39)
63.08
36.92
Race
RESULTS Table 1 provides the characteristics of LHDs, the results from the bivariate analysis of each of these characteristics, and the intent. In the sample, a majority of LHD directors were women, White, between 50 and 59 years of age, worked full-time, had completed a master’s degree, and had an average tenure in their positions of slightly more than 9 years. The majority of LHDs in the sample served singlecounty jurisdictions of less than 25 000 people and was governed locally with an average per capita revenue of almost $62 and per capita expenditures of almost $61. Two thirds of the LHDs conducted a CHA, and approximately 6 of 10 completed a CHIP. Slightly more LHDs in the sample (51.88%) investigated health problems and environmental public health hazards. However, approximately 8 of 10 LHDs in the sample provided communitybased prevention activities. The majority of LHDs in the sample (93.96%) had not completed a comprehensive strategic plan for their agency. On average, an LHD conducted approximately 10 regulatory, inspection, or licensing services to enforce public health laws. In addition, 57.14% of the LHDs conducted QI activities. Slightly more LHDs in the sample (56.09%) were able to distinguish between funds that originated from their state and funds that originated from a federal source and were distributed by a state agency. In addition, a slight majority of LHDs in the sample (55.97%) had approximately the same or greater operating budget from the previous year, but expected the budget to be less the following fiscal year (57.08%). The bivariate analysis, in which we used generalized estimating equations with an unstructured covariance structure and adjusted for the autocorrelation between LHDs in a state, resulted in our observations that gender, race, education, tenure, per capita revenue, per capita expenditure, age of the top executive of an LHD, population served by jurisdiction,
Education Associate’s and bachelor’s
P .16
.14
Tenure (mean)
263 (9.08)
8.79
9.42
.43
Mean per capita revenue
215 (61.56)
63.56
59.12
.26
Mean Per capita expenditure
230 (60.83)
64.99
55.97
.17
< 39 40–49
22 (8.46) 56 (21.54)
63.64 53.57
36.36 46.43
50–59
117 (45.00)
50.43
49.57
‡ 60
65 (25.00)
52.31
47.69
Full-time
253 (95.11)
52.96
47.04
Part-time
13 (4.89)
53.85
46.15
114 (45.35) 149 (56.65)
54.39 52.35
45.61 47.65
< 25 000
70 (26.22)
41.43
58.57
25 000–49 999
51 (19.10)
43.14
56.86
50 000–99 999
42 (15.73)
57.14
42.86
100 000–499 999
64 (23.97)
62.50
37.50
‡ 500 000
40 (14.98)
65.00
35.00
Governance State
27 (10.11)
48.15
51.85
Local
212 (79.40)
52.36
47.64
Shared
28 (10.49)
60.71
39.29
27 (10.11)
55.56
44.44 47.29
Age, y
.18
Work status of top executive
.02
Gender Men Women
.06
Population size
.09
.37
Jurisdiction City
.11
County
203 (76.03)
52.71
Multicity
10 (3.75)
70.00
30.00
27 (10.11)
44.44
55.56
Yes
181 (67.79)
50.28
49.72
No
86 (32.21)
58.14
41.86
Yes
128 (48.12)
39.06
60.94
No
138 (51.88)
55.07
44.93
208 (79.39) 54 (20.61)
54.33 44.44
45.67 55.56
Multicounty Community health assessment
.39
Investigation in public health
< .001
Population-based prevention activities Yes No
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
.22
Continued
Carman and Timsina | Peer Reviewed | Research and Practice | S355
RESEARCH AND PRACTICE
TABLE 1—Continued Community health improvement plan
.21
Yes
161 (60.75)
54.04
45.96
No
104 (39.25)
50.96
49.04
16 (6.04)
62.50
37.50
249 (93.96) 242 (9.57)
52.61 10.40
47.39 8.69
Comprehensive strategic planning
.02
Yes No Regulation, inspection, or licensing activities (mean) QI activities Yes
152 (57.14)
60.53
39.47
No
114 (42.86)
42.98
57.02
Yes
129 (56.09)
54.26
45.74
No
101 (43.91)
55.45
44.55
107 (44.03)
51.40
48.60
136 (55.97)
54.41
45.59
Less
133 (57.08)
51.88
48.12
Approximately same or greater
100 (42.92)
53.00
47.00
Distinction between funds
Current operating budget compared with previous FY Less Approximately same or greater
.6 < .001
.88
.99
Expected budget in the following FY
.98
Note. FY = fiscal year; QI = quality improvement. P values are from bivariate generalized estimating equation analyzing intent to pursue accreditation.
governance structure, and type of jurisdiction were not related to having intent. However, the work status of top executive was significantly associated with having the intent (P = .02). Besides these control variables, the variables that were significantly associated with intent were investigation in public health carried out or contracted by LHDs (P < .001), development of comprehensive strategic planning (P = .02), and conducting any formal QI activities (P < .001). In the bivariate analysis, there were no significant differences between having and not having intent for activities such as CHA, population-based prevention, and CHIP; regulatory, inspection or licensing activities; and the activities that were used to define an effective financial management system. After we adjusted for the autocorrelation between LHDs within a state and controlled for the effect of the covariates listed in Table 2, an LHD that served a population of more than 500 000, compared with an LHD that served a population of less than 25 000, was found to be approximately 3 times more likely to have an intent to pursue accreditation (P = .03). In addition, compared with single-county jurisdiction, multicity jurisdictions were
approximately 6.6 times more likely to have an intent (P = .02), and those LHDs with QI activities were approximately 2.6 times more likely to have an intent compared with those LHDs that did not have any QI activities (P < .001). The service delivery domains, such as having services to investigate disease in public health; services to assure populationbased preventive activities; programs to develop CHIP; services to regulate, inspect, or license; and those proxy standards and measures of financial acumen and sustainability in Domain 11 (e.g., the ability to distinguish between state and federal funds), and having a current and expected operating budget approximately less than the previous one, did not significantly predict intent (Table 2). LHDs that conducted a CHA (AOR = 0.62; 95% CI = 0.38, 1.00; P = .05) and developed a strategic plan (AOR = 0.30; 95% CI = 0.12, 0.74; P = .01) were each less likely to intend to pursue accreditation.
DISCUSSION We expected to find those LHDs who had completed 1 or more of the PHAB
S356 | Research and Practice | Peer Reviewed | Carman and Timsina
prerequisites (i.e., CHA, CHIP, and strategic plan), and thus, had already invested the time needed to complete many accreditation requirements, would indicate their intention to apply for accreditation within the first 2 years. If our hypothesis was proven to be true, the concept of national public health accreditation as a QI vehicle for LHDs could be questioned, and accreditation could be viewed as merely a “stamp of approval” for LHDs already being managed according to the 10 EPHSs, and thus, the PHAB standards. The results of our study revealed the LHDs that indicated intent in the first 2 years of the program were LHDs that served large populations (i.e., > 500 000), multicity jurisdictions, and LHDs that completed QI activities. Larger LHDs might have more staff, were trained in QI, and had more community partners through which to work toward meeting accreditation standards. Also, previous investments in QI activities might have poised LHDs to have an early understanding of the importance of an accreditation system based in QI. Our study revealed that the LHDs that indicated intent to pursue accreditation did not complete 2 of the required PHAB prerequisites (a CHA or an agency strategic plan). Although the impact of challenging economic times, such as the recent 2008 to 2010 recession, might have affected the ability of LHDs to focus on completion of a CHA or agency strategic plan documents,11 both the process of developing these PHAB prerequisites and their use provided critical planning and fundamental resource allocation direction to the agency during these times. From the perspective of “on the ground” leaders of LHDs, the accreditation standards, specifically, the CHA and strategic plan, provided a credible roadmap for future improvement and followed the assessment by Beitsch et al. of accreditation for an LHD as a “roadmap for its QI journey.”8 Our study findings indicated that those LHD leaders with a master’s degree were less likely to pursue accreditation in the first 2 years of the program. Based on a small subpopulation study (associate’s and bachelor’s degrees with intent: n = 37; master’s degree with intent: n = 57), our findings, although inconclusive (P = .05), indicated that LHD leaders with a master’s Degree were less likely to pursue accreditation. This finding requires a large
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
TABLE 2—Adjusted Odds Ratios (AORs) of Local Health Department (LHD) Intent to Pursue Accreditation: Public Health Accreditation, 2010 Variables
AOR (95% CI)
P
White
1.30 (0.53, 3.19)
.55
Other
1.00
Race
Education Associate’s and bachelor’s Master’s
1.00 0.58 (0.33, 1.01)
.05
Doctoral
0.81 (0.33, 2.00)
.64
Tenure
0.99 (0.96, 1.03)
.69
Per capita revenue
0.99 (0.97, 1.02)
.45
Per capita expenditure
1.01 (0.98, 1.04)
.44
Age, y < 39
1.00
40–49 50–59
0.74 (0.35, 1.57) 0.60 (0.28, 1.29)
.43 .18
‡ 60
0.83 (0.32, 2.12)
.69
Work status of top executive Full-time
1.00
Part-time
0.71 (0.27, 1.89)
.49
Gender Male Female Population size
1.00 1.38 (0.84, 2.25)
.2
< 25 000
1.00
25 000–49 999
1.44 (0.64, 3.23)
.37
50 000–99 999
2.30 (0.89, 5.98)
.09
100 000–499 999
2.13 (0.88, 5.12)
.09
‡ 500 000
3.05 (1.12, 8.35)
.03
Governance State Local
1.00 1.04 (0.23, 4.63)
.96
Shared
1.15 (0.20, 6.51)
.88
City
1.56 (0.69, 3.52)
.27
County
1.00
Multicity
6.60 (1.42, 30.78)
.02
Multicounty
0.73 (0.24, 2.22)
.57
0.62 (0.38, 1.00)
.05
Jurisdiction
Community health assessment Yes No
1.00
Investigation in public health Yes
1.35 (0.87, 2.10)
No
1.00
.18
Population-based prevention activities Yes
1.37 (0.66, 2.82)
No
1.00
.39
Continued
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
sample study to further identify relationships between LHD leader degree status and intent. However, if accreditation was considered an innovation in public health, the Diffusion of Innovation theory by Rogers27 could be useful to interpret this finding. In the Diffusion of Innovation age, financial lucidity and risk tolerance are used to explain innovativeness. Although our study evaluated age of LHD leaders, we did not have variables to assess financial lucidity and risk tolerance. There was no maximum likelihood method of estimating parameters when the response variable was discrete and correlated, and the distribution of the response variable was unspecified. As an alternative, we used general estimating equations that extended the concept of maximum likelihood estimation by imposing a covariance structure,24 and therefore, we sufficiently adjusted for the correlation of the responses between LHDs. Our study findings were robust and had minimum variance. However, this was a cross-sectional study, and thus, the causal relationship of estimating intent to seek accreditation could not be determined. The exclusion of other important PHAB domains that were not available in the NACCHO data set might be a study limitation. We used intent as a proxy for accreditation. Intent could only approximate the likelihood of obtaining accreditation. However, because of the limited availability of variables that measured LHDs actually pursuing accreditation in 2010, we were restricted to use intent as its proxy. In addition, intent in some published and unpublished earlier studies was shown to be a consistent predictor of the likelihood that one would (or would not) execute a behavior.28---30 The first accredited health departments were announced in March 2013, which was beyond the reference time for the intent to pursue accreditation in the first 2 years of national voluntary accreditation we used in this study. However, future studies could examine the association between intent to pursue accreditation and the attainment of accredited status. Restricting the study sample only to those who either agreed or strongly agreed or disagreed or strongly disagreed about their intent to seek voluntary accreditation in the next 2 years might have introduced some bias. However, based on population size (test of
Carman and Timsina | Peer Reviewed | Research and Practice | S357
RESEARCH AND PRACTICE
(e-mail: [email protected]). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted January 8, 2015.
TABLE 2—Continued Community health improvement plan Yes
1.03 (0.63, 1.69)
No
1.00
.9
Comprehensive strategic planning Yes
0.30 (0.12, 0.74)
.01
1.00 1.01 (0.95, 1.06)
.8
Yes
2.64 (1.59, 4.37)
< .001
No
1.00
No Regulation, inspection, or licensing activities QI activities
Distinction between funds Yes
0.85 (0.50, 1.47)
No
1.00
Current operating budget compared with previous FY Less Approximately same or greater
0.90 (0.53, 1.53)
.56
.7
1.00
Expected budget in the following FY Less
1.03 (0.57, 1.84)
Approximately same or greater
1.00
.93
Note. AOR = adjusted odds ratio; CI = confidence interval; FY = fiscal year; QI = quality improvement. AORs derived from multivariate generalized estimating equation adjusting for the autocorrelation between LHDs within each state.
mean by type of intent), and governance structure, jurisdiction type or reporting as region (v2 test) did not differ independently with intent categorized as agree or strongly agree, neither agreed nor disagreed (neutral), or disagree or strongly disagree. For ease of interpretation and comparability, we decided to drop the observation that had neutral responses in the analysis. Our results concurred with those found by Shah et al.31 in a study that grouped intent into 3 categories (strongly or disagree, natural, agree or strongly agree) and assessed the impact of PHAB prerequisites on the intention to pursue accreditation using a broader reference period than our study.
Conclusion There is no question that this is a pivotal time in public health. The changing economic environment, combined with the passage of the Affordable Care Act, represent major changes in the operating environment of LHDs. These external stressors, combined with new challenges, demand that LHDs pursue increased quality in the performance of population health services. Early supporters of national public health accreditation described it as “a platform for quality improvement.”3 The results of our
study supported this concept by identifying LHDs that, before accreditation, had not completed a CHA or strategic plan, but also showed intent. These LHDs will experience the benefits of QI through systematic strategic planning and connecting with community partners through CHAs and improvement planning as they complete prerequisites for accreditation. Based on our study findings, national public health accreditation might be the vehicle LHDs could use to improve operating environments, better manage resources, and reap rewards associated with meeting national industry standards. Moreover, future studies on accreditation should consider looking at the interaction effects of the prerequisites of accreditation and population size, resources, and the tenure of top executives of LHDs. j
About the Authors Angela L. Carman is with the University of Kentucky, College of Public Health, Lexington. Both authors are with the National Coordinating Center for Public Health Services and Systems Research and Practice-Based Research Networks, Lexington. Correspondence should be sent to Angela L. Carman, University of Kentucky, College of Public Health, National Coordinating Center for Public Health Services and Systems Research and Practice-Based Research Networks, 111 Washington Avenue, Room 105C, Lexington, KY 40536
S358 | Research and Practice | Peer Reviewed | Carman and Timsina
Contributors A. L. Carman conceptualized the research question, study design, analysis, literature review, content revision, and article preparation. L. Timsina conducted the data analysis and contributed to the study design, literature review, content revision, and article preparation.
Acknowledgments We would like to thank F. Douglas Scutchfield, MD, James W. Holsinger Jr, MD, PhD, David Fardo, PhD, Rachel Hogg, DrPH, and Margaret McGladrey, MA, for their expert advice and comments on this article.
Human Participant Protection The University of Kentucky Institutional Review Board committee determined that this study did not need review because the unit of analysis did not meet the definition of human subjects.
References 1. PHAB. Public Health Department Accreditation Background. 2011. Available at: www.phaboard.org. Accessed November 15, 2011. 2. Ingram RC, Bender K, Wilcox R, Kronstadt J. A consensus-based approach to national public health accreditation. J Public Health Manag Pract. 2014;20 (1):9---13. 3. Russo P. Accreditation of public health agencies: a means, not an end. J Public Health Manag Pract. 2007;13(4):329---331. 4. Bender K, Benjamin G, Fallon MM, Jarris PE, Libbey PM. Exploring accreditation: striving for a consensus model. J Public Health Manag Pract. 2007;13(4):334---336. 5. Centers for Disease Control and Prevention. National Public Health Performance Standards---Essential Services. 2013. Available at: http://www.cdc.gov/ nphpsp/essentialservices.html. Accessed November 14, 2014. 6. McLees AW, Thomas CW, Nawaz S, Young AC, Rider N, Davis M. Advances in public health accreditation readiness and quality improvement: evaluation findings from the national public health improvement initiative. J Public Health Manag Pract. 2014;20(1):29---35. 7. Beaudry ML, Bialek R, Moran JW Jr. Using quality improvement tools and methods throughout the accreditation lifecycle. J Public Health Manag Pract. 2014;20 (1):49---51. 8. Beitsch LM, Riley W, Bender K. Embedding quality improvement into accreditation: evolving from theory to practice. J Public Health Manag Pract. 2014;20(1):57---60. 9. Hamm MS. Quality Improvement Initiatives in Accreditation: Private Sector Examples and Key Lessons for Public Health. Princeton, NJ: Robert Wood Johnson Foundation; 2007. 10. Carman AL, Timsina L, Scutchfield FD. Quality improvement activities of local health departments during the 2008---2010 economic recession. Am J Prev Med. 2014;46(2):171---174. 11. Willard R, Shah GH, Leep CJ, Leighton K. Impact of the 2008-2010 economic recession on local health
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
departments. J Public Health Manag Pract. 2012;18 (2):106---114. 12. National Association of County and City Health Officials. Local Health Department Job Losses and Program Cuts: Findings From January/February 2010 Survey. Washington, DC: National Association of County and City Health Officials; 2010.
30. Speaking of Health. Assessing Health Communication Strategies for Diverse Populations. Washington, DC: The National Academies Press; 2002. 31. Shah GH, Beatty K, Leep C. Do PHAB accreditation prerequisites predict local health departments’ intentions to seek voluntary national accreditation? Front Public Health Serv Syst Res. 2013;2(3):article 4.
13. Honore PA, Scott W. Priority Areas for Improvement of Quality in Public Health. Washington, DC: Department of Health and Human Services; 2010. 14. Riley WJ, Parsons HM, Duffy GL, Moran JW, Henry B. Realizing transformational change through quality improvement in public health. J Public Health Manag Pract. 2010;16(1):72---78. 15. Boe DT, Riley W, Parson H. Improving service delivery in a county health department WIC clinic: an application of statistical process control techniques. Am J Public Health. 2009;99(9):1619---1625. 16. Luo H, Sotnikov S, McLees A, Stokes S. Factors driving the adoption of quality improvement initiatives in local health departments: results from the 2010 profile study. J Public Health Manag Pract. 2015;21(2):176---185. 17. Turnock B. Public Health: What It Is and How It Works. Burlington, MA: Jones and Bartlett; 2011. 18. Public Health Accreditation Board. Acronyms and glossary of terms. 2012. Available at: http://www.phaboard. org/wp-content/uploads/PHAB-Acronyms-and-Glossary-ofTerms-Version-1.0.pdf. Accessed March 12, 2013. 19. Ginter P, Duncan WJ, Swayne L. Strategic Management of Health Care Organizations. 7th ed. San Francisco, CA: Jossey-Bass; 2013. 20. Shah GH, Leep C, Ye J, Sellers K, Liss-Levinson R, Williams K. Public health agencies’ level of engagement in and perceived barriers to PHAB national voluntary accreditation. J Public Health Manag Pract. 2015;21 (2):107---115. 21. Schermerhorn J. Management. 10th ed. Hoboken, NJ: John Wiley & Sons; 2010. 22. Downey LH, Anyaegbunam C, Scutchfield FD. Dialogue to deliberation: expanding the Empowerment education model. Am J Health Behav. 2009;33(1):26---36. 23. National Association of County and City Health Officials. 2010 National Profile of Local Health Departments. Washington, DC: National Association of County and City Health Officials; 2011. 24. Little R, Rubin D. 2nd ed. Statistical Analysis With Missing Data. Hoboken, NJ: Wiley-Interscience; 2002. 25. Institute for Digital Research and Education. Statistical Computing Seminars: Multiple Imputation in SAS, Part 1. 2014. Available at: http://www.ats.ucla.edu/stat/ sas/seminars/missing_data/part1.htm. Accessed February 25, 2014. 26. Landerman LR, Land KC, Pieper CF. An empirical evaluation of the predictive mean matching method for imputing missing values. Sociol Methods Res. 1997;26(1):3-- 33. 27. Rogers E. Diffusion of Innovations. 5th ed. New York: Free Press; 2003. 28. Sheppard BH, Hartwick J, Warshaw PR. The theory of reasoned action: a meta-analysis of past research with recommendations for modifications and future research. J Consum Res. 1988;15(3):325---343. 29. Van den Putte B. 20 Years of the Theory of Reasoned Action of Fishbein and Aizen: A Meta-analysis. Amsterdam: University of Amsterdam; 1991.
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
Carman and Timsina | Peer Reviewed | Research and Practice | S359
Public Health Accreditation: Rubber Stamp or Roadmap for Improvement Angela L. Carman, DrPH, and Lava Timsina, MPH
A variety of events in public health have focused attention on the need to improve public health agency performance. The 1988 Institute of Medicine report, “The Future of Public Health,” characterized public health as a “system in disarray.” In 2002, the Institute of Medicine released a second major report, “The Future of Public Health in the 21st Century,” that recommended exploration of public health accreditation as a means of improving performance and accountability for governmental public health departments. In September 2011, the Public Health Accreditation Board (PHAB) launched the first voluntary accreditation system with consensus standards and measures for state, local, and tribal public health agencies in the United States.1,2 The mission of PHAB is to promote and protect the health of communities by advancing quality and performance of all public health departments in the United States.1,3,4 National public health accreditation domains, standards, and measures are structured around the 10 Essential Public Health Services (EPHSs).5 Each of the 10 EPHSs encompasses a PHAB Domain (1---10) under which standards and measures provide guidance to local health departments (LHDs) for meeting the intent of each essential public health service.1 Additional PHAB domains include Domain 11 (Administrative and Management Capacity) and Domain 12 (Engagement of the Public Health Governing Entity).1 With the release of these standards and the measures grounded in quality improvement (QI),6,7 LHDs, for the first time, have a QI mechanism upon which to benchmark performance nationally.8 Previous studies identified the potential benefits of QI through accreditation, including reduction of costs and increased ability to meet specific customer demands, increased profits, improved efficiency, and productivity and survivability.9,10 However, because difficult economic times (particularly during the recent 2008---2010 economic recession) forced health departments to adjust
Objectives. We identified the characteristics of local health departments (LHDs) that intended to seek accreditation, and also examined the association between that intent and a complete community health assessment (CHA), community health improvement plan, agency strategic plan, or other specific accreditation requirements. Methods. We analyzed data from the 2010 profile survey of LHDs conducted by the National Association of County and City Health Officials (n = 267). Results. Those LHDs that conducted a CHA (adjusted odds ratio [AOR] = 0.62; 95% confidence interval [CI] = 0.38, 1.00; P = .05) and developed a strategic plan (AOR = 0.30; 95% CI = 0.12, 0.74; P = .01) were less likely to have an intent to pursue accreditation in the first 2 years of the program. By contrast, those LHDs that were engaged in quality improvement (QI) activities were approximately 2.6 times more likely to pursue accreditation compared with those LHDs that did not have any QI activities (P < .001). Conclusions. Based on our findings, national public health accreditation might be the vehicle LHDs could use to improve their operating environments, better manage their resources, and reap the rewards associated with meeting national industry standards. (Am J Public Health. 2015;105:S353–S359. doi:10.2105/AJPH. 2015.302568)
services and staffing levels, public health has become strained under the increasing pressures of infectious and chronic diseases, emergency preparedness concerns, and the loss of essential services.11--14 At this time in the history of public health in the United States, the mission of PHAB “to promote and protect the health of communities by advancing the quality and performance of all public health departments in the US.”1,3,4 has become increasingly relevant. Therefore, by providing a national mechanism for QI through benchmarking performance against PHAB standards and measures, public health accreditation provides governmental public health “a roadmap for its QI journey.”8(p57) Combined with investments and training in QI activities, this roadmap can be particularly valuable for those LHDs that have not implemented QI initiatives in the past.15,16 Governmental public health agencies, specifically LHDs seeking accreditation, must satisfy PHAB prerequisites for accreditation by completing a community health assessment (CHA), a community health improvement plan
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
(CHIP), and an agency strategic plan.1 The CHA, addressed in PHAB Domain 1, is a systematic, collaborative method to assess the health needs of a community.17,18 The CHIP is a long-term, systematic plan for addressing those needs,18 and the strategic plan is the result of a deliberative decision-making process that defines the agency’s direction through the identification of a mission and vision with goals and objectives19; both are addressed in PHAB Domain 5.1 Accreditation also requires LHDs to adhere to standards for each of the PHAB domains, such as public health investigations (Domain 2), education and prevention activities (Domain 3), QI (Domain 9), financial management (Domain 11), and governance (Domain 12).1 Beaudry et al.7 outlined opportunities for QI that are evident in the completion of the 3 prerequisites and satisfaction of the standards and measures of each PHAB domain; however, Shah et al.20 also identified perceived LHD barriers to PHAB national voluntary accreditation that included time and effort exceeding benefits.
Carman and Timsina | Peer Reviewed | Research and Practice | S353
RESEARCH AND PRACTICE
We sought to identify the characteristics of LHDs that indicated in the 2010 National Association of County and City Health Officials (NACCHO) survey that they intended to seek accreditation in the first 2 years of voluntary national accreditation, and to determine if an association existed between that intent and the LHD having already completed a CHA, CHIP, agency strategic plan, or other specific accreditation standards mapped to the 10 EPHSs. In this study, we used “intent to pursue accreditation in the first two years” as a proxy for accreditation. We hypothesized that those LHDs that managed activities according to the 10 EPHSs before accreditation and that widely used management functions (e.g., planning, organization of resources, QI, and adherence to goals and objectives)21,22 would be among the first to apply for, and potentially receive, national voluntary public health accreditation quickly because of their previous work.
d
d
d
d
METHODS Our empirical analysis used data from the 2010 profile survey of the LHDs conducted by the NACCHO (response rate = 82%).23 The purpose of the 2010 NACCHO profile survey was to better understand the structure, function, and capacities of LHDs in the United States. Core questionnaires were designed to collect information on core questions and 2 sets of modules collected information based on supplemental questions. Using stratified random sampling (without replacement) with strata defined by the population size of the LHD jurisdiction, LHDs were randomly assigned to receive only the core (1316 LHDs) or the core plus 1 of the 2 modules (core + module 1 = 624 LHDs; core + module 2 = 625).
d
d
Variables We recoded intent to pursue accreditation with a reference time of the next 2 years after the survey23 (“intent”) into 2 categories: intent (agreed or strongly agreed) or no intent (disagreed or strongly disagreed). We assessed 8 of the 12 PHAB standards: d
d
Community health assessment: LHDs that completed a CHA in the previous 5 years. Investigation in public health: we used epidemiological and surveillance activities of communicable or infectious disease and the
laboratory services conducted or contracted out by an LHD as proxies of ensuring investigation. Population-based prevention activities: population-based primary prevention activities in nutrition and physical activity conducted or contracted out by an LHD. Community health improvement plan and strategic plan: we regarded those LHDs that developed CHIPs using the results of a CHA as having developed a CHIP. Similarly, we regarded those LHDs that developed a comprehensive agency-wide strategic plan as having developed a comprehensive strategic plan. Regulation, inspection, or licensing activities: we computed the total number of activities that were regulated, inspected, or licensed directly or contracted out by an LHD as an estimate of the number of services used to enforce public health laws. Quality improvement activities: PHAB defines QI as a formal, systematic approach applied to the processes underlying public health programs and services to achieve measurable improvements.18 We regarded those LHDs that implemented any kind of formal QI activity as having the QI activities. Effective financial management system: we assessed administrative and management capacity by using proxy measures, such as: (1) the ability to distinguish between funds that originated from the state and funds that originated from a federal source and were distributed by a state agency, (2) a current operating budget compared with the previous fiscal year, and (3) the expected budget in the next fiscal year compared with the current fiscal year. Control variables: the control variables of our study were the top executive’s demographic and work-related variables and characteristics of an LHD: age, gender, race, highest level of education, work status, and tenure of top executives; and per capita revenue, per capita expenditure, population size, and governance and jurisdiction classifications of an LHD.
Study Design and Statistical Analyses We included a cross-sectional study of a subsample of LHDs from the NACCHO 2010
S354 | Research and Practice | Peer Reviewed | Carman and Timsina
profile survey that “agreed/strongly agreed” or “disagreed/strongly disagreed” with an intent to pursue accreditation. We dropped LHDs with neutral or missing responses on the intent (n = 181) from the analysis. Our final study sample included 267 LHDs. Less than 13% of the categorical covariates and less than 20% of the continuous covariates were missing across the variables in the data. We used PROC MI in SAS version 9.3 (SAS Institute, Cary, NC) to impute missing data. This approach drew from a multivariate normal distribution of all the variables in the imputation model using a Markov Chain, Monte Carlo (MCMC) algorithm.24,25 We obtained 12 different complete data sets in the imputation process. We visually inspected all parameters in the multiple imputation process using trace plots and autocorrelation plots. The trace plots showed no particular long-term pattern that suggested the convergence of the MCMC algorithm produced the imputed values from the model of a stable distribution.25 Similarly, the autocorrelation plots suggested that there was no autocorrelation between the imputed data sets drawn from successive iterations of the same multiple imputation chain.25 The regression results based on this imputed data were quite stable and close to the results obtained from maximum likelihood.24,26 For each imputed data set, we performed multivariate logistic regression using generalized estimating equations with the REPEATED statement in the GENMOD procedure. We assumed an unstructured correlation for all responses of LHDs within each state. We wrote the model as: logit (lij) = b0+ b1Xij+ b2Zij, where lij = marginal expectation of having intent = E(Yij Xij, Zij); Yij (i = number of individual; j = number of cluster/state) denoted whether the jth LHD in ith state had an intent; Xij and Zij were the vectors of independent and control variables, respectively; b1 and b2 were the corresponding effects; the variance of the response variable was Var(Yij) = lij(1-lij); and the correlation structure between 2 LHDs in a state was assumed to be Corr(Yij, Yik) = ajk. We then used SAS’s PROC MIANALYZE to combine the results from these imputed data sets. Compared with the compound symmetry correlation structure estimates and SEs, the unstructured correlation structure in the
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
generalized estimating equations were found to be robust and had the least SE. We computed the adjusted odds ratio (AOR) and 95% confidence intervals (CIs) for each predictor as measures of their association with having intent. SAS 9.3 was used for all analyses.
TABLE 1—Characteristics of Local Health Departments (n = 267) and Their Intent to Pursue Public Health Accreditation, 2010 Intent to Pursue Accreditation Variables
No. (%)
Yes, % or Mean
No, % or Mean
White
245 (93.16)
52.65
47.35
Other
18 (6.84)
61.11
38.89
77 (30.08)
48.05
51.95
Master’s
114 (44.53)
50.00
50.00
Doctoral
65 (25.39)
63.08
36.92
Race
RESULTS Table 1 provides the characteristics of LHDs, the results from the bivariate analysis of each of these characteristics, and the intent. In the sample, a majority of LHD directors were women, White, between 50 and 59 years of age, worked full-time, had completed a master’s degree, and had an average tenure in their positions of slightly more than 9 years. The majority of LHDs in the sample served singlecounty jurisdictions of less than 25 000 people and was governed locally with an average per capita revenue of almost $62 and per capita expenditures of almost $61. Two thirds of the LHDs conducted a CHA, and approximately 6 of 10 completed a CHIP. Slightly more LHDs in the sample (51.88%) investigated health problems and environmental public health hazards. However, approximately 8 of 10 LHDs in the sample provided communitybased prevention activities. The majority of LHDs in the sample (93.96%) had not completed a comprehensive strategic plan for their agency. On average, an LHD conducted approximately 10 regulatory, inspection, or licensing services to enforce public health laws. In addition, 57.14% of the LHDs conducted QI activities. Slightly more LHDs in the sample (56.09%) were able to distinguish between funds that originated from their state and funds that originated from a federal source and were distributed by a state agency. In addition, a slight majority of LHDs in the sample (55.97%) had approximately the same or greater operating budget from the previous year, but expected the budget to be less the following fiscal year (57.08%). The bivariate analysis, in which we used generalized estimating equations with an unstructured covariance structure and adjusted for the autocorrelation between LHDs in a state, resulted in our observations that gender, race, education, tenure, per capita revenue, per capita expenditure, age of the top executive of an LHD, population served by jurisdiction,
Education Associate’s and bachelor’s
P .16
.14
Tenure (mean)
263 (9.08)
8.79
9.42
.43
Mean per capita revenue
215 (61.56)
63.56
59.12
.26
Mean Per capita expenditure
230 (60.83)
64.99
55.97
.17
< 39 40–49
22 (8.46) 56 (21.54)
63.64 53.57
36.36 46.43
50–59
117 (45.00)
50.43
49.57
‡ 60
65 (25.00)
52.31
47.69
Full-time
253 (95.11)
52.96
47.04
Part-time
13 (4.89)
53.85
46.15
114 (45.35) 149 (56.65)
54.39 52.35
45.61 47.65
< 25 000
70 (26.22)
41.43
58.57
25 000–49 999
51 (19.10)
43.14
56.86
50 000–99 999
42 (15.73)
57.14
42.86
100 000–499 999
64 (23.97)
62.50
37.50
‡ 500 000
40 (14.98)
65.00
35.00
Governance State
27 (10.11)
48.15
51.85
Local
212 (79.40)
52.36
47.64
Shared
28 (10.49)
60.71
39.29
27 (10.11)
55.56
44.44 47.29
Age, y
.18
Work status of top executive
.02
Gender Men Women
.06
Population size
.09
.37
Jurisdiction City
.11
County
203 (76.03)
52.71
Multicity
10 (3.75)
70.00
30.00
27 (10.11)
44.44
55.56
Yes
181 (67.79)
50.28
49.72
No
86 (32.21)
58.14
41.86
Yes
128 (48.12)
39.06
60.94
No
138 (51.88)
55.07
44.93
208 (79.39) 54 (20.61)
54.33 44.44
45.67 55.56
Multicounty Community health assessment
.39
Investigation in public health
< .001
Population-based prevention activities Yes No
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
.22
Continued
Carman and Timsina | Peer Reviewed | Research and Practice | S355
RESEARCH AND PRACTICE
TABLE 1—Continued Community health improvement plan
.21
Yes
161 (60.75)
54.04
45.96
No
104 (39.25)
50.96
49.04
16 (6.04)
62.50
37.50
249 (93.96) 242 (9.57)
52.61 10.40
47.39 8.69
Comprehensive strategic planning
.02
Yes No Regulation, inspection, or licensing activities (mean) QI activities Yes
152 (57.14)
60.53
39.47
No
114 (42.86)
42.98
57.02
Yes
129 (56.09)
54.26
45.74
No
101 (43.91)
55.45
44.55
107 (44.03)
51.40
48.60
136 (55.97)
54.41
45.59
Less
133 (57.08)
51.88
48.12
Approximately same or greater
100 (42.92)
53.00
47.00
Distinction between funds
Current operating budget compared with previous FY Less Approximately same or greater
.6 < .001
.88
.99
Expected budget in the following FY
.98
Note. FY = fiscal year; QI = quality improvement. P values are from bivariate generalized estimating equation analyzing intent to pursue accreditation.
governance structure, and type of jurisdiction were not related to having intent. However, the work status of top executive was significantly associated with having the intent (P = .02). Besides these control variables, the variables that were significantly associated with intent were investigation in public health carried out or contracted by LHDs (P < .001), development of comprehensive strategic planning (P = .02), and conducting any formal QI activities (P < .001). In the bivariate analysis, there were no significant differences between having and not having intent for activities such as CHA, population-based prevention, and CHIP; regulatory, inspection or licensing activities; and the activities that were used to define an effective financial management system. After we adjusted for the autocorrelation between LHDs within a state and controlled for the effect of the covariates listed in Table 2, an LHD that served a population of more than 500 000, compared with an LHD that served a population of less than 25 000, was found to be approximately 3 times more likely to have an intent to pursue accreditation (P = .03). In addition, compared with single-county jurisdiction, multicity jurisdictions were
approximately 6.6 times more likely to have an intent (P = .02), and those LHDs with QI activities were approximately 2.6 times more likely to have an intent compared with those LHDs that did not have any QI activities (P < .001). The service delivery domains, such as having services to investigate disease in public health; services to assure populationbased preventive activities; programs to develop CHIP; services to regulate, inspect, or license; and those proxy standards and measures of financial acumen and sustainability in Domain 11 (e.g., the ability to distinguish between state and federal funds), and having a current and expected operating budget approximately less than the previous one, did not significantly predict intent (Table 2). LHDs that conducted a CHA (AOR = 0.62; 95% CI = 0.38, 1.00; P = .05) and developed a strategic plan (AOR = 0.30; 95% CI = 0.12, 0.74; P = .01) were each less likely to intend to pursue accreditation.
DISCUSSION We expected to find those LHDs who had completed 1 or more of the PHAB
S356 | Research and Practice | Peer Reviewed | Carman and Timsina
prerequisites (i.e., CHA, CHIP, and strategic plan), and thus, had already invested the time needed to complete many accreditation requirements, would indicate their intention to apply for accreditation within the first 2 years. If our hypothesis was proven to be true, the concept of national public health accreditation as a QI vehicle for LHDs could be questioned, and accreditation could be viewed as merely a “stamp of approval” for LHDs already being managed according to the 10 EPHSs, and thus, the PHAB standards. The results of our study revealed the LHDs that indicated intent in the first 2 years of the program were LHDs that served large populations (i.e., > 500 000), multicity jurisdictions, and LHDs that completed QI activities. Larger LHDs might have more staff, were trained in QI, and had more community partners through which to work toward meeting accreditation standards. Also, previous investments in QI activities might have poised LHDs to have an early understanding of the importance of an accreditation system based in QI. Our study revealed that the LHDs that indicated intent to pursue accreditation did not complete 2 of the required PHAB prerequisites (a CHA or an agency strategic plan). Although the impact of challenging economic times, such as the recent 2008 to 2010 recession, might have affected the ability of LHDs to focus on completion of a CHA or agency strategic plan documents,11 both the process of developing these PHAB prerequisites and their use provided critical planning and fundamental resource allocation direction to the agency during these times. From the perspective of “on the ground” leaders of LHDs, the accreditation standards, specifically, the CHA and strategic plan, provided a credible roadmap for future improvement and followed the assessment by Beitsch et al. of accreditation for an LHD as a “roadmap for its QI journey.”8 Our study findings indicated that those LHD leaders with a master’s degree were less likely to pursue accreditation in the first 2 years of the program. Based on a small subpopulation study (associate’s and bachelor’s degrees with intent: n = 37; master’s degree with intent: n = 57), our findings, although inconclusive (P = .05), indicated that LHD leaders with a master’s Degree were less likely to pursue accreditation. This finding requires a large
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
TABLE 2—Adjusted Odds Ratios (AORs) of Local Health Department (LHD) Intent to Pursue Accreditation: Public Health Accreditation, 2010 Variables
AOR (95% CI)
P
White
1.30 (0.53, 3.19)
.55
Other
1.00
Race
Education Associate’s and bachelor’s Master’s
1.00 0.58 (0.33, 1.01)
.05
Doctoral
0.81 (0.33, 2.00)
.64
Tenure
0.99 (0.96, 1.03)
.69
Per capita revenue
0.99 (0.97, 1.02)
.45
Per capita expenditure
1.01 (0.98, 1.04)
.44
Age, y < 39
1.00
40–49 50–59
0.74 (0.35, 1.57) 0.60 (0.28, 1.29)
.43 .18
‡ 60
0.83 (0.32, 2.12)
.69
Work status of top executive Full-time
1.00
Part-time
0.71 (0.27, 1.89)
.49
Gender Male Female Population size
1.00 1.38 (0.84, 2.25)
.2
< 25 000
1.00
25 000–49 999
1.44 (0.64, 3.23)
.37
50 000–99 999
2.30 (0.89, 5.98)
.09
100 000–499 999
2.13 (0.88, 5.12)
.09
‡ 500 000
3.05 (1.12, 8.35)
.03
Governance State Local
1.00 1.04 (0.23, 4.63)
.96
Shared
1.15 (0.20, 6.51)
.88
City
1.56 (0.69, 3.52)
.27
County
1.00
Multicity
6.60 (1.42, 30.78)
.02
Multicounty
0.73 (0.24, 2.22)
.57
0.62 (0.38, 1.00)
.05
Jurisdiction
Community health assessment Yes No
1.00
Investigation in public health Yes
1.35 (0.87, 2.10)
No
1.00
.18
Population-based prevention activities Yes
1.37 (0.66, 2.82)
No
1.00
.39
Continued
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
sample study to further identify relationships between LHD leader degree status and intent. However, if accreditation was considered an innovation in public health, the Diffusion of Innovation theory by Rogers27 could be useful to interpret this finding. In the Diffusion of Innovation age, financial lucidity and risk tolerance are used to explain innovativeness. Although our study evaluated age of LHD leaders, we did not have variables to assess financial lucidity and risk tolerance. There was no maximum likelihood method of estimating parameters when the response variable was discrete and correlated, and the distribution of the response variable was unspecified. As an alternative, we used general estimating equations that extended the concept of maximum likelihood estimation by imposing a covariance structure,24 and therefore, we sufficiently adjusted for the correlation of the responses between LHDs. Our study findings were robust and had minimum variance. However, this was a cross-sectional study, and thus, the causal relationship of estimating intent to seek accreditation could not be determined. The exclusion of other important PHAB domains that were not available in the NACCHO data set might be a study limitation. We used intent as a proxy for accreditation. Intent could only approximate the likelihood of obtaining accreditation. However, because of the limited availability of variables that measured LHDs actually pursuing accreditation in 2010, we were restricted to use intent as its proxy. In addition, intent in some published and unpublished earlier studies was shown to be a consistent predictor of the likelihood that one would (or would not) execute a behavior.28---30 The first accredited health departments were announced in March 2013, which was beyond the reference time for the intent to pursue accreditation in the first 2 years of national voluntary accreditation we used in this study. However, future studies could examine the association between intent to pursue accreditation and the attainment of accredited status. Restricting the study sample only to those who either agreed or strongly agreed or disagreed or strongly disagreed about their intent to seek voluntary accreditation in the next 2 years might have introduced some bias. However, based on population size (test of
Carman and Timsina | Peer Reviewed | Research and Practice | S357
RESEARCH AND PRACTICE
(e-mail: [email protected]). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted January 8, 2015.
TABLE 2—Continued Community health improvement plan Yes
1.03 (0.63, 1.69)
No
1.00
.9
Comprehensive strategic planning Yes
0.30 (0.12, 0.74)
.01
1.00 1.01 (0.95, 1.06)
.8
Yes
2.64 (1.59, 4.37)
< .001
No
1.00
No Regulation, inspection, or licensing activities QI activities
Distinction between funds Yes
0.85 (0.50, 1.47)
No
1.00
Current operating budget compared with previous FY Less Approximately same or greater
0.90 (0.53, 1.53)
.56
.7
1.00
Expected budget in the following FY Less
1.03 (0.57, 1.84)
Approximately same or greater
1.00
.93
Note. AOR = adjusted odds ratio; CI = confidence interval; FY = fiscal year; QI = quality improvement. AORs derived from multivariate generalized estimating equation adjusting for the autocorrelation between LHDs within each state.
mean by type of intent), and governance structure, jurisdiction type or reporting as region (v2 test) did not differ independently with intent categorized as agree or strongly agree, neither agreed nor disagreed (neutral), or disagree or strongly disagree. For ease of interpretation and comparability, we decided to drop the observation that had neutral responses in the analysis. Our results concurred with those found by Shah et al.31 in a study that grouped intent into 3 categories (strongly or disagree, natural, agree or strongly agree) and assessed the impact of PHAB prerequisites on the intention to pursue accreditation using a broader reference period than our study.
Conclusion There is no question that this is a pivotal time in public health. The changing economic environment, combined with the passage of the Affordable Care Act, represent major changes in the operating environment of LHDs. These external stressors, combined with new challenges, demand that LHDs pursue increased quality in the performance of population health services. Early supporters of national public health accreditation described it as “a platform for quality improvement.”3 The results of our
study supported this concept by identifying LHDs that, before accreditation, had not completed a CHA or strategic plan, but also showed intent. These LHDs will experience the benefits of QI through systematic strategic planning and connecting with community partners through CHAs and improvement planning as they complete prerequisites for accreditation. Based on our study findings, national public health accreditation might be the vehicle LHDs could use to improve operating environments, better manage resources, and reap rewards associated with meeting national industry standards. Moreover, future studies on accreditation should consider looking at the interaction effects of the prerequisites of accreditation and population size, resources, and the tenure of top executives of LHDs. j
About the Authors Angela L. Carman is with the University of Kentucky, College of Public Health, Lexington. Both authors are with the National Coordinating Center for Public Health Services and Systems Research and Practice-Based Research Networks, Lexington. Correspondence should be sent to Angela L. Carman, University of Kentucky, College of Public Health, National Coordinating Center for Public Health Services and Systems Research and Practice-Based Research Networks, 111 Washington Avenue, Room 105C, Lexington, KY 40536
S358 | Research and Practice | Peer Reviewed | Carman and Timsina
Contributors A. L. Carman conceptualized the research question, study design, analysis, literature review, content revision, and article preparation. L. Timsina conducted the data analysis and contributed to the study design, literature review, content revision, and article preparation.
Acknowledgments We would like to thank F. Douglas Scutchfield, MD, James W. Holsinger Jr, MD, PhD, David Fardo, PhD, Rachel Hogg, DrPH, and Margaret McGladrey, MA, for their expert advice and comments on this article.
Human Participant Protection The University of Kentucky Institutional Review Board committee determined that this study did not need review because the unit of analysis did not meet the definition of human subjects.
References 1. PHAB. Public Health Department Accreditation Background. 2011. Available at: www.phaboard.org. Accessed November 15, 2011. 2. Ingram RC, Bender K, Wilcox R, Kronstadt J. A consensus-based approach to national public health accreditation. J Public Health Manag Pract. 2014;20 (1):9---13. 3. Russo P. Accreditation of public health agencies: a means, not an end. J Public Health Manag Pract. 2007;13(4):329---331. 4. Bender K, Benjamin G, Fallon MM, Jarris PE, Libbey PM. Exploring accreditation: striving for a consensus model. J Public Health Manag Pract. 2007;13(4):334---336. 5. Centers for Disease Control and Prevention. National Public Health Performance Standards---Essential Services. 2013. Available at: http://www.cdc.gov/ nphpsp/essentialservices.html. Accessed November 14, 2014. 6. McLees AW, Thomas CW, Nawaz S, Young AC, Rider N, Davis M. Advances in public health accreditation readiness and quality improvement: evaluation findings from the national public health improvement initiative. J Public Health Manag Pract. 2014;20(1):29---35. 7. Beaudry ML, Bialek R, Moran JW Jr. Using quality improvement tools and methods throughout the accreditation lifecycle. J Public Health Manag Pract. 2014;20 (1):49---51. 8. Beitsch LM, Riley W, Bender K. Embedding quality improvement into accreditation: evolving from theory to practice. J Public Health Manag Pract. 2014;20(1):57---60. 9. Hamm MS. Quality Improvement Initiatives in Accreditation: Private Sector Examples and Key Lessons for Public Health. Princeton, NJ: Robert Wood Johnson Foundation; 2007. 10. Carman AL, Timsina L, Scutchfield FD. Quality improvement activities of local health departments during the 2008---2010 economic recession. Am J Prev Med. 2014;46(2):171---174. 11. Willard R, Shah GH, Leep CJ, Leighton K. Impact of the 2008-2010 economic recession on local health
American Journal of Public Health | Supplement 2, 2015, Vol 105, No. S2
RESEARCH AND PRACTICE
departments. J Public Health Manag Pract. 2012;18 (2):106---114. 12. National Association of County and City Health Officials. Local Health Department Job Losses and Program Cuts: Findings From January/February 2010 Survey. Washington, DC: National Association of County and City Health Officials; 2010.
30. Speaking of Health. Assessing Health Communication Strategies for Diverse Populations. Washington, DC: The National Academies Press; 2002. 31. Shah GH, Beatty K, Leep C. Do PHAB accreditation prerequisites predict local health departments’ intentions to seek voluntary national accreditation? Front Public Health Serv Syst Res. 2013;2(3):article 4.
13. Honore PA, Scott W. Priority Areas for Improvement of Quality in Public Health. Washington, DC: Department of Health and Human Services; 2010. 14. Riley WJ, Parsons HM, Duffy GL, Moran JW, Henry B. Realizing transformational change through quality improvement in public health. J Public Health Manag Pract. 2010;16(1):72---78. 15. Boe DT, Riley W, Parson H. Improving service delivery in a county health department WIC clinic: an application of statistical process control techniques. Am J Public Health. 2009;99(9):1619---1625. 16. Luo H, Sotnikov S, McLees A, Stokes S. Factors driving the adoption of quality improvement initiatives in local health departments: results from the 2010 profile study. J Public Health Manag Pract. 2015;21(2):176---185. 17. Turnock B. Public Health: What It Is and How It Works. Burlington, MA: Jones and Bartlett; 2011. 18. Public Health Accreditation Board. Acronyms and glossary of terms. 2012. Available at: http://www.phaboard. org/wp-content/uploads/PHAB-Acronyms-and-Glossary-ofTerms-Version-1.0.pdf. Accessed March 12, 2013. 19. Ginter P, Duncan WJ, Swayne L. Strategic Management of Health Care Organizations. 7th ed. San Francisco, CA: Jossey-Bass; 2013. 20. Shah GH, Leep C, Ye J, Sellers K, Liss-Levinson R, Williams K. Public health agencies’ level of engagement in and perceived barriers to PHAB national voluntary accreditation. J Public Health Manag Pract. 2015;21 (2):107---115. 21. Schermerhorn J. Management. 10th ed. Hoboken, NJ: John Wiley & Sons; 2010. 22. Downey LH, Anyaegbunam C, Scutchfield FD. Dialogue to deliberation: expanding the Empowerment education model. Am J Health Behav. 2009;33(1):26---36. 23. National Association of County and City Health Officials. 2010 National Profile of Local Health Departments. Washington, DC: National Association of County and City Health Officials; 2011. 24. Little R, Rubin D. 2nd ed. Statistical Analysis With Missing Data. Hoboken, NJ: Wiley-Interscience; 2002. 25. Institute for Digital Research and Education. Statistical Computing Seminars: Multiple Imputation in SAS, Part 1. 2014. Available at: http://www.ats.ucla.edu/stat/ sas/seminars/missing_data/part1.htm. Accessed February 25, 2014. 26. Landerman LR, Land KC, Pieper CF. An empirical evaluation of the predictive mean matching method for imputing missing values. Sociol Methods Res. 1997;26(1):3-- 33. 27. Rogers E. Diffusion of Innovations. 5th ed. New York: Free Press; 2003. 28. Sheppard BH, Hartwick J, Warshaw PR. The theory of reasoned action: a meta-analysis of past research with recommendations for modifications and future research. J Consum Res. 1988;15(3):325---343. 29. Van den Putte B. 20 Years of the Theory of Reasoned Action of Fishbein and Aizen: A Meta-analysis. Amsterdam: University of Amsterdam; 1991.
Supplement 2, 2015, Vol 105, No. S2 | American Journal of Public Health
Carman and Timsina | Peer Reviewed | Research and Practice | S359
