J. DRUG EDUCATION, Vol. 43(2) 141-153, 2013
ASSESS THE IMPACT OF AN ONLINE TOBACCO PREVENTION TRAINING PROGRAM ON TEACHERS AND THEIR STUDENTS*
W. WILLIAM CHEN, PH.D., MA, MCHES University of Florida, Gainesville JIUNN-JYE SHEU, PH.D., MSPH, MCHES University of Toledo, Ohio CHUNG-BANG WENG, PH.D., MA, MS National Taipei University, Taiwan
ABSTRACT
School-based tobacco prevention programs have been proven effective in reducing tobacco use. This evaluation aimed to assess the impact of an online tobacco prevention teacher training program on teachers and their students in Florida schools. A total of 344 teachers, including 72 K-3 grade teachers, 44 4th-5th grade teachers, and 228 6th-12th grade teachers completed the online training program and 323 (93.9%) were followed up to assess impacts on their 6,490 students. Results suggested that the online tobacco prevention education program for teachers was effective with high satisfaction and the impact on students was significant in improving knowledge and attitude about tobacco use and in increasing the proportion of 6-12th grade students who decided not to use tobacco. The evaluation study recommended the online education program be continued and expanded in the future.
*This study was supported by a grant from the Office of Healthy Schools, Florida Department of Education. 141 Ó 2013, Baywood Publishing Co., Inc. doi: http://dx.doi.org/10.2190/DE.43.2.c http://baywood.com
142 / CHEN, SHEU AND WENG
INTRODUCTION According to the World Health Organization (2005), tobacco use ranks among the top three modifiable risk factors for chronic disease and premature death and continues to pose an enormous public health threat worldwide. In the United States, nearly half century after the Surgeon General’s first report on smoking and health (Public Health Service, 1964), tobacco use prevention continues to be a key public health priority. A significant drop of tobacco use over the years has been witnessed, but the trend of historic near 30-year smoking decline among the nation’s youth and adolescents appears to have stopped (Johnston, O’Malley, Bachman, & Schulenberg, 2008). According to the Youth Risk Behavior Surveillance System (YRBSS) by the Division of Adolescent and School Health, Centers for Disease Control and Prevention (CDC, n.d.), current tobacco use (at least one day in the past 30 days) prevalence among Florida students in Grades 9 through 12 was nearly 21.7% in 2009, declining from 26.6% in 2001, 22.7% in 2003, and 21.6% in 2005. However, the prevalence rate was reversed and was found ascending from 20.2% in 2007 at an alarmingly rate. The prevalence rate of 21.7% in 2009 indicated more than one in every five high school students currently uses tobacco. In addition, among students who currently smoke cigarettes, the percentage who had tried to quit smoking cigarettes in Florida is significantly (p = .01) lower than the national average. Additionally, survey of high school and middle school students by the Florida Department of Health (2009) also found the prevalence rates of current tobacco use at 22.4% and 9% respectively. These consistent data support the need for tobacco prevention education, particularly in schools. Schools can provide a unique environment to conduct tobacco prevention education, primarily because they capture the priority populations at highest risk for tobacco acquisition. Most of the young people can be reached through schools. Thus, many randomized trials had been conducted to evaluate the effectiveness of school-based tobacco use prevention programs in schools. For students to receive the benefits of behavior change for disease prevention, programs must be implemented effectively. Schools can play a significant role in moving the objectives of tobacco free environment forward. Over the past years, school-based tobacco prevention programs had been found effective. Critical reviews on youth smoking prevention programs indicated that school-based tobacco prevention interventions are effective in improving knowledge level, reducing smoking initiation and continuation in the short term (Baska, Straka, Baskova, & Madar, 2004; Dobbins, Decorby, Manske, & Goldblatt, 2008; Fray, 2009; Muller-Riemenschneider, Bockelbrink, Reinhold, Rasch, Greiner, & Willich, 2008; Peterson, Kealey, Mann, Marek, & Sarason, 2000; Skara & Sussman, 2003; Wiehe, Garrison, Christakis, Ebel, & Rivara, 2005). The researchers also found the intervention effectiveness has not been
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
143
maintained for more than 3 years post-intervention and suggested continuous implementation of school-based tobacco prevention interventions (Dobbins et al., 2008). In Florida, only 8.6% of high school students and 16.1% of middle school students received comprehensive tobacco use prevention education and 50.4% of high school students and 72.8% of middle school students received some, but not comprehensive, form of tobacco prevention education (Florida Department of Health, 2009). In order to sustain these effects, schools will need to do more, particularly in teacher training programs for tobacco prevention and control. To prepare teachers for tobacco prevention education, continuing education via the Internet had shown some success. The effectiveness of online prevention programs in schools has been reported in the literature. Literature review indicated that teachers’ online professional development not only contributed significantly to teaching effectiveness, but also could be implemented successfully in the classroom (Atkinson & O’Connor, 2007). In addition, online training provided a new world of opportunity in professional development (Carter, 2004). These experiences could be a model for online teacher training for tobacco prevention in the state. The World Health Organization (WHO, 1999) recommended interactive health information which involves active promotion, and uses tailored messages for individual adolescents to reach large numbers. Online programs provide the best opportunity to achieve such quality and audiences including behavior changes across various health issues (Christensen, Griffiths, & Jorm, 2004; Evers, Prochaska, Prochaska, Driskell, Cummins, & Velicer, 2003; Ritterband, GonderFrederick, Cox, Clifton, West, & Borowitz, 2003). Tobacco control is among the most widely studied and successful online programs reported (Walters, Wright, & Shegog, 2006). For example, ASPIRE (Prokhorov, Kelder, Shegog, Murray, Peters, Agurcia-Parker, et al., 2008), Consider This (Buller, Borland, Woodall, Hall, Hines, Burris-Woodall, et al., 2008), Smoking Zine (Norman, Maley, Li, & Skinner, 2008), Headbutt (Shegog, McAlister, Hu, Ford, Meshack, & Peters, 2005), and Preventing Teen Smoking with Virtual Reality (Nemire, Beil, & Swan, 1999) all reported effectiveness in reducing and preventing tobacco use. The Palm Beach School District has been a recipient of the Tobacco Prevention and Intervention Teacher Training (TPITT) Project for several years and successfully used online training for teacher training in tobacco prevention with positive impact evaluation evidence (Lacey, 2008). In the 2011 grant year, the Palm Beach School District had been contracted to provide the Statewide Online Tobacco Prevention and Intervention Teacher Training (SOTPTT) using TrainU, an online training program designed for teachers’ continuing education. To improve and evaluate the effectiveness of SOTPTT, an evaluation was conducted to assess the outcomes of the Program both in the trained teachers and their students receiving the tobacco prevention education. This article aimed to provide a summative evaluation of the Statewide Online Tobacco Prevention and Intervention Teacher Training Project (SOTPTT). The
144 / CHEN, SHEU AND WENG
outcome evaluation included assessment of learning outcomes of teachers who received training, impact on students who were taught by the trained teachers, and program satisfaction of teachers who participated in the project. METHODS Procedure The literature review and theoretical framework are fundamental components in this evaluation. A Logic Model (Cato, Chen, & Corbett-Perez, 1998; Chen & Cato, 1999; W. K. Kellogg Foundation, 2004) was developed to analyze the intervention and guide the evaluation design. A two-group quasi-experimental design was implemented. To determine the Program’s outcomes, the data were collected from both the trained teachers and the students taught by the trained teachers. To ensure the quality of data, the research team reviewed and reassessed the instruments used previously in the Palm Beach School District’s TPITT’s evaluation project. In addition, instruments published in the literature were reviewed and selected items were integrated into the final instruments. As part of the instrument validation process, an expert panel was invited to examine the face validity and content validity. The internal consistency reliability was also established. As required by the SOTPTT, the participating teachers had to complete online pre- and post-assessments before and after online training on TrainU; teach six tobacco prevention lessons; distribute and collect the students’ pre- and postassessments before and after the six lessons were taught; and complete satisfaction survey, which included the coverage, length, difficulty, administration, and quality of the program components and the curriculum and materials provided on TrainU after the completion of the Program. The participating teachers who completed the aforementioned requirements received the continuing education credits approved by their school districts. The teachers who registered but did not complete the training were surveyed to understand their reasons of lapse and future motivators. Participants All teachers from 67 school districts in the state and laboratory schools were eligible to participate. A total of 344 teachers, including 72 K-3rd grade teachers, 44 4-5th grade teachers, and 228 6-12th grade teachers completed the online training program and were sent students’ assessment packages to assess their impacts on students. Among them, 323 teachers (93.9%) had completed at least six student lessons, collected student impact surveys, and returned the assessment package to the evaluation team. As a result, a total of 6,490 pairs of valid student pre-/post-assessments were received and entered into analysis to examine the impact of online tobacco prevention teacher training on their students.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
145
Data Collection and Analysis The evaluation protocol was submitted to and approved by the Institutional Review Board (IRB) of the University of Florida. The data obtained are kept confidential. The identifiers were only used to match participants’ pre- and post-assessment responses. After data analysis, the identifiers were deleted from data file to ensure confidentiality. No individual response is disclosed or reported elsewhere. All paper surveys were delivered to the confidential recycling unit for the proper protection of privacy. The students who were taught by the trained teachers completed a pre- and post-assessment to measure the impact teachers’ online training has on them as regard to their changes in knowledge about tobacco hazards, media influences over tobacco use, and refusal skills. The trained teachers requested the pre-/ post-assessments at about the mid-point of their online training and received sufficient number of paper pre-/post-assessments by postal mail. Specific instructions for the students to complete the surveys were included with the pre-/post-assessments. The trained teachers conducted students’ pre-assessment prior to teaching the lessons and collected the post-assessment immediately after the completion of the lessons. Postage paid return envelopes were provided for the teachers to send the pre-/post-assessments back to the evaluation team. All grades had three same areas in the pre- and post-assessment. They are the knowledge about tobacco hazards, media influences over tobacco use, and refusal skills. In the 4-5th and 6-12th grade surveys, two additional areas (outcome evaluation and overall assessment) were measured. In the area of outcome expectations, students were asked if they expect smoking to have positive outcomes, such as help to relax, stay thin, deal with problem or stress, and have something to do when bored. In the area of overall assessment, students were asked their decisions if they would enjoy smoking or choose not to smoke in the future. The evaluation data were analyzed for instrument validity and reliability. The data from the surveys were entered into Microsoft Excel, Access, and IBM SPSS for data management and statistical analysis. The students’ t-tests, paired t-tests, and one-way analysis of variance (ANOVA) were conducted in this analysis. The Cronbach’s alpha for teachers’ pre-/post-assessment (13 items) was .91, teachers’ satisfaction survey (8 items) was .85, K-3rd grade students’ pre-/postassessment (5 items) was .69, 4-5th grade students’ pre-/post-assessment (12 items) was .72, and 6-12th grade students’ pre-/post-assessment (15 items) was .82. RESULTS The results of this evaluation include teachers’ training outcomes and teachers’ impact on student’s learning.
146 / CHEN, SHEU AND WENG
Teacher Training Outcomes Table 1 shows the teachers’ training outcomes. The pre- and post-assessment scores from teachers who completed training and classroom teaching were compared to examine the training effects. Among the teachers who completed all requirements, 274 pairs of pre- and post-assessment data had no missing values and were matched and analyzed. Results showed teachers’ scores improved from 37.58 (SD = 6.49) to 48.87 (SD = 4.15) (maximum score 52). Overall, the participating teachers’ knowledge about tobacco prevention had 30% improvement after participation in the program with high statistical significance (t273 = 26.36, p < .001). This result showed very significant improvement in teachers’ scores after training both in total items and in the separate five categories assessed, including the knowledge about tobacco hazards, the knowledge about risk and resiliency factors related to tobacco use, the knowledge about current trends in tobacco use, marketing, and products, the knowledge about tobacco prevention resources, and the familiarity of tobacco prevention curriculum. Among the five categories assessed, the level of familiarity with tobacco prevention curriculum had the largest increase (50%) after training. In addition, the trained teachers stated their level of knowledge about tobacco prevention resources increased by 37%. Teachers’ satisfaction was measured by eight items in Likert scale (strongly disagree, disagree, agree, strongly agree). The results from 273 completed surveys showed the mean scores of 30.43 (maximum score 32) and standard deviation of 2.30. The teachers who participated in the training program showed high satisfaction to the program. In the area of teachers’ satisfaction on training, nearly all teachers agreed or strongly agreed that they are comfortable using computers (98.2%). The majority of the teachers agreed or strongly agreed the program was informative (97.7%) and user friendly (98.9). In addition, the teachers’ opinions showed the assignments were reasonable and appropriate (98.2%), and the program increased their interest (99.6%), made them feel confident (99.6%), and felt it was a positive experience (99.6%). The trained teachers were very satisfied with the quality of the training program with the mean scores of 18.99 (SD = 1.52, maximum score 20). The teachers also reported a high level of comfort in using computers (mean = 7.57 and SD = 0.72 based on the possible scores of 8). In addition, after the training, their confidence in teaching tobacco prevention lessons was very high (3.88 on the scale of 4). Impact on Students’ Learning Table 2 shows students’ pre- and post-assessment results by the grade levels. On average, the students had about 24% improvement in their knowledge about tobacco. In all grade levels, the post-assessment scores were found higher than the pre-assessment scores with very high statistical significance. The K-3rd
9.06 ± 1.66 3.16 ± 0.67 10.61 ± 2.42 5.00 ± 1.45
2 3 4 5 39.24 ± 10.00
9.75 ± 1.52
1
All
37.58 ± 6.49
Pre-assessment mean ± SD
All
Category
N/A
7.49 ± 0.82
14.54 ± 1.64
3.92 ± 0.35
11.27 ± 1.08
11.65 ± 0.99
48.87 ± 4.15
Post-assessment mean ± SD
Note: Categories 1. knowledge about tobacco hazards 2. knowledge about risk and resiliency factors related to tobacco use 3. knowledge about current trends in tobacco use, marketing, and products 4. knowledge about tobacco prevention resources 5. familiarity of tobacco prevention curriculum
Did not complete (129)
Completed all requirements (274)
Status
N/A
+49.8%
+37.0%
+24.1%
+24.4%
+19.5%
+30.0%
% change
Table 1. Teachers’ Training Outcomes
N/A
25.60
24.87
16.58
20.24
18.47
26.36
Paired t
N/A
< .001
< .001
< .001
< .001
< .001
< .001
Significance ONLINE TOBACCO PREVENTION TEACHER TRAINING / 147
1,266
K-3
• • • • •
6-12
• • • • •
4-5
Knowledge about tobacco hazards Knowledge about refusal skills Knowledge about media literacy Outcome expectations Overall assessment
4,339
Knowledge about tobacco hazards Knowledge about refusal skills Knowledge about media literacy Outcome expectations Overall assessment
885
• Knowledge about tobacco hazards • Knowledge about refusal skills • Knowledge about media literacy
6,490
Number of students
All
Grade
55.10 ± 24.18 87.96 ± 32.54 54.02 ± 39.42 81.61 ± 31.43 83.72 ± 32.65
67.56 ± 19.76
78.82 ± 22.61 91.48 ± 27.94 46.47 ± 35.79 84.96 ± 24.77 96.27 ± 18.97
76.92 ± 17.06
68.23 ± 30.16 73.04 ± 44.39 53.70 ± 49.88
65.69 ± 27.22
68.47 ± 21.37
Pre-assessment % mean ± SD
75.39 ± 23.34 93.38 ± 24.87 79.43 ± 32.49 87.01 ± 27.75 88.40 ± 27.66
81.41 ± 18.59
95.22 ± 12.25 97.16 ± 16.62 75.72 ± 32.01 91.64 ± 19.96 97.78 ± 14.73
91.07 ± 12.49
94.71 ± 15.80 95.72 ± 20.24 89.28 ± 30.95
93.18 ± 15.71
85.02 ± 18.09
Post-assessment % mean ± SD
36.8% 6.2% 47.0% 6.6% 5.6%
20.4%
20.8% 6.2% 62.9% 8.2% 1.6%
18.5%
38.5% 30.1% 64.8%
41.9%
24.1%
% increase
44.64 9.93 38.51 10.87 9.61
41.70
20.65 5.55 20.82 7.16 1.99
23.37
27.84 17.31 22.51
32.09
55.64
Paired t
Table 2. Pre- and Post-Assessment of Students’ Knowledge about Tobacco Hazards by Grade Levels
< .001 < .001 < .001 < .001 < .001
< .001
< .001 < .001 < .001 < .001 < .05
< .001
< .001 < .001 < .001
< .001
< .001
Significance
148 / CHEN, SHEU AND WENG
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
149
graders appeared to have the highest improvement (41.9%), followed by 6-12th (20.4%), and 4-5th graders (18.5%). Students’ improvement data were analyzed using the trained teachers as the analysis unit. The students’ mean scores after the six lessons taught by each trained teacher increased by almost 17%. For individual grade groups, students in K-3rd, 4-5th, and 6-12th grades showed improvement of 27.67%, 15.71%, and 13.71% respectively. Results of further statistical analysis by one-way ANOVA indicated that teachers in K-3rd grades had the most significant impact on their students. Their mean percent improvement was found significantly better than the 4-5th and 6-12th grades separately (F = 28.75, df = 2, 309, p < .001, Tukey HSD, Scheffe, LSD: p < .001). To further examine teachers’ impact on students’ learning, four levels in the students’ pre- and post-assessment score changes were created: Increased significantly, Increased, Decreased, and Decreased significantly. To be qualified as increased or decreased significantly, the students’ post-assessment mean scores must be increased or decreased, respectively, by p < .05 significance level from their pre-assessment mean scores. Overall, the vast majority of the students taught by trained teachers had improved (94.6%) their knowledge about tobacco. Among K-3rd, 4-5th, and 6-12th graders, 97%, 97.4%, and 93.2% of the students showed improved outcomes respectively. The majority of the teachers who completed the program had their students’ post-assessment mean scores significantly better than their pre-assessment. For example, approximately 90% of the 4th and 5th grade students had significantly better knowledge on tobacco prevention. In the meantime, a total of 16 (5.1%) classes showed decreased knowledge about tobacco prevention. Among them, the majority (14 classes) was in the 6-12 grades. Results analyzed by grade category showed K-3rd grade students had 64.8%, 38.5%, and 30.1% improvement in the knowledge about media literacy, knowledge about tobacco hazards, and knowledge about refusal skills respectively after being taught by the trained teachers. The program had the largest impact on K-3rd grade students in their knowledge in media literacy, followed by the knowledge about tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001). In addition, 4-5th grade students had 62.9%, 20.8%, 8.2%, 6.2%, and 1.6% improvement in the knowledge about media literacy, knowledge about tobacco hazards, outcome expectations, knowledge about refusal skills, and overall assessment respectively after being taught by the trained teachers. The program had the largest impact on 4-5th grade students in their knowledge about media literacy, followed by the knowledge on tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001) except the student’s overall assessment of tobacco use (p < .05). Furthermore, 6-12th grade students had 47.0%, 36.8%, 6.6%, 5.6%, and 6.2% improvement in the knowledge about media literacy, knowledge about tobacco
150 / CHEN, SHEU AND WENG
hazards, outcome expectations, overall assessment, and knowledge about refusal skills, respectively, after being taught by the trained teachers. The program had the largest impact on 6-12th grade students in their knowledge about media literacy, followed by the knowledge on tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001). The majority of 6-12th graders (3,469; 79.9%) chose not to smoke in pre-assessment and kept their decision in the post-assessment. The impact on students’ projected decision not to smoke increased by 31% after six lessons had been taught among 6-12th graders. DISCUSSION Online tobacco training programs appeared successful in training teachers, but its impacts on students have not been assessed extensively. This evaluation study not only assessed the impact of online training on teachers but also examined its impact on students who were taught by the trained teachers. Specifically, this study examined among participating teachers what their changes in knowledge about tobacco prevention education are before and after the program, what are their satisfaction with the quality of the online training program and their confidence in teaching tobacco prevention lessons. In addition, the impacts on the students who were taught by the trained teachers, in the areas of knowledge about tobacco hazards, medias influences over tobacco use, refusal skills, attitudes toward tobacco use, and projected decision not to use tobacco, were also examined. Results of the evaluation revealed that the online training program was indeed effective not only in improving teachers’ knowledge about tobacco prevention education, but also their satisfaction with the online training program. Most significantly, the training program also had a successful impact on students. Students’ knowledge about tobacco hazards, the influence of media on tobacco use, and refusal skills were improved in all grade levels. For students in grades 4-5th and 6-12th, the program also showed improvement in their attitude toward tobacco use and their projected decision not to use tobacco. To examine if the teachers who completed the training had better knowledge in tobacco prevention prior to the training than the teachers who did not complete the program, the pre-assessment scores of the teachers who completed the program were compared with the pre-assessment scores of the teachers who did not complete the program. The pre-assessment scores between the teachers who completed the program and the teachers who did not complete the program were found to have no statistical significance. The overall scores for the trained teachers before online training was 37.58 (out of possible 52 points from 13 questions) compared with 39.24 for the teachers who signed up for the training program but did not complete the program. This result showed the participating teachers were not different from the teachers who did not complete the program and may imply that all teachers, if they completed the program, could have as significant knowledge improvement in tobacco prevention as the participating teachers.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
151
One of the major challenges for the online tobacco prevention education for school teachers is lack of incentives to attract teachers to spend a significant portion of their free time to attend the lessons. Financial awards were not permitted to encourage participation by state regulation. As a result, continuing education credits for teacher re-certification were requested and approved by participating school districts. The continuing education credits provided to the trained teachers brought a major force of motivation to participate in this training program. A total of 60 continuing education credits could be awarded when teachers completed training and all requirements including data collection. Another driving force is that the training being delivered online and the teaching requirements of tobacco prevention lessons could be conveniently completed in the classes the teachers taught. Such design saved teachers’ time and costs and increased the accessibility and availability of intervention materials and consultation. The education authorities are suggested to offer similar continuing education credits for health related trainings in the future. Even with such an incentive, a significant number of teachers who had signed up for the online program either had never accessed the program or did not complete the training. The major reasons provided included lack of times, unexpected family emergency, or personal health problems. The high incompletion rate would be a major barrier for future programs, and further incentives need to be considered for improvement. Interpretation of the results needs to be cautious and should not be generalized as this was a quasi-experimental evaluation design based on a convenience sample and no control group was included in the evaluation. While every precaution was taken to make sure the validity of the data, teachers’ data were collected online and students’ data were collected by trained teachers. Additional limitations of this study include the long-term effects on knowledge, attitudes, media literacy, refusal skills, and actual tobacco use not being assessed; a structured assessment on intention to use tobacco, smoking susceptibility, or past-month smoking rates not being included as outcome indicators; and the attribution of short-term knowledge change to stay smoke-free or quit smoking. Further longitudinal surveillance studies are recommended. Online training programs for tobacco prevention have not been used widely in the past and could be more strategically adopted if they can be proven to be effective for training teachers. This evaluation study provided preliminary data to support the use of online program for training teachers to conduct tobacco prevention. The evaluation results indicated that participating teachers not only improved their knowledge about tobacco and tobacco prevention but also their confidence in conducting prevention education for their students after training. Additionally, the evaluation study was able to provide preliminary data supporting the impacts of the program on students who were taught by their trained teachers. It is recommended that this online training program be continued and expanded in the state and results evaluated continuously in the future.
152 / CHEN, SHEU AND WENG
REFERENCES Atkinson, T. S., & O’Connor, K. A. (2007). Establishing professional development partnerships online: Reaching out to veteran teachers. TechTrends: Linking Research and Practice to Improve Learning, 51(3), 21-29. Baska, T., Straka, S., Baskova, M., & Madar, R. (2004). Effectiveness of school programs in tobacco control. Central European Journal of Public Health, 12(4), 184-186. Buller, D. B., Borland, R., Woodall, W. G., Hall, J. R., Hines, J. M., Burris-Woodall, P., et al. (2008). Randomized trials on Consider This, a tailored, internet-delivered smoking prevention program for adolescents. Health Education & Behavior, 35(2), 260-281. Carter, K. (2004). Online training: What’s really working? What does today’s successful online professional development look like? Technology & Learning, 24(10), 32. Cato, B., Chen, W. W., & Corbett-Perez, S. (1998). Logic Model—A tool for planning and evaluating health and recreation prevention projects. Journal of Physical Education, Recreation, and Dance, 69(8), 57-61. Centers for Disease Control and Prevention (CDC). (n.d.). Youth online: Comprehensive results. Retrieved from http://apps.nccd.cdc.gov/youthonline/App/QuestionsOr Locations.aspx?CategoryId=2 Chen, W. W., & Cato, B. (1999). Using a Logic Model to plan and evaluate community intervention program—A case study. International Quarterly of Community Health Education, 18(4), 451-460. Christensen, H., Griffiths, K. M., & Jorm, A. F. (2004). Delivering interventions for depression by using the internet: Randomised controlled trial. British Medical Journal, 328(7434), 265. Dobbins, M., Decorby, K., Manske, S., & Goldblatt, E. (2008). Effective practices for school-based tobacco use prevention. Preventive Medicine, 46(4), 289-297. Evers, K. E., Prochaska, J. M., Prochaska, J. O., Driskell, M., Cummins, C. O., & Velicer, W. F. (2003). Strengths and weaknesses of health behavior change programs of the Internet. Journal of Health Psychology, 8(1), 63-70. Florida Department of Health. (2009). 2008 Florida Youth Tobacco Survey county data book. Retrieved from http://www.hpcnef.org/files/health-needs-assesments/2008 DataBook_web.pdf Fray, B. R. (2009). School-based smoking prevention programs with the promise of long-term effects. Tobacco Induced Diseases, 5, 6. Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2008). Monitoring the Future national results on adolescent drug use: Overview of key findings, 2007. Bethesda, MD: National Institute on Drug Abuse. Lacey, C. H. (2008). Evaluation of the Tobacco Prevention and Intervention Teacher Training Project. Palm Beach, FL: School District of Palm Beach County. Muller-Riemenschneider, F., Bockelbrink, A., Reinhold, T., Rasch, A., Greiner, W., & Willich, S. N. (2008). Long-term effectiveness of behavioural interventions to prevent smoking among children and youth. Tobacco Control, 17(5), 301-312. Nemire, K., Beil, J. A., & Swan, R. W. (1999). Preventing teen smoking with virtual reality. CyberPsychology & Behavior, 2(1), 35-47. Norman, C. D., Maley, O., Li, X., & Skinner, H. A. (2008). Using the Internet to assist smoking prevention and cessation in schools: A randomized, controlled trial. Health Psychology, 27(6), 799-810.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
153
Peterson, A. V., Jr., Kealey, K. A., Mann, S. L., Marek, P. M., & Sarason, I. G. (2000). Hutchinson Smoking Prevention Project: Long-term randomized trial in school-based tobacco use prevention-results on smoking. Journal of the National Cancer Institute, 92(24), 1979-1991. Prokhorov, A. V., Kelder, S. H., Shegog, R., Murray, N., Peters, R., Jr., Agurcia-Parker, et al. (2008). Impact of A Smoking Prevention Interactive Experience (ASPIRE), an interactive, multimedia smoking prevention and cessation curriculum for culturally diverse high-school students. Nicotine and Tobacco Research, 10(9), 1477-1485. Public Health Service. (1964). Smoking and health: Report of the Advisory Committee to the Surgeon General of the Public Health Service. Public Health Service Publication No. 1103. U.S. Department of Health, Education, and Welfare. Ritterband, L. M., Gonder-Frederick, L. A., Cox, D. J., Clifton, A. D., West, R. W., & Borowitz, S. M. (2003). Internet interventions: In review, in use, and into the future. Professional Psychology, 34(5), 527-534. Shegog, R., McAlister, A. L., Hu, S., Ford, K. C., Meshack, A. F., & Peters, R. J. (2005). Use of interactive health communication to affect smoking intentions in middle school students: A pilot test of the “Headbutt” risk assessment program. American Journal of Health Promotion, 19(5), 334-338. Skara, S., & Sussman, S. (2003). A review of 25 long-term adolescent tobacco and other drug use prevention program evaluations. Preventive Medicine, 37(5), 451-474. W. K. Kellogg Foundation. (2004). Using Logic Models to bring together planning, evaluation, and action: Logic Model development guide. Battle Creek, MI: W. K. Kellogg Foundation. Walters, S. T., Wright, J. A., & Shegog, R. (2006). A review of computer and Internetbased interventions for smoking behavior. Addictive Behaviors, 31(2), 264-277. Wiehe, S. E., Garrison, M. M., Christakis, D. A., Ebel, B. E., & Rivara, F. P. (2005). A systematic review of school-based smoking prevention trials with long-term follow-up. Journal of Adolescent Health, 36(3), 162-169. World Health Organization (WHO). (1999). Programming for adolescent health and development: Report of a WHO/UNFPA/UNICEF study group on programming for adolescent health. Report 886. Saillon, Switzerland: World Health Organization. World Health Organization (WHO). (2005). Preventing chronic diseases: A vital investment. Geneva, Switzerland: World Health Organization. Retrieved from http://www. who.int/chp/chronic_disease_report/en/
Direct reprint requests to: W. William Chen, Ph.D., MCHES Professor, Department of Health Education and Behavior FLG 19, University of Florida Gainesville, FL 32611-8210 e-mail: [email protected]
Copyright of Journal of Drug Education is the property of Baywood Publishing Company, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
ASSESS THE IMPACT OF AN ONLINE TOBACCO PREVENTION TRAINING PROGRAM ON TEACHERS AND THEIR STUDENTS*
W. WILLIAM CHEN, PH.D., MA, MCHES University of Florida, Gainesville JIUNN-JYE SHEU, PH.D., MSPH, MCHES University of Toledo, Ohio CHUNG-BANG WENG, PH.D., MA, MS National Taipei University, Taiwan
ABSTRACT
School-based tobacco prevention programs have been proven effective in reducing tobacco use. This evaluation aimed to assess the impact of an online tobacco prevention teacher training program on teachers and their students in Florida schools. A total of 344 teachers, including 72 K-3 grade teachers, 44 4th-5th grade teachers, and 228 6th-12th grade teachers completed the online training program and 323 (93.9%) were followed up to assess impacts on their 6,490 students. Results suggested that the online tobacco prevention education program for teachers was effective with high satisfaction and the impact on students was significant in improving knowledge and attitude about tobacco use and in increasing the proportion of 6-12th grade students who decided not to use tobacco. The evaluation study recommended the online education program be continued and expanded in the future.
*This study was supported by a grant from the Office of Healthy Schools, Florida Department of Education. 141 Ó 2013, Baywood Publishing Co., Inc. doi: http://dx.doi.org/10.2190/DE.43.2.c http://baywood.com
142 / CHEN, SHEU AND WENG
INTRODUCTION According to the World Health Organization (2005), tobacco use ranks among the top three modifiable risk factors for chronic disease and premature death and continues to pose an enormous public health threat worldwide. In the United States, nearly half century after the Surgeon General’s first report on smoking and health (Public Health Service, 1964), tobacco use prevention continues to be a key public health priority. A significant drop of tobacco use over the years has been witnessed, but the trend of historic near 30-year smoking decline among the nation’s youth and adolescents appears to have stopped (Johnston, O’Malley, Bachman, & Schulenberg, 2008). According to the Youth Risk Behavior Surveillance System (YRBSS) by the Division of Adolescent and School Health, Centers for Disease Control and Prevention (CDC, n.d.), current tobacco use (at least one day in the past 30 days) prevalence among Florida students in Grades 9 through 12 was nearly 21.7% in 2009, declining from 26.6% in 2001, 22.7% in 2003, and 21.6% in 2005. However, the prevalence rate was reversed and was found ascending from 20.2% in 2007 at an alarmingly rate. The prevalence rate of 21.7% in 2009 indicated more than one in every five high school students currently uses tobacco. In addition, among students who currently smoke cigarettes, the percentage who had tried to quit smoking cigarettes in Florida is significantly (p = .01) lower than the national average. Additionally, survey of high school and middle school students by the Florida Department of Health (2009) also found the prevalence rates of current tobacco use at 22.4% and 9% respectively. These consistent data support the need for tobacco prevention education, particularly in schools. Schools can provide a unique environment to conduct tobacco prevention education, primarily because they capture the priority populations at highest risk for tobacco acquisition. Most of the young people can be reached through schools. Thus, many randomized trials had been conducted to evaluate the effectiveness of school-based tobacco use prevention programs in schools. For students to receive the benefits of behavior change for disease prevention, programs must be implemented effectively. Schools can play a significant role in moving the objectives of tobacco free environment forward. Over the past years, school-based tobacco prevention programs had been found effective. Critical reviews on youth smoking prevention programs indicated that school-based tobacco prevention interventions are effective in improving knowledge level, reducing smoking initiation and continuation in the short term (Baska, Straka, Baskova, & Madar, 2004; Dobbins, Decorby, Manske, & Goldblatt, 2008; Fray, 2009; Muller-Riemenschneider, Bockelbrink, Reinhold, Rasch, Greiner, & Willich, 2008; Peterson, Kealey, Mann, Marek, & Sarason, 2000; Skara & Sussman, 2003; Wiehe, Garrison, Christakis, Ebel, & Rivara, 2005). The researchers also found the intervention effectiveness has not been
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
143
maintained for more than 3 years post-intervention and suggested continuous implementation of school-based tobacco prevention interventions (Dobbins et al., 2008). In Florida, only 8.6% of high school students and 16.1% of middle school students received comprehensive tobacco use prevention education and 50.4% of high school students and 72.8% of middle school students received some, but not comprehensive, form of tobacco prevention education (Florida Department of Health, 2009). In order to sustain these effects, schools will need to do more, particularly in teacher training programs for tobacco prevention and control. To prepare teachers for tobacco prevention education, continuing education via the Internet had shown some success. The effectiveness of online prevention programs in schools has been reported in the literature. Literature review indicated that teachers’ online professional development not only contributed significantly to teaching effectiveness, but also could be implemented successfully in the classroom (Atkinson & O’Connor, 2007). In addition, online training provided a new world of opportunity in professional development (Carter, 2004). These experiences could be a model for online teacher training for tobacco prevention in the state. The World Health Organization (WHO, 1999) recommended interactive health information which involves active promotion, and uses tailored messages for individual adolescents to reach large numbers. Online programs provide the best opportunity to achieve such quality and audiences including behavior changes across various health issues (Christensen, Griffiths, & Jorm, 2004; Evers, Prochaska, Prochaska, Driskell, Cummins, & Velicer, 2003; Ritterband, GonderFrederick, Cox, Clifton, West, & Borowitz, 2003). Tobacco control is among the most widely studied and successful online programs reported (Walters, Wright, & Shegog, 2006). For example, ASPIRE (Prokhorov, Kelder, Shegog, Murray, Peters, Agurcia-Parker, et al., 2008), Consider This (Buller, Borland, Woodall, Hall, Hines, Burris-Woodall, et al., 2008), Smoking Zine (Norman, Maley, Li, & Skinner, 2008), Headbutt (Shegog, McAlister, Hu, Ford, Meshack, & Peters, 2005), and Preventing Teen Smoking with Virtual Reality (Nemire, Beil, & Swan, 1999) all reported effectiveness in reducing and preventing tobacco use. The Palm Beach School District has been a recipient of the Tobacco Prevention and Intervention Teacher Training (TPITT) Project for several years and successfully used online training for teacher training in tobacco prevention with positive impact evaluation evidence (Lacey, 2008). In the 2011 grant year, the Palm Beach School District had been contracted to provide the Statewide Online Tobacco Prevention and Intervention Teacher Training (SOTPTT) using TrainU, an online training program designed for teachers’ continuing education. To improve and evaluate the effectiveness of SOTPTT, an evaluation was conducted to assess the outcomes of the Program both in the trained teachers and their students receiving the tobacco prevention education. This article aimed to provide a summative evaluation of the Statewide Online Tobacco Prevention and Intervention Teacher Training Project (SOTPTT). The
144 / CHEN, SHEU AND WENG
outcome evaluation included assessment of learning outcomes of teachers who received training, impact on students who were taught by the trained teachers, and program satisfaction of teachers who participated in the project. METHODS Procedure The literature review and theoretical framework are fundamental components in this evaluation. A Logic Model (Cato, Chen, & Corbett-Perez, 1998; Chen & Cato, 1999; W. K. Kellogg Foundation, 2004) was developed to analyze the intervention and guide the evaluation design. A two-group quasi-experimental design was implemented. To determine the Program’s outcomes, the data were collected from both the trained teachers and the students taught by the trained teachers. To ensure the quality of data, the research team reviewed and reassessed the instruments used previously in the Palm Beach School District’s TPITT’s evaluation project. In addition, instruments published in the literature were reviewed and selected items were integrated into the final instruments. As part of the instrument validation process, an expert panel was invited to examine the face validity and content validity. The internal consistency reliability was also established. As required by the SOTPTT, the participating teachers had to complete online pre- and post-assessments before and after online training on TrainU; teach six tobacco prevention lessons; distribute and collect the students’ pre- and postassessments before and after the six lessons were taught; and complete satisfaction survey, which included the coverage, length, difficulty, administration, and quality of the program components and the curriculum and materials provided on TrainU after the completion of the Program. The participating teachers who completed the aforementioned requirements received the continuing education credits approved by their school districts. The teachers who registered but did not complete the training were surveyed to understand their reasons of lapse and future motivators. Participants All teachers from 67 school districts in the state and laboratory schools were eligible to participate. A total of 344 teachers, including 72 K-3rd grade teachers, 44 4-5th grade teachers, and 228 6-12th grade teachers completed the online training program and were sent students’ assessment packages to assess their impacts on students. Among them, 323 teachers (93.9%) had completed at least six student lessons, collected student impact surveys, and returned the assessment package to the evaluation team. As a result, a total of 6,490 pairs of valid student pre-/post-assessments were received and entered into analysis to examine the impact of online tobacco prevention teacher training on their students.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
145
Data Collection and Analysis The evaluation protocol was submitted to and approved by the Institutional Review Board (IRB) of the University of Florida. The data obtained are kept confidential. The identifiers were only used to match participants’ pre- and post-assessment responses. After data analysis, the identifiers were deleted from data file to ensure confidentiality. No individual response is disclosed or reported elsewhere. All paper surveys were delivered to the confidential recycling unit for the proper protection of privacy. The students who were taught by the trained teachers completed a pre- and post-assessment to measure the impact teachers’ online training has on them as regard to their changes in knowledge about tobacco hazards, media influences over tobacco use, and refusal skills. The trained teachers requested the pre-/ post-assessments at about the mid-point of their online training and received sufficient number of paper pre-/post-assessments by postal mail. Specific instructions for the students to complete the surveys were included with the pre-/post-assessments. The trained teachers conducted students’ pre-assessment prior to teaching the lessons and collected the post-assessment immediately after the completion of the lessons. Postage paid return envelopes were provided for the teachers to send the pre-/post-assessments back to the evaluation team. All grades had three same areas in the pre- and post-assessment. They are the knowledge about tobacco hazards, media influences over tobacco use, and refusal skills. In the 4-5th and 6-12th grade surveys, two additional areas (outcome evaluation and overall assessment) were measured. In the area of outcome expectations, students were asked if they expect smoking to have positive outcomes, such as help to relax, stay thin, deal with problem or stress, and have something to do when bored. In the area of overall assessment, students were asked their decisions if they would enjoy smoking or choose not to smoke in the future. The evaluation data were analyzed for instrument validity and reliability. The data from the surveys were entered into Microsoft Excel, Access, and IBM SPSS for data management and statistical analysis. The students’ t-tests, paired t-tests, and one-way analysis of variance (ANOVA) were conducted in this analysis. The Cronbach’s alpha for teachers’ pre-/post-assessment (13 items) was .91, teachers’ satisfaction survey (8 items) was .85, K-3rd grade students’ pre-/postassessment (5 items) was .69, 4-5th grade students’ pre-/post-assessment (12 items) was .72, and 6-12th grade students’ pre-/post-assessment (15 items) was .82. RESULTS The results of this evaluation include teachers’ training outcomes and teachers’ impact on student’s learning.
146 / CHEN, SHEU AND WENG
Teacher Training Outcomes Table 1 shows the teachers’ training outcomes. The pre- and post-assessment scores from teachers who completed training and classroom teaching were compared to examine the training effects. Among the teachers who completed all requirements, 274 pairs of pre- and post-assessment data had no missing values and were matched and analyzed. Results showed teachers’ scores improved from 37.58 (SD = 6.49) to 48.87 (SD = 4.15) (maximum score 52). Overall, the participating teachers’ knowledge about tobacco prevention had 30% improvement after participation in the program with high statistical significance (t273 = 26.36, p < .001). This result showed very significant improvement in teachers’ scores after training both in total items and in the separate five categories assessed, including the knowledge about tobacco hazards, the knowledge about risk and resiliency factors related to tobacco use, the knowledge about current trends in tobacco use, marketing, and products, the knowledge about tobacco prevention resources, and the familiarity of tobacco prevention curriculum. Among the five categories assessed, the level of familiarity with tobacco prevention curriculum had the largest increase (50%) after training. In addition, the trained teachers stated their level of knowledge about tobacco prevention resources increased by 37%. Teachers’ satisfaction was measured by eight items in Likert scale (strongly disagree, disagree, agree, strongly agree). The results from 273 completed surveys showed the mean scores of 30.43 (maximum score 32) and standard deviation of 2.30. The teachers who participated in the training program showed high satisfaction to the program. In the area of teachers’ satisfaction on training, nearly all teachers agreed or strongly agreed that they are comfortable using computers (98.2%). The majority of the teachers agreed or strongly agreed the program was informative (97.7%) and user friendly (98.9). In addition, the teachers’ opinions showed the assignments were reasonable and appropriate (98.2%), and the program increased their interest (99.6%), made them feel confident (99.6%), and felt it was a positive experience (99.6%). The trained teachers were very satisfied with the quality of the training program with the mean scores of 18.99 (SD = 1.52, maximum score 20). The teachers also reported a high level of comfort in using computers (mean = 7.57 and SD = 0.72 based on the possible scores of 8). In addition, after the training, their confidence in teaching tobacco prevention lessons was very high (3.88 on the scale of 4). Impact on Students’ Learning Table 2 shows students’ pre- and post-assessment results by the grade levels. On average, the students had about 24% improvement in their knowledge about tobacco. In all grade levels, the post-assessment scores were found higher than the pre-assessment scores with very high statistical significance. The K-3rd
9.06 ± 1.66 3.16 ± 0.67 10.61 ± 2.42 5.00 ± 1.45
2 3 4 5 39.24 ± 10.00
9.75 ± 1.52
1
All
37.58 ± 6.49
Pre-assessment mean ± SD
All
Category
N/A
7.49 ± 0.82
14.54 ± 1.64
3.92 ± 0.35
11.27 ± 1.08
11.65 ± 0.99
48.87 ± 4.15
Post-assessment mean ± SD
Note: Categories 1. knowledge about tobacco hazards 2. knowledge about risk and resiliency factors related to tobacco use 3. knowledge about current trends in tobacco use, marketing, and products 4. knowledge about tobacco prevention resources 5. familiarity of tobacco prevention curriculum
Did not complete (129)
Completed all requirements (274)
Status
N/A
+49.8%
+37.0%
+24.1%
+24.4%
+19.5%
+30.0%
% change
Table 1. Teachers’ Training Outcomes
N/A
25.60
24.87
16.58
20.24
18.47
26.36
Paired t
N/A
< .001
< .001
< .001
< .001
< .001
< .001
Significance ONLINE TOBACCO PREVENTION TEACHER TRAINING / 147
1,266
K-3
• • • • •
6-12
• • • • •
4-5
Knowledge about tobacco hazards Knowledge about refusal skills Knowledge about media literacy Outcome expectations Overall assessment
4,339
Knowledge about tobacco hazards Knowledge about refusal skills Knowledge about media literacy Outcome expectations Overall assessment
885
• Knowledge about tobacco hazards • Knowledge about refusal skills • Knowledge about media literacy
6,490
Number of students
All
Grade
55.10 ± 24.18 87.96 ± 32.54 54.02 ± 39.42 81.61 ± 31.43 83.72 ± 32.65
67.56 ± 19.76
78.82 ± 22.61 91.48 ± 27.94 46.47 ± 35.79 84.96 ± 24.77 96.27 ± 18.97
76.92 ± 17.06
68.23 ± 30.16 73.04 ± 44.39 53.70 ± 49.88
65.69 ± 27.22
68.47 ± 21.37
Pre-assessment % mean ± SD
75.39 ± 23.34 93.38 ± 24.87 79.43 ± 32.49 87.01 ± 27.75 88.40 ± 27.66
81.41 ± 18.59
95.22 ± 12.25 97.16 ± 16.62 75.72 ± 32.01 91.64 ± 19.96 97.78 ± 14.73
91.07 ± 12.49
94.71 ± 15.80 95.72 ± 20.24 89.28 ± 30.95
93.18 ± 15.71
85.02 ± 18.09
Post-assessment % mean ± SD
36.8% 6.2% 47.0% 6.6% 5.6%
20.4%
20.8% 6.2% 62.9% 8.2% 1.6%
18.5%
38.5% 30.1% 64.8%
41.9%
24.1%
% increase
44.64 9.93 38.51 10.87 9.61
41.70
20.65 5.55 20.82 7.16 1.99
23.37
27.84 17.31 22.51
32.09
55.64
Paired t
Table 2. Pre- and Post-Assessment of Students’ Knowledge about Tobacco Hazards by Grade Levels
< .001 < .001 < .001 < .001 < .001
< .001
< .001 < .001 < .001 < .001 < .05
< .001
< .001 < .001 < .001
< .001
< .001
Significance
148 / CHEN, SHEU AND WENG
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
149
graders appeared to have the highest improvement (41.9%), followed by 6-12th (20.4%), and 4-5th graders (18.5%). Students’ improvement data were analyzed using the trained teachers as the analysis unit. The students’ mean scores after the six lessons taught by each trained teacher increased by almost 17%. For individual grade groups, students in K-3rd, 4-5th, and 6-12th grades showed improvement of 27.67%, 15.71%, and 13.71% respectively. Results of further statistical analysis by one-way ANOVA indicated that teachers in K-3rd grades had the most significant impact on their students. Their mean percent improvement was found significantly better than the 4-5th and 6-12th grades separately (F = 28.75, df = 2, 309, p < .001, Tukey HSD, Scheffe, LSD: p < .001). To further examine teachers’ impact on students’ learning, four levels in the students’ pre- and post-assessment score changes were created: Increased significantly, Increased, Decreased, and Decreased significantly. To be qualified as increased or decreased significantly, the students’ post-assessment mean scores must be increased or decreased, respectively, by p < .05 significance level from their pre-assessment mean scores. Overall, the vast majority of the students taught by trained teachers had improved (94.6%) their knowledge about tobacco. Among K-3rd, 4-5th, and 6-12th graders, 97%, 97.4%, and 93.2% of the students showed improved outcomes respectively. The majority of the teachers who completed the program had their students’ post-assessment mean scores significantly better than their pre-assessment. For example, approximately 90% of the 4th and 5th grade students had significantly better knowledge on tobacco prevention. In the meantime, a total of 16 (5.1%) classes showed decreased knowledge about tobacco prevention. Among them, the majority (14 classes) was in the 6-12 grades. Results analyzed by grade category showed K-3rd grade students had 64.8%, 38.5%, and 30.1% improvement in the knowledge about media literacy, knowledge about tobacco hazards, and knowledge about refusal skills respectively after being taught by the trained teachers. The program had the largest impact on K-3rd grade students in their knowledge in media literacy, followed by the knowledge about tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001). In addition, 4-5th grade students had 62.9%, 20.8%, 8.2%, 6.2%, and 1.6% improvement in the knowledge about media literacy, knowledge about tobacco hazards, outcome expectations, knowledge about refusal skills, and overall assessment respectively after being taught by the trained teachers. The program had the largest impact on 4-5th grade students in their knowledge about media literacy, followed by the knowledge on tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001) except the student’s overall assessment of tobacco use (p < .05). Furthermore, 6-12th grade students had 47.0%, 36.8%, 6.6%, 5.6%, and 6.2% improvement in the knowledge about media literacy, knowledge about tobacco
150 / CHEN, SHEU AND WENG
hazards, outcome expectations, overall assessment, and knowledge about refusal skills, respectively, after being taught by the trained teachers. The program had the largest impact on 6-12th grade students in their knowledge about media literacy, followed by the knowledge on tobacco hazards. The statistics in all areas reached very high statistical significance (p < .001). The majority of 6-12th graders (3,469; 79.9%) chose not to smoke in pre-assessment and kept their decision in the post-assessment. The impact on students’ projected decision not to smoke increased by 31% after six lessons had been taught among 6-12th graders. DISCUSSION Online tobacco training programs appeared successful in training teachers, but its impacts on students have not been assessed extensively. This evaluation study not only assessed the impact of online training on teachers but also examined its impact on students who were taught by the trained teachers. Specifically, this study examined among participating teachers what their changes in knowledge about tobacco prevention education are before and after the program, what are their satisfaction with the quality of the online training program and their confidence in teaching tobacco prevention lessons. In addition, the impacts on the students who were taught by the trained teachers, in the areas of knowledge about tobacco hazards, medias influences over tobacco use, refusal skills, attitudes toward tobacco use, and projected decision not to use tobacco, were also examined. Results of the evaluation revealed that the online training program was indeed effective not only in improving teachers’ knowledge about tobacco prevention education, but also their satisfaction with the online training program. Most significantly, the training program also had a successful impact on students. Students’ knowledge about tobacco hazards, the influence of media on tobacco use, and refusal skills were improved in all grade levels. For students in grades 4-5th and 6-12th, the program also showed improvement in their attitude toward tobacco use and their projected decision not to use tobacco. To examine if the teachers who completed the training had better knowledge in tobacco prevention prior to the training than the teachers who did not complete the program, the pre-assessment scores of the teachers who completed the program were compared with the pre-assessment scores of the teachers who did not complete the program. The pre-assessment scores between the teachers who completed the program and the teachers who did not complete the program were found to have no statistical significance. The overall scores for the trained teachers before online training was 37.58 (out of possible 52 points from 13 questions) compared with 39.24 for the teachers who signed up for the training program but did not complete the program. This result showed the participating teachers were not different from the teachers who did not complete the program and may imply that all teachers, if they completed the program, could have as significant knowledge improvement in tobacco prevention as the participating teachers.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
151
One of the major challenges for the online tobacco prevention education for school teachers is lack of incentives to attract teachers to spend a significant portion of their free time to attend the lessons. Financial awards were not permitted to encourage participation by state regulation. As a result, continuing education credits for teacher re-certification were requested and approved by participating school districts. The continuing education credits provided to the trained teachers brought a major force of motivation to participate in this training program. A total of 60 continuing education credits could be awarded when teachers completed training and all requirements including data collection. Another driving force is that the training being delivered online and the teaching requirements of tobacco prevention lessons could be conveniently completed in the classes the teachers taught. Such design saved teachers’ time and costs and increased the accessibility and availability of intervention materials and consultation. The education authorities are suggested to offer similar continuing education credits for health related trainings in the future. Even with such an incentive, a significant number of teachers who had signed up for the online program either had never accessed the program or did not complete the training. The major reasons provided included lack of times, unexpected family emergency, or personal health problems. The high incompletion rate would be a major barrier for future programs, and further incentives need to be considered for improvement. Interpretation of the results needs to be cautious and should not be generalized as this was a quasi-experimental evaluation design based on a convenience sample and no control group was included in the evaluation. While every precaution was taken to make sure the validity of the data, teachers’ data were collected online and students’ data were collected by trained teachers. Additional limitations of this study include the long-term effects on knowledge, attitudes, media literacy, refusal skills, and actual tobacco use not being assessed; a structured assessment on intention to use tobacco, smoking susceptibility, or past-month smoking rates not being included as outcome indicators; and the attribution of short-term knowledge change to stay smoke-free or quit smoking. Further longitudinal surveillance studies are recommended. Online training programs for tobacco prevention have not been used widely in the past and could be more strategically adopted if they can be proven to be effective for training teachers. This evaluation study provided preliminary data to support the use of online program for training teachers to conduct tobacco prevention. The evaluation results indicated that participating teachers not only improved their knowledge about tobacco and tobacco prevention but also their confidence in conducting prevention education for their students after training. Additionally, the evaluation study was able to provide preliminary data supporting the impacts of the program on students who were taught by their trained teachers. It is recommended that this online training program be continued and expanded in the state and results evaluated continuously in the future.
152 / CHEN, SHEU AND WENG
REFERENCES Atkinson, T. S., & O’Connor, K. A. (2007). Establishing professional development partnerships online: Reaching out to veteran teachers. TechTrends: Linking Research and Practice to Improve Learning, 51(3), 21-29. Baska, T., Straka, S., Baskova, M., & Madar, R. (2004). Effectiveness of school programs in tobacco control. Central European Journal of Public Health, 12(4), 184-186. Buller, D. B., Borland, R., Woodall, W. G., Hall, J. R., Hines, J. M., Burris-Woodall, P., et al. (2008). Randomized trials on Consider This, a tailored, internet-delivered smoking prevention program for adolescents. Health Education & Behavior, 35(2), 260-281. Carter, K. (2004). Online training: What’s really working? What does today’s successful online professional development look like? Technology & Learning, 24(10), 32. Cato, B., Chen, W. W., & Corbett-Perez, S. (1998). Logic Model—A tool for planning and evaluating health and recreation prevention projects. Journal of Physical Education, Recreation, and Dance, 69(8), 57-61. Centers for Disease Control and Prevention (CDC). (n.d.). Youth online: Comprehensive results. Retrieved from http://apps.nccd.cdc.gov/youthonline/App/QuestionsOr Locations.aspx?CategoryId=2 Chen, W. W., & Cato, B. (1999). Using a Logic Model to plan and evaluate community intervention program—A case study. International Quarterly of Community Health Education, 18(4), 451-460. Christensen, H., Griffiths, K. M., & Jorm, A. F. (2004). Delivering interventions for depression by using the internet: Randomised controlled trial. British Medical Journal, 328(7434), 265. Dobbins, M., Decorby, K., Manske, S., & Goldblatt, E. (2008). Effective practices for school-based tobacco use prevention. Preventive Medicine, 46(4), 289-297. Evers, K. E., Prochaska, J. M., Prochaska, J. O., Driskell, M., Cummins, C. O., & Velicer, W. F. (2003). Strengths and weaknesses of health behavior change programs of the Internet. Journal of Health Psychology, 8(1), 63-70. Florida Department of Health. (2009). 2008 Florida Youth Tobacco Survey county data book. Retrieved from http://www.hpcnef.org/files/health-needs-assesments/2008 DataBook_web.pdf Fray, B. R. (2009). School-based smoking prevention programs with the promise of long-term effects. Tobacco Induced Diseases, 5, 6. Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2008). Monitoring the Future national results on adolescent drug use: Overview of key findings, 2007. Bethesda, MD: National Institute on Drug Abuse. Lacey, C. H. (2008). Evaluation of the Tobacco Prevention and Intervention Teacher Training Project. Palm Beach, FL: School District of Palm Beach County. Muller-Riemenschneider, F., Bockelbrink, A., Reinhold, T., Rasch, A., Greiner, W., & Willich, S. N. (2008). Long-term effectiveness of behavioural interventions to prevent smoking among children and youth. Tobacco Control, 17(5), 301-312. Nemire, K., Beil, J. A., & Swan, R. W. (1999). Preventing teen smoking with virtual reality. CyberPsychology & Behavior, 2(1), 35-47. Norman, C. D., Maley, O., Li, X., & Skinner, H. A. (2008). Using the Internet to assist smoking prevention and cessation in schools: A randomized, controlled trial. Health Psychology, 27(6), 799-810.
ONLINE TOBACCO PREVENTION TEACHER TRAINING /
153
Peterson, A. V., Jr., Kealey, K. A., Mann, S. L., Marek, P. M., & Sarason, I. G. (2000). Hutchinson Smoking Prevention Project: Long-term randomized trial in school-based tobacco use prevention-results on smoking. Journal of the National Cancer Institute, 92(24), 1979-1991. Prokhorov, A. V., Kelder, S. H., Shegog, R., Murray, N., Peters, R., Jr., Agurcia-Parker, et al. (2008). Impact of A Smoking Prevention Interactive Experience (ASPIRE), an interactive, multimedia smoking prevention and cessation curriculum for culturally diverse high-school students. Nicotine and Tobacco Research, 10(9), 1477-1485. Public Health Service. (1964). Smoking and health: Report of the Advisory Committee to the Surgeon General of the Public Health Service. Public Health Service Publication No. 1103. U.S. Department of Health, Education, and Welfare. Ritterband, L. M., Gonder-Frederick, L. A., Cox, D. J., Clifton, A. D., West, R. W., & Borowitz, S. M. (2003). Internet interventions: In review, in use, and into the future. Professional Psychology, 34(5), 527-534. Shegog, R., McAlister, A. L., Hu, S., Ford, K. C., Meshack, A. F., & Peters, R. J. (2005). Use of interactive health communication to affect smoking intentions in middle school students: A pilot test of the “Headbutt” risk assessment program. American Journal of Health Promotion, 19(5), 334-338. Skara, S., & Sussman, S. (2003). A review of 25 long-term adolescent tobacco and other drug use prevention program evaluations. Preventive Medicine, 37(5), 451-474. W. K. Kellogg Foundation. (2004). Using Logic Models to bring together planning, evaluation, and action: Logic Model development guide. Battle Creek, MI: W. K. Kellogg Foundation. Walters, S. T., Wright, J. A., & Shegog, R. (2006). A review of computer and Internetbased interventions for smoking behavior. Addictive Behaviors, 31(2), 264-277. Wiehe, S. E., Garrison, M. M., Christakis, D. A., Ebel, B. E., & Rivara, F. P. (2005). A systematic review of school-based smoking prevention trials with long-term follow-up. Journal of Adolescent Health, 36(3), 162-169. World Health Organization (WHO). (1999). Programming for adolescent health and development: Report of a WHO/UNFPA/UNICEF study group on programming for adolescent health. Report 886. Saillon, Switzerland: World Health Organization. World Health Organization (WHO). (2005). Preventing chronic diseases: A vital investment. Geneva, Switzerland: World Health Organization. Retrieved from http://www. who.int/chp/chronic_disease_report/en/
Direct reprint requests to: W. William Chen, Ph.D., MCHES Professor, Department of Health Education and Behavior FLG 19, University of Florida Gainesville, FL 32611-8210 e-mail: [email protected]
Copyright of Journal of Drug Education is the property of Baywood Publishing Company, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
