J. DRUG EDUCATION, Vol. 20(3)199-225,1990

EVALUATION OF A COMPREHENSIVE ELEMENTARY SCHOOL CURRICULUM-BASED DRUG EDUCATION PROGRAM

RUDY AMBTMAN

Alwholism Foundation of Manitoba PAUL MADAK University of Manitoba DENISE KOSS Manitoba Health Organizations, Inc. MARK J. STROPLE Saskatchewan Alcohol and Drug Abuse Commission

ABSTRACT

This study was designed to evaluate the effectiveness of a province-wide chemical abuse prevention education program for grades two through six. Two separate studies were conducted. Study 1measured pre- and post-scores on a test with 1101 experimental and 991 control subjects in a quasiexperimental design. Study 2 surveyed 500 elementary school teachers on their opinions and practices regarding drug education, and on the prevalence of the program in schools. The study found that: 1)students had a high initial knowledge level; 2) students showed modest but statistically significant knowledge gains for urban schools and grades three, four, and five of m a 1 schools; 3) the program was most effective with students with a low base knowledge; 4) an estimated 20 to 30 percent of Manitoba elementary teachers had been in-serviced and/or were using the program across the province; and 5) the program was received well among teachers.

DRUG EDUCATION PROGRAMS Drug abuse prevention programs have become an important part of student education across North America. Despite the fact that these programs are popular and 199

0 1990,Baywood Publishing Co.,Inc.

doi: 10.2190/7MXX-K123-1KQL-CHHC http://baywood.com

200 I AMBTMAN E T A L

have been taught for many years [l],much controversy exists over their success in achieving drug education goals. Moral objections which advocated temperance and scare tactics were the first approaches used in drug education programs. Currently three major approaches to drug education are in use [2-41: 1. A knowledge/attitudes/behaviorapproach; 2. A values/decision-making approach; and 3. A social competency approach.

The knowledge/attitudes/behavior approach is based on the assumption that inadequate knowledge of the consequences of drug abuse results in inappropriate attitudes and behaviors. This approach focuses on providing information on the consequences of drug abuse in order to promote appropriate attitude and behavior changes. The values/decision-makingapproach assumes that drug abuse occurs as a result of inappropriate values and inadequate decision-making skills. The emphasis of this approach is to promote self-understanding and responsible decisionmaking, and thus responsible decisions about drug use. The last approach assumes that substance abuse is caused by deficient psychosocial skills. This approach concentrates on modeling health-promoting behaviors and teaching skills to resist social influences (e.g., those of peers and the media) that promote substance abuse.

EFFECT OF DRUG EDUCATION ON STUDENTS Many studies have been done to substantiate that alcohol and drug education programs can increase knowledge and may also alter attitudes towards substance use [l, 4-71. The question of the extent to which alcohol and drug education programs modify actual drug use is not as easily answered [8]. Many of the programs have not been evaluated fully. Major flaws in methods of data collection and interpretation have raised doubts about the validity of evaluations that were done 19,101. In fact, some evaluations have shown an increase in drug use after program implementation [ll,121. Several researchers have assessed the reasons for the successful performance or failure of drug education programs in schools [l,4,9,13-153. There appears to be some agreement that the more successful programs make students cognizant of the social factors (e.g., from peers, family, and advertising) influencing their decisions to use or not use drugs and provide them with the skills to resist or cope with these pressures. In other words, these are the programs which follow the social competency approach. A number of other factors have been identified as influencing the success of drug education programs. One of these factors is the duration of the programs. According to Goodstadt, a typical drug education program ranges from two to five sessions extended over a two-week period [9]. Schaps et al. found that 57 percent

DRUG EDUCATION PROGRAM / 201

of 127 drug education programs consisted of only a film or a single presentation made by a guest speaker [lo]. Bangert-Drowns noted that one-third of thirty-three studies used in a meta analysis were less than one week in duration [8]. Evaluation research has shown that greater gains in health knowledge, practices, and attitudes were achieved as the number of classroom hours of health education increased [16, 171. More hours were required to produce a significant attitude change than changes in either knowledge or practices [17J Connell et al. concluded that stable levels of increase can be obtained for knowledge, practices, and attitudes with approximately fifty classroom hours [16]. Connell and Turner found that teachers were able to decrease the amount of classroom instruction time during the second year that they taught a program and achieve a greater increase in student knowledge, attitudes, and practices than were teachers teaching the program for the first year [18].In addition, the authors found a greater effect on knowledge, attitudes, and practices of students who had been exposed to two units of health instructions than on students exposed to only one unit. These results suggest that school health instruction should be extended through several grades. Bangert-Drowns found that a critical factor in the success of drug education programs was student participation [8]. The author recommended that programs be designed with active participation of students built in, in order to obtain maximum behavioral change in consumption behaviors. Another factor influencing program success has been inadequate program implementation [19, 201. Inadequate program implementation has been linked to a lack of teacher in-servicing [19,21]. In general, the better the teacher in-service training, the more likely the curriculum is taught as designed and the better the results. Fors and Dorster found that teachers with full training adhered to the curriculum more than 80 percent of the time [21]. In contrast, teachers with partial or no training adhered to the program 70 and 60 percent of the time, respectively. Impacts of 90 and 85 percent were obtained for attitudes and practices respectively when programs were fully implemented. Full implementation was defined as more than 80 percent of program activities taught, with greater than the program average degree of fidelity to program materials. Smaller gains of 5 percent for program specific knowledge and 20 percent for general knowledge were obtained with full implementation. Similarly, DiCicco et al. observed a 10.5 percent increase in desirable attitude responses with proper implementation as compared to no increase with improper implementation [19]. They also observed an 8.4 percent increase in knowledge with proper versus improper implementation. However, one of the main reasons that drug education programs have not met with success is the assumption that a few hours of drug education can overcome a lifetime of learning about drugs from peers, parents, and the community [9,12]. In addition, what is being taught in the classroom is inconsistent with what is being modeled and reinforced in the community [22]. Thus, while drug education in

202 / AMBTMANETAL

schools can play a role in drug use prevention, greater success can be achieved by involving the community as a whole.

TUNING IN TO HEALTH Despite the preponderance and variety of drug abuse prevention programs available, few program have proven sufficiently successful to warrant recommendation [12]. Moreover, drug education programs that provide links with other areas of the school curriculum were lacking [9]. Tuning In To Health: Alcohol and Other Drug Decisions PITH) was designed to integrate all the factors which research had shown to be necessary for success. It was developed by the Alcoholism Foundation of Manitoba (AFM, the provincial addictions agency) in a collaborative effort with Manitoba Education, the provincial education body.' A close working relationship between the two organizations ensured that TITH was consistent with the principles of Manitoba's health education curriculum. Although it is an optional unit, TITH is unique in that it can be taught as an integral part of the provincial health education curriculum. The program is longitudinal, spanning grades two to nine. In addition to the longitudinal nature of the program, attempts were made to include those elements which have demonstrated effects on the attitudes and behaviors regarding chemical use of students. Each section of the program has a number of student activities built, and a system of teacher in-service training was developed. Teachers are in-serviced, either directly or by an AFM field worker or by a fellow teacher who was specifically trained to in-service other teachers. The purpose of the TITH program is to reduce the future incidence of problems associated with drugs by helping students to understand [23]:

1. What drugs are; 2. The effects of drugs on the body; 3. The factors that influence people to use or not use drugs; 4. Decision-making as a way to deflect influences that promote drug use; and 5 . Alternatives to drug use. The general objectives of the program are to:

1. Define a drug as a substance which changes the functioning of the human body; 2. Identify the short- and long-term effects of alcohol and other drugs; The original (1983) elementary program was an interim document, and developed by L Gamey, K Rampling, V. Paape, and K. Hatcher. The revised elementary program was co-authored by Strople and KCKSin 1986 [U]. The Junior High section was developed by Saranchuk and KCKSin that same year [24].

DRUG EDUCATION PROGRAM / 203

3. Examine the factors that influence non-use/use of drugs; 4. Analyze decisions on the non-use/use of drugs; 5. Examine alternatives to the use of drugs; and 6. Promote a positive, healthy lifestyle which follows prevailing legal, social, and community norms on the non-use or use of drugs.

The TITH programs for Early Years (grades two and three) and Middle Years (grades four through six) are grade-specific containing the following units: Hazardous Products (grade two), Medicines (grade three), Drugs and Decisions (grade four), Drugs and Influences (grade five), and Smoking (grade six) [23]. The Junior High TlTH program is not grade specific [24]. It consists of thirty-six lessons divided into four modules: Alcohol, Tobacco, Over-the-Counter and Prescription Drugs, and Other Drugs. These modules all include information on the effects of drugs on the body, factors influencing the use/non-use of the drugs, decision making, and lifestyle.

THE PRESENT STUDY A two-study (outcome and process) evaluation of TlTH was designed in order to determine the level of success achieved by the elementary section of the program and to make recommendations for program improvement. The Junior High program still is in the early implementation stages and is scheduled for evaluation at a later date. A secondary goal of the study was to contribute to the scientific knowledge regarding elementary school chemical substance use education. For example, Bangert-Drowns demonstrated that for elementary grades only four studies could be found suitable for analysis [8]. Only one of these spanned more than two grades

1251. Study 1was a study to assess changes in student knowledge as a result of being taught TITH. Study 2 of the evaluation consisted of a survey among Manitoba teachers to determine teacher satisfaction with the design and implementation of the program. In addition, teachers were surveyed on their comfort level with substance-related issues, their background, and their views on drug education.

METHOD Study 1:Student Outcome Participant selection - Study participants consisted of grades two through six public school students across the province of Manitoba (Canada). The unit of sampling was schools, rather than classes or individual students. Using the latter two units would have been preferable from a theoretical point of view, but would have posed insurmountable administrative problems in the field.

204 / AMBTMAN E T A L

Initially, all forty-seven public school divisions and an additional six school districts across Manitoba were approached with a request to participate in the study, either in the experimental or control groups. Of these fifty-three divisions and districts, twenty-three (43.4%) agreed to participate, ten (18.9%) refused participation, and twenty (37.7%) either did not reply formally or responded too late to be included in the study. The consenting districts and divisions were subsequently divided based on two primary dimensions: TITH-using or not (experimental vs. control) and urban (i.e., Winnipeg) vs. rural (non-Winnipeg). Schools within each of the four resulting cells were pooled, eliminating school division as a factor. Samples were drawn randomly within each cell. Since class sizes varied greatly from school to school (Le., from four to thirty students), sampling of schools continued until a minimum of 120 students at each grade level (two through six) in each cell was obtained. Where more than one class per grade level was available in a school, only one class was randomly selected to study, so that each school provided up to five classes for the study. Thirty-one schools participated in the study. Numbers of schools per cell were seven (Winnipeg TITH), nine (Rural TITH), six (Winnipeg Control), and nine (Rural Control). Instruments - One questionnaire was developed for each grade level. The first part of the instrument consisted of a number of personal items (grade, sex, birthdate), primarily intended for descriptive purposes and to enable the researchers to match pre- and post-questionnaires. Since questionnaires were anonymous, students’ initials rather than names were used. The second part measured students’ knowledge of essential elements of the TlTH unit for the students’ grade level using primarily a true/false format, with some three to four alternative multiple choice questions and a few questions requiring matching or ordering. Numbers of knowledge questions varied from twenty-six to fifty-two, depending on the grade level. The final section of the questionnaires contained fourteen true/false questions asking about previous exposure to drug education. The first and last sections were identical for each grade level. The various questionnaires were piloted in two elementary schools. Observed by a researcher, teachers administered the questionnaires to their class. Subsequently, teachers completed a teacher feedback questionnaire on the instruments. Separate discussions with students and teachers followed administrationof the instrument. Minor revisions to the questionnaires occurred in order to remove confusing questions, questions inappropriate for the reading level of a grade, or questions with a near 100 percent correct rate.

Procedure - Since all the participating TITH schools did not teach TITH until the second half of the school year, both TITH and control schools were sent the (‘pre-’) questionnaires early in January 1988. The participating teachers administered the instruments shortly after receiving the questionnaires. Completed

DRUG EDUCATION PROGRAM / 205

questionnaires were returned directly to a consulting firm contracted to collect the data. Since TITH units were typically taught over a relatively protracted period of time, all schools received and completed the second (‘post-’) questionnaire in the latter half of April 1988. The pre- and post-questionnaireswere identical. , Teachers were given instructions on how to administer the questionnaires and were given instruction forms to read to their students. Teachers of grades two and three read the entire questionnaire to their students. Students in grades four through six worked through the questionnaire at their own pace, but were told they could ask their teacher for explanations of questions. Questionnaires were coded and later matched based on the student initials and birthdate on both questionnaires. At the request of most of the school divisions, each school was asked to deal with the issue of parental consent. Typically schools sent letters to parents advising them of the study and asking parents to notify the school if they did not want their children to participate. Letters to that effect were provided to the schools on request. There were no reports of parental refusals, suggesting that this was not an issue with parents. Teachers were asked to complete a questionnaire on the test administration @re- and post-) and on the method of teaching TITH @st-, for experimental classes only).

Study 2: The Process of TlTH Use

Participant selection - Participants were members of the Manitoba Teachers’ Society (MTS). The Society is a labor union and professional organization to which all teachers in Manitoba belong. The MTS was asked to randomly select 500 teachers currently teaching at the elementary level (grades one through six).’ Instrument - The selected teachers were each sent a questionnaire consisting of two sections totalling forty-three items. First section questions were directed at a number of demographic and personal variables (e.g., which class the respondent was currently teaching, region of work, previous experience and education, and comfort level with teaching various chemical use/abuse education programs). The first section was to be completed by all respondents. The second section, to be completed by TITH users only, consisted of a number of items enquiring about the respondent’s training in the use of TlTH and explored teachers’ experience with teaching the program to their students.

*

In addition to the Elementary teachers, a total of 500 Junior High school teachers were sent similar questionnaires. Since the Junior High section of TTIH was not evaluated with students, the Junior High teacher survey is not discussed here.

206 / AMBTMAN E T A L

Procedure - The MTS mailed the questionnaires to 500 elementary school teachers, randomly selected from its membership list. Questionnaires were accompanied by an introductory letter from the MTS urging the recipient to complete the questionnaire and to return it in the postage prepaid envelope with the MTS return address. Questionnaires were sent in May 1988.Recipients were not given a deadline to return the questionnaires. The questionnaires collected by the MTS were processed and analyzed by the AFM.

RESULTS Study 1 A total of 2406 pre- and 2364 post-questionnaires were returned by the various schools, resulting in 2092 matches being made. This 13.1 percent attrition rate is considered reasonably low, given the approximately three- to four-month lapse between both administrations. Unmatched questionnaires resulted primarily from unclear or missing identifiers, natural absenteeism of students (e.g., sickness, move to another school) and to one Winnipeg control school which initially consented to participate, but after partial cooperation with the pretest subsequently refused further involvement in the study. One grade four class in one Winnipeg TlTH school also refused to administer the posttest. This attrition resulted in some cells having smaller numbers than expected (ranging from 69 to 143). Following the pilot of the initial questionnaire, it became apparent that frequencies of correct scores for a number of questions for most grades were quite high. This occurred despite the fact that the pilot students had not received training in the pertinent unit of TITH. It was recognized that this would likely lead to a ceiling effect in the actual study by leaving little room for improvement between pre- and posttests. However, it was felt that the content of most questions was sufficiently critical to the essence of the program to maintain them in the survey. In addition, many of the questions could not be changed to make them more difficult given the cognitive and reading levels of the population. It was therefore decided that the ceiling effect was a necessary risk. Upon review of the prequestionnaire results these high scores were indeed encountered (Table 1). Several steps were taken to verify that the variables used in the analyses were either normally distributed, or were normalized, and did not show non-linearity when combined with other variables. First it was decided that, as the primary indication of change in scores from pre- to posttest, a simple subtraction of posttest score less pretest score would be used. This new variable, entitled CHANGE,was normally distributed for each grade, in contrast to the two variables from which it was derived.

DRUG EDUCATION PROGRAM / 207

Second, one of the major variables of interest was the pre-score. This variable had a significantly negative skew for all grades, except grade four. For all other grades, this variable was normalized using accepted practices (i.e., square root conversion of the reflexed variable). Third, all the major variables which where used in the same statistical analyses were examined for non-linearity and multivariate outliers using paired regression analyses. None were found to be non-linear. Outliers were temporarily removed only for those analyses where they were deemed a problem. Results of subsequent analyses were fairly consistent over each grade level, although there were some major differences between non-Winnipeg (or‘mral’) and Winnipeg (or ‘urban’) schools. These differences are discussed below for each rural and urban setting separately. Analyses of covariance were performed with CHANGE as the dependent variable, the experimental condition (i.e., TITH or control) and the geographic variable (i.e., rural or urban) as the independent variables, and the pretest scores as the covariates. The pretest score was chosen as the covariate primarily to control for the ceiling effect described above. These analyses showed highly significant (p c .001) effects for the experimental condition and covariate for all grades. The main effect of the rural-urban variable was significant (p c .02 or better) for grades two through five, but not significant for grade six. Significant interactions between the two independent variables were found only for grades four and five (p c .001and c .01,respectively). Given the rural-urban differences, these two locales were analyzed separately. CHANGE and pretest scores were inversely related, showing statistically significant but moderate Pearson’s correlation coefficients (using the transformed pretest scores) ranging from -.30 (grade five) to -.45 (grade 2). The teacher questionnaires accompanying the posttests for TITH schools were intended to be included in the statistical analysis of the results of the experimental students. However, the return rate for completed questionnaires was only 85 percent. This would have caused approximately 15 percent of the students in selected classes to be excluded from these analyses. Since the missing data could have dramatically biased the results of any statistical analysis, the questionnaires were not used for further statistical analyses. Informal inspection of the questionnaires suggested that the pattern of teaching TITH was very similar in Winnipeg and non-Winnipeg schools with respect to length of lessons (typically between sixteen and forty minutes) and length of period the unit was taught (under two months). However, non-Winnipeg schools had a greater tendency not to teach the entire unit. Especially lessons toward the end of a unit were taught less often.

Winnipeg Schools - For the urban schools, findings for the pre- and postquestionnaires were quite uniform across all five grade levels. &Tests for all pretest comparisons between experimental and control schools were not significant (Table 1).On the posttests the differences were all significant (p c .01or better). Differences between experimental and control schools on CHANGE

8

7 7

h

6

4

h

v)

6

h

2

Y

I

1

h

h

u)

6

&

4

h

h

v)

v)

&

&

n

208

I

I

(.O1)

L2b!b

ol

1

I

J (.OdOll

I

38.02

11 1

126

k0!86

112

127

20

119

08

86

125

130

113

119

128

111

133

128

117

110

126

120

Grade 5

H

28.26

ol

H ol

(05)

12qyJ

(*01)

28.80

Grade 6

H

H

ol

“’3

-34.95

ol

(ns)

36.18 (.q

- 34.43

(ns)

(ns)

(ns)

ures in parentheses W e e n spans indicate level of significance (e.g., p < .01; ns = p > .05) of t-tests (two-tailed) (ma matched cases). ximum possible scores: 28,26,50,46,52, for Grades two through six, respectively.

210 I AMBTMAN ET AL.

scores were also significant for all grade levels (p < .01 or better). All change occurred for both experimental and control schools in a positive direction and all experimental groups showed a greater CHANGE than the control groups. The greatest difference in CHANGE scores was found for grades three and five, the smallest effect for grade six. Rural SchooZs - For the rural schools, the overall situation appeared somewhat less clearcut (Table 1). Non-significant differences for 1-tests between experimental and control groups on the pretest were found only for grades four and five. For grades two, three, and six, statistically significant differences were found between experimental and control schools (p < .01, < .05, and < .01, respectively). For these grades, experimental schools had higher pretest scores. This may suggest that rural experimental schools were qualitatively different from control schools in some manner (see below). The posttest differences between experimental and control schools for all grade levels were typically greater than on the pretest (p < .01 or better, Table 1). However, only grades three, four, and five showed significant differences on the CHANGE scores, all in the direction of greater improvement for the experimental schools (p c .01or better). When the prescore was included as a covariate in an ANOVA, CHANGE scores for grade 2 also showed a highly significant difference between TITH and control groups. With the same analysis of variance, the F value of the experimental condition for grade six was much smaller. Since there appeared to be qualitative differences between rural control and experimental schools, further analyses were necessary. Examination of Table 1 reveals that there were no significant differences between any of the experimental urban and rural schools on the pretest. However, in all but one (grade six) there were significant differences between rural and urban control groups (p < .05 or better). Generally, rural controls had lower pretest means than their urban counterparts. Further examination of Table 1 indicates that rural control groups had the lowest scores on the pretest of all four groups for all but one (grade five) grade level. Although this would suggest that children in the rural schools generally had the least exposure to any form of information on dangerous or addictive substances, the findings of self-reported exposure only partially confinned this. Only in grades four and five were significant differences on pre-exposure totals found (Table 2). For these grades, rural experimental groups typically reported more preexposure than the other three groups, while rural control groups only were low in comparison to their experimental rural counterparts, not in comparison to the urban groups. This finding is even more puzzling since, as indicated above, only grades four and five of the rural schools had nonsignificant differences on the pretest between experimental and control schools. These two observations are unexpected, but are

2

h)

TlTH Control

TITH

3 Control

Control 7.95-(.03)4.88

TlTH

4

5 Control 8.60-(.01)-9.43

TITH

Note: Fgures in parentheses indicate significance level (e.g., p < .01; ns = p > .05) of t-tests (two-tailed).

Rural

Winnipeg

Location

2

Table 2. Mean Reported Exposure to Chemical Information Prior to Pretest (Study 1)

8.91-(n

TlTH

212 / AMBTMAN ET AL

consistent with the finding that only in grades four and five did a statistical interaction exist between the rural-urban and TlTH Control variables. It is therefore difficult to determine what caused the rural TITH and rural Control discrepancy on the pretest scores.

Study 2 The results of the elementary teacher survey show a total of 208 returned questionnaires, a 41.6 percent return rate. This is a fairly typical percentage for a reasonably long mailed survey without prompts to participate, but is somewhat low compared to the optimum possible return rate of 71 to 88 percent for teachers found by Sudman and Bradburn [26]. The demographics of the sample appeared reasonably representative of the population. Therefore, the survey will be discussed in some detail, bearing the moderate response rate in mind. The majority of respondents (79.3%)indicated they had never used any of the TITH units. This appears to reflect the utilization rate of the program at the time of the survey. AFh4 field workers provided a listing of school divisions or individual schools having been in-serviced on the use of, or using, TITH. This list consisted of a maximum of 173 schools with at least one class covering grades two to six. The provincial listing of schools used for this study listed a total of 610 potential schools (excluding private schools) [27]. In other words, a maximum of 28.4 percent of potential schools were presumably in-serviced. Since not all of these schools were using TITH at the time of the survey, the fact that only one in five respondents had used TlTH appears reasonably representative of the actual situation in the province. Feedback from TlTH users and non-users was qualitatively different because non-users only completed the first ten questions of the survey. Therefore, these two groups will be discussed separately. The non-urers appeared reasonably well distributed, with the various grades one to five teachers ranging from 10.3 to 17.0 percent of the total groups (Table 3). Grade six teachers appeared somewhat under-represented with 7.9 percent of the total. Teachers teaching split grades, a combination of grades or special education classes comprised 23.6 percent of the total. Geographic representation also appeared reasonably proportional to the population, with Winnipeg teachers constituting 56.4 percent, teachers in other schools south of the fifty-third parallel (where most of Manitoba’s rural population is located) 29.6 percent, and teachers north of the fifty-third parallel 7.2 percent. Teachers not using TITH typically had not received a university level course in health education (80.6%).The vast majority of respondents (92.1%) had never taught any programs related to tobacco, alcohol, or other drugs. Nevertheless, most of these teachers indicated that they felt either “comfortable” or “very comfortable” with teaching subjects in the various drug categories. Table 4 shows

DRUG EDUCATION PROGRAM / 213

Table 3. Grade Taught by Use of Tm( (Study 2)

TITH Use Current Grade Taught 2 3 4 5 6 Split/Other TOTAL

TITH Non-Users n 28

%

17.0 16.4 19 11.5 22 13.3 13 7.9 56’ 33.9 165 100.0

27

TITH Users

n 8 9 4 9 7 8 45

% 17.8 20.0 8.9 20.0 15.6 17.8 100.0

Includes grade one teachers.

the comfort level with individual drug categories. Clearly less than 10 percent of respondents indicated being uncomfortable with teaching issues related to alcohol, smoking, prescription and over-the-counter drugs, and caffeine. Slightly over 20 percent indicated some level of discomfort with the topic of street drugs. The non-TITH teachers’ beliefs about when drug education should start in school are reported in Table 5. As can be determined, there are few differences in the means, standard deviations, or ranges for all the various drug categories. Analyses between urban and rural respondents showed only one statistically significant difference. Winnipeg teachers felt that prescription and over-thecounter drugs should be taught at a later age than did non-Winnipeg respondents (means, 4.0 and 3.2, respectively; two-tailed r-test,p c .05). Non-users reported overwhelmingly (84.8% in favor, 11.5% against) that they felt the AFM “should have a role in developing and evaluating school-based alcohol and drug education programs.” Of those against the idea, the majority (58.3%) felt that Manitoba Education should have the primary role in developing alcohol and drug education programs. Teachers using TITH (n = 45) were reasonably well distributed in terms of current grade taught, with the exception of grade four, which appeared somewhat underrepresented (Table 3). In contrast to the non-users, these teachers usually reported having received a university level course on health education (57.5%). Approximately 51 percent worked in Winnipeg and 44 percent in towns south of the fifty-third parallel. Only one teacher north of the fifty-third parallel responded. Findings regarding comfort level with teaching various substance use related materials showed percentages for TITH users of “comfortable” to be slightly higher than those for non-users (Table 4). Although means for all five drug categories were higher than those for non-users, only one drug category

Table 4. Self-Reported Comfort Level of Elementary School Teachers by TITH Use (Study 2)

ug Category

l l T H Use Comkxtabb'

TlTH Non-Users Uncomfortableb Mean Score'

Cornfortabb

TITH Users Uncornbmbb

M

l

93.3

5.4

1.6

97.8

2.2

1.4

ng

93.9

4.2

1.5

97.8

2.2

1.4

ptionlOver-theer Drugs

89.1

9.7

1.7

95.6

4.4

1.4

Drugs

77.6

20.6

1.9

80.0

20.0

1.8

ne

92.1

5.4

1.6

95.6

4.4

1.4

egories "very comfortab~and 'comfortable' com~nedin percentage d total. egories very Uncomfortabwand 'UnComfortaMe' Combined h percentage d total. ge from 1 =very comfortable to4 = veryuncomiortable.

DRUG EDUCATION PROGRAM / 215

Table 5. Teachers' Beliefs about Timing of Drug Education by TlTH Use (Study 2) TITH Non-Users Drug Category

nni use nm Users

Mean'

Range

Meanb

Range

t-Testp.d

Alcohol Smoking Prescription/Over-theCounter Drugs

3.8

(K-10)

3.7

(K-7)

.80

3.0

(K-9)

3.1

(K-6)

.74

3.7

(K-12)

2.9

(K-6)

.06

Street Drugs

4.0

(K-12)

3.4

(K-8)

.11

Caffeine

3.7

(K-13)'

2.9

(K-7)

.05

Note: Means represent average grade level indicated by teachers as appropriate to start teaching the subject. Scale ranged from 'Kindergarten' to 'Never.' a SD = 2.1-2.5

SD = 1.9-2.0 13 = 'never' Two-tailed.

(Prescription and Over-the-counter drugs) showed a statistically significant difference (two-tailed t-test; p c .Ol) between TlTH users and non-users. Also, no statistically significant differences were found between Winnipeg and nonWinnipeg teachers in reported comfort levels. TITH users also showed a tendency to report that they believed drug programs should be taught at an earlier age than did non-TITH users (Table 5). However, the differences in means between TITH users and non-users reached statistical significance (two-tailed t-test; p c .05) for only two drug groups. TITH non-users reported that caffeine and prescription/over-the-counterdrugs should be taught at a later age than TlTH users. No statistically significant differences were found between Winnipeg and non-Winnipeg teachers regarding when the various drug categories should start being taught. As was the case with non-TITH users, the AFM appeared to receive good support for its role in drug education with 86.7 percent of respondents indicating that AFM involvement in curriculum development was appropriate. Responses to questions on in-services received for TlTH revealed that 46.7 percent had received a session provided by the AFM, 33.3 percent by another teacher, and 20 percent no in-service at all. Of those having received an in-service, the majority had received a session of a half-day or less (63.9%).The in-serviced group typically felt the in-service was appropriate in length (88.9%). Other feedback regarding the in-services was generally positive, with all but one respondent agreeing that all topics were covered, 91.7 percent reporting the

216 I AMBTMAN ETAL.

in-service provided enough background to teach the program to their pupils, and 94.4 percent rating the overall quality of the in-service as either “good” or “very good.” There were no statistically significant differences in the way respondents rated workshops by their colleagues compared to those provided by the AFM. Also, no statistically significant differences were found in reported comfort levels with the various drug categories between in-serviced and non-in-serviced teachers. Of all TITH users, approximately one-quarter reported a need for further training. This occurred especially in those who had never received an in-service. Over 57 percent of respondents indicated they felt that an in-service to teach TITH should be mandatory, with approximately 38 percent opposed. Teachers reported that slightly over half of schools (51.1%) held informational sessions on the program prior to it being taught. This occurred especially in urban schools as opposed to rural schools (60.9 vs. 40.9%).All but one respondent indicated that the Manitoba Health Education Curriculum unit on social-emotional well-being was taught in their school. This unit was generally taught before TITH (84.4%), rather than concurrent with it (6.7%). The majority of respondents (84.4%) rated the goals of “H as “very clear,” the remainder as “somewhat clear” or “somewhat unclear.” Slightly fewer (77.8%) felt that instructions to teachers in the TITH manuals were “very clear.” The vast majority (93.3%)judged the program as complete, with no need to add content areas. TITH was reported as either taught in a one- to three-week block (33.3%)or over one to two months (37.8%). Only 17.8 percent of respondents reported teaching a unit over three or more months. Lessons ranged typically from sixteen to forty minutes (75.6%). These findings were consistent with reports from teachers participating in Study 1. Numbers of respondents providing feedback on separate units of TITH ranged from ten to eighteen per unit. Table 6 summarizes their responses. Generally, the length of the units was seen as “right.”The overall quality of the various units was typically rated as “good” or “very good.” The reading level of the units was generally judged as appropriate with a slight minority of respondents judging it “diffcul t.” Lessons within units were typically viewed as presented in a logical sequence. Students’ reactions to activities and information in the program were typically judged as positive, especially for the lower grades. The unit on smoking appeared to stand out somewhat in the above ratings. Although it received generally favorable ratings, these ratings were not as unanimously positive as for most of the other units. There was somewhat of a tendency to rate the units as ‘‘too long” and it was the only unit to receive one rating each of “very poor” and “poor.” This contrasts with students’ reactions to the program, which were generally seen as very favorable.

nit

Table 6. Reported Use and Teachers' Ratings of TITH Units

Aspects of TlTH Units Reported Use of UniP

ous s

48.9%

es

37.8%

bnsth Right

Overall Qualityb

Good (1.4)

L@al Sequence be4

Reading bveF

Student Reacbn to R ~ c t i ~ i t i t 3 ~ In ~

Appropriate (6.3%)

All

Very Positive (1.6)

Very (1.7

Right (leaning to too short)

Good

Appropriate (17.6%)

All

(1.3)

Very Positive (15)

Very (1-5

nd ns

31.1%

Right

Very Good (1 5)

Appropriate (25.0%)

All

Positive (1.3)

Very (1 *6

nd es

26.7%

Right

Good

Appropriate (15.4%)

All

Positive (1*3)

Very (1-5

g

28.9%

Appropriate (15.4%)

91.7%

Very Positive (1-5)

Very (1-5

(1 4

Right (leaning to too long)

Good (0.8) (with some mixed)

all answered all questions. all quality: very g00d = 2 points; good = 1 point; poor = -1 point; very poor = -2 points; rims in parrepresent mean respondents judged reading level appropriate. Fgures in parrepresent percentagesof respondents rating text as 'd 2-16 respondents); i.e.. the higher the percentage the more difficult the text. positive = 2 points; positii = 1 point; negative = -1 point; very negative = -2 points; Figures in parentheses represent means

218 I AMBTMAN ET AL.

Table 7. Numbers of Lessons Reported Taught, with ModificationRates (Study 2) Lesson Number Unit

7

2

3

4

5

6

7

Hazardous Products (G2)

206

2ob

206

1d'

1?

N/A

Medicines (G3)

16'

1s"

1%

N/A

N/A

Drugs (L Decisions (G4)

1s" 13'

19d 1 4 '

126

11'

128

N/A

Drugs & Influences (G5)

128

13'

1 9

13d

136 136 1 9 lld

Smoking (G6)

158

15'

15'

1 9 15'

lld 14b 1 4 '

Note: N/A = Number of lessons for each of the five units range from five to seven. Modification Rate = a Less than 40 percent. 40 to 50 percent. 51 to 60 percent. *over 60 percent.

Responses to specific lessons indicated that teachers regularly modified lessons. Table 7 lists both the numbers of lessons taught and the modification rate. It is notable that the grade five unit is the most modified, units for grades three and four the least. It can also be observed that lessons toward the end of a unit are less often taught than those in the beginning. The latter finding is also supportedby feedback received from teachers participating in Study 1.

DISCUSSION In reviewing both studies, it must be noted that the student study can be interpreted with much greater confidence than the teacher survey. The former study had a much higher return rate and occurred under more controlled circumstances than the latter. Conclusions from the teacher study must therefore be interpreted with considerably more caution. In the teacher survey several issues of note stand out. The first of these is the observation that teachers rate the various aspects of TITH highly. Length, overall quality, reading level, sequence, and student reactions both to activities and to the information of each unit were judged very positively. This suggests that the program is well received among its users. Second, judging from the teacher survey and other indications, the program was, at the time of the survey (May 1988), in use in 20 to 30 percent of Manitoba public schools. Given that not all classes in every school would be using TITH yet, the lower percentage is probably closer to the actual penetration rate into class-

rooms.

DRUG EDUCATlON PROGRAM / 219

Third, teachers reported that they generally felt quite comfortable with teaching drug related issues. The topic of street drugs stood out. One in five teachers reported not feeling comfortable with this subject. Only one difference, for prescription drugs, in self reported comfort levels between TITH users and non-users reached statistical significance. Taken at face value, this would suggest that teaching TITH does not reduce a teacher’s level of comfort with the topic, except for prescription drugs. Finding the latter difference is more in line with expectations (i.e., that exposure reduces apprehension) than the finding for the other drug categories. For example, Gamey showed that, for a small sample of teachers, those in-serviced reported greater comfort level with TITH materials than no in-service controls [B]. Since the present survey did not support this latter finding, it must be concluded that neither participating in an in-service nor teaching TITH has much effect on self-reported level of comfort. The finding that one in five teachers taught TITH without ever having received an in-service may be of some concern. That is, if this is a typical pattern, a substantial number of teachers will be teaching TITH while left to their own devices. It is questionable whether these individuals ever receive a full understanding of the principles of the program or teach it as required [19,21].Certainly, they will not have the benefit to discuss concerns and questions they may have about the program in a structured setting. Moreover, less than half of the respondents indicated that they had been in-serviced by an AFM staff member. Although it is presumably less costly to have teachers in-service other teachers, it is less certain whether this approach is beneficial in terms of quality control. Unfortunately, the survey is inconclusive in this respect. A final issue, also related to quality control, is that of modification rates of the various units. Teachers indicated quite substantial modifications of various lessons. Given their high ratings of the various units, this cannot be interpreted to mean that they changed lessons because they had problems with the materials. Rather the high modification rate is more likely indicative of a creative use of the materials. This conclusion would be supported by the observation that earlier lessons were less likely to be modified. In other words, once a teacher has provided students with an introduction to the unit, and has become somewhat familiar with the materials, the teacher is more likely to make changes in subsequent units. Of greater concern is the tendency reported by teachers in both studies to teach later lessons in a unit less often, compared to earlier lessons. This may be a function of starting the program too late in the school year and not being able to finish it in time. On the other hand it may be that later units were perceived as less relevant and were consequently dropped in favor of other materials. Given that it was observed in the student outcome study that most schools tended to teachTITH toward the latter part of the school year, the former conclusion (of a shortage of time) seems the more likely. If this holds true, the most important information should be presented in earlier units.

220 / AMBTMANETAL

Given the relative diversity of findings between rural and Winnipeg schools, the student-outcomesurvey will be discussed under two separate headings. Interpretive concerns are discussed for Winnipeg schools, but hold equally well for rural areas.

Winnipeg Schools Winnipeg experimental schools clearly made greater gains on the questionnaires than their control counterparts. Since there were few other differences between experimental and control schools, this may be interpreted to mean that the TITH program had a definite impact when taught to grades two through six in urban schools. Although actual gains appear small, two factors warrant consideration. First, these gains are of a similar magnitude reported by others (e.g., [Z]).Second, they must be interpreted against a baseline phenomenon evident in the results. That is, a substantial base of knowledge existed in students prior to being taught TITH. As mentioned above, the pretest averages across all groups corresponded with percentage scores ranging from 60.6 to 78.8 percent for the five grades. Even with a guessing factor taken into account, this constituted a fairly substantial knowledge base. With such high initial scores, it would be much more difficult to make significant gains. In other words, low percentage improvementsare much more impressive when the baseline is already quite high, as is the case in the present study. The observation that the baseline is quite high has other implications for the interpretation of findings. The most serious of these is that each TITH unit appears to attempt to teach a significant amount of knowledge which students already have. Although it may be that the program reinforces existing knowledge, it does appear to point to a weakness in TlTH, in that the program currently seems to underestimate the basic knowledge of a significant section of its intended audience. In fact, TITH appeared to mostly improve scores of those students who had the least knowledge. For those who already had a substantial base knowledge, the program did very little in most grades. For example, for the eighty-seven students in grade six who scored 40 or more on the pretest (out of a possible 52), thirty-nine received the same or a worse score on the posttest and forty-eight gained in score. While no question on baseline knowledge was asked in the independent survey of elementary teachers, Winnipeg respondents appeared quite positive about the student response to both activities and information of each of the five grade levels. This suggests that the program certainly is congruent with the opinions and experiences of teachers using TITH. In other words, there was no indication that students were bored with the materials given the high level of initial knowledge. Nevertheless, this high initial knowledge level may pose a problem for the program. The significant inverse relationship between the CHANGE score and the pretests suggests that results (i.e., knowledge gain) could likely be increased if TITH were used with students who had a lower baseline knowledge.

DRUG EDUCATION PROGRAM / 221

Operationally, this may be accomplished by lowering the grade level for each unit. For example, “Medicines” could be taught in grade one, and so forth. However, there may be difficulty with simply grafting the materials in their present form to a lower grade curriculum, given that the reading level of the materials was carefully calibrated for each separate grade. The latter was confirmed by the teacher survey. Teachers judged the materials to be “appropriate” for the reading level of each grade. This would suggest that the materials were properly written for the various target groups. Nevertheless, it appears feasible to teach the various topics of each TITH unit at an earlier grade. The surveyed teachers, both those using TITH and those not using the program, indicated that some chemical/drug topics could be taught at a much earlier age. Respondents generally appeared to indicate that most chemical/drug related topics could be taught in grades three or four. If any units were to be revised, the teaching order of when each drug category is taught is of some interest. For example, teachers judged smoking to be an issue which should be started relatively early (i.e., early grade three), while this is currently the grade six unit in TITH. To ameliorate this problem, the data point to another alternative. That is, given that TITH seems to work best with students starting with a low base level of knowledge, it may be possible to develop a greater range of materials within each existing package, targeting those students who have more initial knowledge. A final comment on the interpretation of the high baseline is that pre-/postquestionnaires were administered in the second half of the school year. This timing was chosen since TITH units are typically taught in the latter part of the school year. It would therefore likely be possible to optimize the effectiveness of the program if it were used earlier in the school year. Whether this is a feasible recommendation to educators is not known at the present time. Certainly it is not something over which the AFM will have much control. Moreover, the objective of drug education is to have students acquire certain knowledge and behaviors. Whether this results from TITH or some other source is relatively immaterial.

Rural Schools

As indicated above, interpretation of the data for rural schools must be preceded with a note of caution, given that several control groups did not appear equivalent based on the pretest variation between grades two, three, and six experimental and control schools. From a face value examination of the listing of rural schools, it became apparent that the experimental and control schools differed in some ways. The control schools appeared to be somewhat smaller than the experimental schools (mean number of teachers 13.7 versus 20.9, respectively), and the range of grades seemed slightly broader [27]. That is, three out of nine experimental schools had

222 I AMBTMAN ETAL.

only kindergarten to grade six, while only one of the control schools did so (the other schools ranged from K-8 to K-12). In addition, the experimental schools had fewer northern schools than the controls (1/9 vs. 2/9,respectively). Moreover, three of the nine experimentalschoolswere located in a town which, although technically “rural,” is by many considered an extension of Winnipeg given its almost suburban location. Generally, the rural control schools were located in somewhat smaller towns than the experimentalschools (means, 3500 and 4441,respectively). The overall impression in the differences between experimental and control schools was that the control schools were more stereotypic “rural-Northern” than experimental schools. The concomitant greater isolation of the schools and its pupils may underlie the lesser knowledge demonstrated in the pretest. In other words, it may be that rural experimental schools were more similar in nature to urban schools than to the rural controls. This could explain why more differences were found in the pretest between rural control and rural experimental schools than between their urban counterparts. One other possible explanation of the higher initial baseline knowledge level of rural TlTH schools exists. The fact that TlTH was used in these schools may be reflective of a greater concern regarding drug issues in the communities where these schools were located, compared to a lesser concern in communities without TlTH schools. In the former communities, drugs would be more of a topic of interest and consequentlystudents would have more exposure to related information. If these initial differences are considered random (i.e., non-systematic) variations and only the CHANGE score for each grade level is taken into consideration, then it can be concluded that grades, three, four, and five of experimental groups showed significant improvements in scores as a result of exposure to TITH, when compared with controls. Change was especially dramatic in grade four, which showed an increase even greater than the Winnipeg TITH schools. Grades two and six showed significant differences only between experimental and controls on the CHANGE score if the pre-score was taken into account. Again this may be partly attributable to the ceiling effect of the high baseline. Nevertheless, the fact remains that this high initial rate does not prevent urban schools from gaining statistically significant differences. In other words, although the apparent greater isolation of the control schools may have explained the initial lag, the fact that for grades two and six no significant differences in change scores without covariate were found suggests that the experimental schools at these grade levels may not fully exploit their inherent advantage.

Conclusion In summary, the first step in reaching all Manitoba’s children with a drug education program was successfully accomplished. TITH is a program recommended by the provincial education body and has been implemented on a long

DRUQ EDUCATION PROGRAM / 223

term basis by a wide range of public school divisions across Manitoba. Thus, 'IITH has accomplished one of the key factors required for effectiveness as a prevention program: long-term education. When fully implemented, most students in Manitoba will receive at least a five-year program (grades two through six) and likely an additional three years (Junior High, grades seven through nine). Moreover, teachers will gain increasing exposure to the program, improving their effectiveness [18]. Overall, TITH appeared to be a very well received program among teachers from a consumer point of view. With some exceptions, it also appeared to have a definite influence on the knowledge level of students in the area of chemical use and the health and social context of this use. A number of recommendations were made by the researchers to program management, with the aim of increasing the effectiveness of TITH. ACKNOWLEDGMENTS

The authors gratefully acknowledge the assistance of Dr. James Walker in the data collection, Shirley Guim in the literature survey, Carol Atkinson in the preparation of materials and the manuscript, Carol Gieni in the instrument construction, Brian Broszeit in the data analyses, and members of the permanent staff of the Planning and Research Directorate of the Alcoholism Foundation of Manitoba for their advice and support in this study. The help of the Manitoba Teachers' Society in consultations regarding the teacher survey and the data collection of the survey was high appreciated. REFERENCES 1. G. G. Milgram, Alcohol and Drug Education Programs, Journal of Drug Education, 17, pp. 43-55, 1987. 2. G . J. Botvin, Substance Abuse Prevention Research: Recent Developments and Future. Directions, Journal of School Health, 56, pp. 369-374,1986. 3. J. M. Moskowitz, J. H. Malvin, G. A. Schaeffer, and E. Schaps, An Experimental Evaluation of a Drug Education Course, Journal of Drug Education, 14, pp. 9-22, 1984. 4. M. L. Thompson, R. Daugherty, and V. Carver, Alcohol Education in Schools: Toward a Lifestyle Risk-Reduction Approach, Journal of School Health, 54, pp. 7943,1984. 5. G. M. Gonzalez, Alcohol Education Can Prevent Alcohol Problems: A Summary of Some Unique Research Findings, Journal of Alcohol and Drug Education, 273, pp. 2-12,1982. 6. National Institute on Drug Abuse, Fifth Special Report fo fhe U. S. Congress on Alcohol and Healfh, U . S . Department of Health and Human Services, Rockville, Massachusetts, 1983.

224 J AMBTMAN E T A L

7. R. G. Smart and A. C. Ogborne, Northern Spirits Drinking in Canada: Then andNow, Addiction Research Foundation, Toronto, Ontario, Canada, 1986. 8. R. L Bangert-Drowns, The Effects of School Based Substance Abuse Education: A Meta-Analysis, Journal of Drug Education, 18, pp. 243-264.1988. 9. M. S. Goodstadt, School-Based Drug Education in North America: What Is Wrong? What Can Be Done?, Journal of School Health, 56, pp. 578-581,1986. 10. E. Schaps, R. DiBartolo, J. Moskowitz, C. S. Palley, and S. Churgin, A Review of 127 Drug Abuse Prevention Program Evaluations, Journal of Drug Issues, 9, pp. 17-43, 1981. 11. D. J. Hanson, The Effectiveness of Alcohol and Drug Education, Journal ofAlcohol and Drug Education, 27, pp. 1-13,1982. 12. R. A. Weisheit, The Social Context of Alcohol and Drug Education: Implications for Program Evaluations, Journal of Alcohol and Drug Education, 29, pp. 72-81, 1983. 13. B. Flay and J. Sobel, The Role of Mass Media in Preventing Adolescent.Substance Abuse, in Preventing Adolescent Drug Abuse: Intervention Strategies, T. J. Glynn, G. G. Leukefeld, and J. P. Ludford (eds.), U.S. Government Printing Office, Superintendent of Documents, Washington, D.C., (NIIDA Research Monograph No. 47,DHEW Pub. NO.(ADM) 83-1280).1983. 14. M. A. Sheppard, M. S. Goodstadt, and B. Williamson, Drug Education: Why We Have So Little Impact, Journal of Drug Education, 15, pp. 14,1985. 15. H. Swadi and H. Zeitlin, Drug Education to School Children: Does It Really Work?, British Journal OfAddiction, 82, pp. 741-746,1987. 16. D. B. Connell, R. R. Turner, and E. F. Mason, Summary of Findings of the School Health Education Evaluation: Health Promotion, Effectiveness, Implementation, and Costs, Journal of School Health, 55, pp. 316-321,1985. 17. W. J. Gunn, D. C. Iverson, and M. Katz, Design of the School Health Education Evaluation,Journal of School Health, 55, pp. 301-304,1985. 18. D. B. Connell and R. R. Turner, The Impact of Instructional Experience and the Effects of Cumulative Instruction, Journal of School Health, 55, pp. 324-331,1985. 19. L. DiCicco, R. Biron, J. Carifio, C. Deutsch, D. J. Mills, A. Orenstein, R. H. Unterberger, and R. E. White. Evaluation of the CASPAR Alcohol Education Curriculum, Journal of Studies on Alcohol, 45, pp. 160-169,1984. 20. M. S.Goodstadt, M. A. Sheppard, and G. C. Chan, An Evaluation of Two SchoolBased Alcohol Education Programs, Journal of Studies on Alcohol, 43, pp. 352-369, 1982. 21. S.W. Fors and M. E. Dorster, Implication of Results: Factors for Success, Journal of School Health, 55, pp. 332-334,1985. 22. D. K. Lohrmann and S. W. Fors, Can School-Based Educational Programs Really Be Expected to Solve the Adolescent Drug Abuse Problem?, Journal of Drug Education, 16, pp. 327-339,1986. 23. M. J. Strople and D. Koss,Tuning In To Health: Alcohol and Other Drug Decisions, Alcoholism Foundation of Manitoba, Winnipeg, Manitoba, Canada, 1986. 24. R. Saranchuk and D. Koss,Tuning In To Hea1th:Alcohol and Other Drug Decision for Junior High Students, Alcoholism Foundation of Manitoba, Winnipeg, Manitoba, Canada, 1986.

DRUG EDUCATION PROGRAM / 225

25. A. L Kearney and M. H. Hines, Evaluation of the Effectiveness of a Drug Prevention Education Program, Journal ofDrug Education, 10, pp. 127-134,1980. 26. S. Sudman and N. Bradbum, Improving Mailed Questionnaire Design, in Making Effective Use of Mailed Questionnaires, D. C . Lockhart (ed.), New Directions for Program Evaluation, No. 21, Jossey-Base, San Francisco, 1984. 27. Manitoba Education, Schools in ManitobalEcoles du Manitoba, Manitoba Education Communications Branch, Winnipeg, Manitoba, Canada, 1986-87. 28. D. L.Gamey, The Effectivenessof the Concerns-BasedAdoptionModel in Implementing a Drug hevention Program, unpublished master’s thesis, University of Manitoba, Winnipeg, Manitoba, Canada, 1985.

Direct reprint requests to:

Rudy Ambtman Director of Clinical Programs Manitoba Health Mental Health Division 305-1200Portage Avenue Winnipeg MB R3G OT5 Canada