AJPH RESEARCH
Replicating the Safer Sex Intervention: 9-Month Impact Findings of a Randomized Controlled Trial Meredith Kelsey, PhD, Jessica T. Walker, PhD, Jean Layzer, MEd, Cristofer Price, ScM, and Randall Juras, PhD Objectives. To test the effects of the Safer Sex Intervention (SSI) on female adolescents’ sexual behavior and possible antecedents of behavior such as sexual health attitudes, knowledge, motivation, intentions, and skills. Methods. A randomized controlled trial compared SSI (n = 1196) with no intervention (n = 613) among female adolescents aged 13 to 20 years at 3 sites across the United States from 2012 to 2015. Intent-to-treat impacts were estimated at 9 months after baseline, overall, and for key subgroups. Results. Compared with control participants, SSI participants were less likely to have sexual intercourse without birth control, more likely to report positive attitudes toward protection and intention to use condoms, and more confident of their ability to refuse sex. SSI did not affect sexual risk knowledge or motivation to delay childbearing. Positive impacts on sexual behavior and sexual risk were observed among key subgroups of youths who were aged 18 years or older, Hispanic, not sexually experienced at baseline, and enrolled at the Minnesota site. Conclusions. SSI produced meaningful changes in sexual behavior and sexual risk and successfully addressed some potential antecedents of sexual risk behavior. (Am J Public Health. 2016;106:S53–S59. doi:10.2105/AJPH.2016.303372) See editorials, p. S5–S31.
I
n the United States, pregnancy occurs at a rate of 57.4 per 1000 female adolescents, and 1 in 4 sexually active female adolescents has a sexually transmitted infection (STI).1,2 Both outcomes can negatively affect youths’ well-being and future prospects. For this reason, preventing pregnancy and STIs among adolescents is a top public health priority— one in which the federal government has invested heavily, specifically through the Office of Adolescent Health’s Teen Pregnancy Prevention Program.3 The Safer Sex Intervention (SSI) is an evidence-based intervention being replicated by Teen Pregnancy Prevention Program grantees, and it is intended to reduce the incidence of STIs and increase condom use among young sexually active female adolescents. Previous findings from a study of SSI, conducted by the developer in the 1990s in a Boston, Massachusetts, hospital with young women being treated for an STI, indicated that the program reduced their likelihood of having multiple sexual partners
Supplement 1, 2016, Vol 106, No. S1
AJPH
and provided evidence that SSI improved knowledge of sexual risks, attitudes toward and intentions around condom use, and use of condoms.4 In 2010, the Office of Adolescent Health awarded 5-year grants to 75 grantees to implement evidence-based programs intended to reduce adolescent pregnancy. Of the 28 evidence-based adolescent pregnancy prevention programs that grantees might choose to replicate, SSI was the only one designed for a clinic setting and one of the few that adopted an individualized approach to the challenge of changing sexual behavior among adolescents. Given the shortage of evidence-based alternatives for clinic settings and individualized programming, understanding SSI’s
effectiveness is important to both practitioners and policymakers. In this study, we examined whether SSI is effective when replicated in a larger number of clinic settings, with different and more diverse populations, and in 3 geographic locations. Although the program was designed for sexually experienced young women, program staff believed the program would also be relevant for young women who were sexually inexperienced but seeking reproductive health services, and a small number of such youths were included in the present study. We tested whether SSI has positive effects on sexual behaviors that could lead to pregnancy or STI and such antecedents of behavior as attitudes, knowledge, motivation, intentions, and skills. We present findings after 9 months, focusing on the following research questions: 1. Did SSI reduce sexual behavior and risky sexual activity? 2. Did SSI have positive effects on intermediate outcomes thought to lead to behavior change (attitudes toward protection and risky behavior, knowledge of pregnancy risk and STI risk, motivation to delay childbearing, intentions to avoid sexual activity and risky sexual behavior, refusal skills, and condom negotiation skills)?
METHODS The study features an experimental design in which female adolescents were randomly assigned to a group that received SSI or to
ABOUT THE AUTHORS Meredith Kelsey, Jessica T. Walker, Cristofer Price, and Randall Juras are with Abt Associates, Inc., Cambridge, MA. Jean Layzer is with Belmont Research Associates, Belmont, MA. Correspondence should be sent to Meredith Kelsey, PhD, 55 Wheeler Street, Cambridge, MA 02138 (e-mail: Meredith.kelsey@ abtassoc.com). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted July 4, 2016. doi: 10.2105/AJPH.2016.303372
Kelsey et al.
Peer Reviewed
Research
S53
AJPH RESEARCH
a control group that did not. The clinical trial registration number is NCT02540304.
Safer Sex Intervention SSI is a clinic-based, sexual health intervention for female adolescents delivered in one-on-one, face-to-face sessions with a female health educator. The intervention is individually tailored, includes medically accurate information and discussion about safe sexual behaviors, emphasizes skills to protect against risk, and promotes self-efficacy and selfesteem. In the 50- to 60-minute initial session, the health educator guides the participant through a sequence of topics, allowing time for role play and other activities. Topics include the consequences of unprotected sex, risk perception, preventing pregnancy and STIs, condoms, where to obtain condoms, secondary abstinence, and talking about sex. Three subsequent 15- to 30-minute booster sessions, similar in content, are delivered 1, 3, and 6 months after the initial session. The theoretical framework for the intervention draws on social cognitive theory, the transtheoretical model of behavior change, and the motivational interviewing technique.4 The health educator helps the adolescent identify her needs, motivations, and intentions; gradually identify obstacles to behavior change; and make plans to address them. Through subsequent booster sessions, the health educator tracks the participant’s progress through the stages of change. Figure 1 shows the program elements, hypothesized outcomes, and pathways by which SSI seeks to achieve these outcomes.
Setting Replication sites were selected for the study from the small number of grantees that received funding to implement SSI. Three grantees, located in Minnesota, Tennessee, and Florida, were willing to participate and able to meet the requirements of a large-scale rigorous study. Across the 3 sites, SSI was implemented in 38 clinics by trained female health educators. Health educators were given dedicated space within the clinic, and clinicians identified and referred eligible female adolescents to them.
Training, Preparation, and Fidelity Health educators and supervisory staff from all 3 sites attended a 3-day training
S54
Research
Peer Reviewed
Kelsey et al.
provided by the program developer. The training introduced staff to the intervention and how to deliver it with fidelity and focused on motivational interviewing. Supervisory staff received additional training and were certified as trainers to train replacement staff and provide refresher trainings to all staff. All 3 sites provided ongoing training and practice in motivational interviewing. Each site was required to implement the program with fidelity, and fidelity was assessed, monitored, and reported to OAH at regular intervals by program staff. To measure fidelity, OAH provided fidelity checklists and required that they be completed for every session of the program. These data indicate that each site delivered the intervention to youths with fidelity to the core elements of the program model.
Sample Eligibility criteria for the program (and study) were based on gender (female), age (13– 20 years), sexual debut (sexually active or planning to be), and pregnancy status (pregnant adolescents were not eligible but could participate postpartum). Participants were enrolled over a 19-month period, beginning in July 2012. Across all 3 sites, a total of 2108 youths consented, completed a baseline survey, and were randomized (treatment group, n = 1403; control group, n = 705).
program. To protect its integrity, the randomization process was designed and implemented by an independent study team. Participants assigned to the treatment group received SSI. Those assigned to the control group received the usual standard of care offered in the clinic, including medical services sought and contraceptive information. Both groups could receive nonprogram services and informational materials offered by the clinic or available in the community. Clinics varied in the level of contraceptive information offered to members of the control group as part of their usual standard of care.
Data Collection Participants were surveyed at baseline (July 2012–February 2014) and 9 months later (April 2013–March 2015). A Web-based system was used to capture and store survey responses. The baseline survey was completed individually at the clinic on a dedicated computer. At follow-up, participants were e-mailed a unique link to the survey, which they completed independently in any convenient location, using a computer or mobile device. Of the 2108 eligible participants, 86% (n = 1809) completed a 9-month follow-up survey, forming the analytic sample. The 9-month completion rates were very similar across sites and across the study groups (treatment group, n = 1196; control group, n = 613).
Enrollment and Random Assignment
Measures
Clinic staff used the eligibility criteria to identify potential study participants when they presented for clinic services. Youths were then referred to a health educator, who confirmed eligibility, described the study, obtained informed consent, and facilitated all study enrollment activities. Parental consent was waived for unaccompanied minors who sought sexual health services at the clinic. Once youths consented, they completed a self-administered baseline survey before being randomly assigned. Random assignment occurred independently in each of the clinics with a treatment-to-control ratio of approximately 2:1. Individuals within clinics were randomly assigned within randomization blocks on the basis of age to address potential clinic-level variation in participant characteristics and program implementation and potential age-related differences in responses to the
Participants were asked questions that allowed us to measure 2 sets of outcomes: (1) behavioral outcomes (i.e., measures of sexual behavior and sexual risk) and (2) intermediate outcomes (i.e., factors believed to be antecedents of behavioral outcomes). They were also asked about their backgrounds, including race and ethnicity. Because of welldocumented racial/ethnic disparities in rates of adolescent pregnancy and STIs and variation in race/ethnicity across the 3 sites, we examined racial/ethnic differences in program impacts. We briefly describe these measures here (for additional details, see Appendix A, available as a supplement to the online version of this article at http://www.ajph.org). Behavioral outcomes. To address the study’s key questions about the impact of SSI on sexual behavior and sexual risk, we identified 10 measures:
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
Inputs
Process
Intermediate Outcomes
Outcomes
Trained female health educator
Provide medically accurate information on sexual risk behavior and consequences
Positive attitudes toward protection Improved knowledge and understanding
Safer sexual behavior
One-on-one individualized face-to-face sessions
P Positive, trusting relationship
Motivational interview
Facilitate selfassessment of motivational status
Support for motivational change process
Improved motivation/intention to avoid sexual risk
Consistent effective condom use Consistent use of contraceptives Abstinence
Demonstrate condom use and teach negotiation/refusal skills
Role-play, practice to improve skills
Improved negotiation/refusal /condom use skills
Three “booster” sessions
Reduction in sexual activity Reduction in no. of sexual partners
Reduction in STIs Reduction in unwanted pregnancies Reduction in adolescent births
Access to condoms and safer sex info
Note. STI = sexually transmitted infection.
FIGURE 1—Logic model for the Safer Sex Intervention
1. currently sexually active (in the past 90 days), 2. sexual intercourse without birth control in the past 90 days (sexual intercourse defined on the survey as “a male putting his penis into a female’s vagina”; birth control defined as a condom or other form of contraceptive), 3. sexual intercourse in the past 90 days, 4. oral sex in the past 90 days, 5. anal sex in the past 90 days, 6. sexual intercourse without a condom in the past 90 days, 7. oral sex without a condom in the past 90 days, 8. anal sex without a condom in the past 90 days, 9. sexual intercourse with more than 1 partner (lifetime), and 10. sexual intercourse with more than 5 partners (lifetime). For the latter 2 measures, participants reported the actual number of lifetime partners, which we recoded to minimize the
Supplement 1, 2016, Vol 106, No. S1
AJPH
influence of outliers. Otherwise, questions about behaviors were framed as yes–no questions. Responses for each behavior were coded on a scale on which 0 = no and 1 = yes. Intermediate outcomes. We constructed measures to assess the intermediate outcomes (those that may precede behavioral outcomes). A list of the items composing each scale, coding details, and additional information on reliability are provided in Appendix A. Attitudes toward protection were measured on a scale ranging from 1 to 4, with higher scores indicating more positive attitudes toward the use of condoms and other contraceptive methods (12 items; a = .78). Attitudes toward risky behavior were measured on a scale ranging from 0 to 1 and reflected the proportion of statements the respondent agreed with about the acceptability of risky sexual behavior (higher scores represent greater support for risky behavior; 7 items; a = .79). Knowledge of pregnancy risk was measured on a scale ranging from 0 to
1 and reflected the proportion of items about pregnancy risk factors answered correctly (4 items; a = .54). The knowledge of STI risk scale ranged from 0 to 1 and reflected the proportion of items about STI risk factors answered correctly (12 items; a = .68). Motivation to delay childbearing was measured on a scale ranging from 1 to 4, with higher scores indicating greater motivation (3 items; a = .88). Condom negotiation skills were measured on a scale ranging from 1 to 4, with higher scores indicating stronger ability to negotiate condom use (7 items; a = .84). Refusal skills were measured on a scale ranging from 1 to 4, with higher scores indicating stronger ability to abstain from sexual behavior (6 items; a = .83). To assess intentions to engage in sexual behaviors and use protection, we examined 4 dichotomous items separately. Race/ethnicity. To measure race/ethnicity, we asked participants to identify their race from among 6 major categories and separately
Kelsey et al.
Peer Reviewed
Research
S55
AJPH RESEARCH
indicate whether they were of Hispanic descent. We then classified participants as Hispanic, non-Hispanic White, non-Hispanic Black, or non-Hispanic other race/ethnicity.
Analytic Approach To test the impact of SSI, we compared outcomes for the treatment and control groups using a regression framework. For continuous outcomes, we ran traditional regression models; for dichotomous outcomes, we ran linear probability models. We included baseline covariates (e.g., demographic variables, risk behaviors such as baseline sexual activity and substance use), preintervention measure of the outcome variable when available (available for all except attitudes toward risky behavior, refusal skills, and condom negotiation skills), and a series of random assignment block indicators to increase statistical precision of the impact estimates for a given sample size5 and reduce attrition bias from missing data.6 To maintain the integrity of the random assignment design, we conducted an intent-to-treat analysis, using assignment status as the key variable of interest. Data for the impact estimates were pooled across the 3 sites. To address the few instances of missing outcome data, we used case deletion and a dummy-variable adjustment in regression models to account for missing covariates.6 To address multiple comparisons problems (i.e., to reduce the chance of spurious findings from a large number of tests), before analyzing data we prespecified 2 primary or confirmatory outcomes in our analysis plan as the main indicators of program effectiveness: sexual activity in the past 90 days and sexual intercourse without birth control in the past 90 days. To control for having 2 confirmatory outcomes, we applied a formal Benjamini-Hochberg correction, which controls for the false positive rate by adjusting P-value thresholds.7 We considered all other outcomes to be secondary or exploratory; thus, they did not warrant a multiple comparisons adjustment.8 Our main analysis included study participants who were sexually experienced at baseline and participants who were not. As part of our exploratory analyses and to better understand what works for whom, we estimated impacts for key subgroups (site, age,
S56
Research
Peer Reviewed
Kelsey et al.
race/ethnicity, sexual experience at baseline) and tested for differences among subgroups. We implemented subgroup analyses by including subgroup indicators and treatment · subgroup interaction terms in the model and testing for significance of the interaction term. We only present impact estimates for individual subgroups when there was a statistically significant difference between subgroups, which helps guard against overinterpretation of potentially spurious findings across the large number of tests. Because subgroup analyses were exploratory, we did not adjust for multiple comparisons.
RESULTS At baseline, the study sample was on average aged 17 years and diverse in race/ ethnicity: 18% Hispanic, 36% non-Hispanic Black, 33% non-Hispanic White, and 13% non-Hispanic other race/ethnicity. Threequarters lived with 1 or both biological parents. Ninety-four percent had ever been sexually active. Three-quarters had ever used alcohol, two-thirds had ever used marijuana, and half had ever smoked cigarettes. Baseline equivalence was assessed using models with the same structural components as the impact model (i.e., randomization block indicators and treatment group indicator). At baseline, no significant treatment–control group differences were found (Table 1).
intercourse, oral sex, or anal sex, nor did it affect the likelihood of having sexual intercourse with more than 1 partner or more than 5 partners in one’s lifetime. We found some statistically significant differences in behavioral impacts between sexually experienced study participants and the relatively small group who were sexually inexperienced at baseline (Table A, available as a supplement to the online version of this article at http://www.ajph.org). SSI reduced the likelihood of having sexual intercourse with more than 1 lifetime partner by an average of 21% points (70%) for participants who were sexually inexperienced at baseline. Although the numbers affected are small (at baseline, 6% of participants reported never having been sexually active), the impact is large, meaningful, and significantly different than the negligible impact on sexually experienced youths. Among Hispanic youths, SSI significantly decreased the likelihood of having more than 1 lifetime sexual partner by 10 percentage points (13%), on average. We found no impacts on this outcome for any other racial/ethnic group. Among youths aged 18 years and older, SSI significantly decreased rates of engaging in oral sex without a condom by 7 percentage points (11%) on average. We found no impacts on this outcome for younger youths. The impacts on sexual intercourse in the past 90 days also differed significantly between the 2 age groups; however, in neither did the effect reach a conventional level of statistical significance.
Program Impacts on Behavior We observed a large positive impact on 1 of 2 primary outcomes: sexual intercourse without birth control (Table 2). On average, SSI participants were 6 percentage points (21%) less likely to report engaging in sexual intercourse without birth control than were the control group. This impact was significant even after applying a multiple comparisons correction. However, participants in the treatment group were no less likely to report being sexually active in the past 90 days than were their control group counterparts. SSI had no impact on any other measures of sexual activity. The program was not effective in reducing sexual intercourse, oral sex, or anal sex in the past 90 days. It did not affect rates of condom use during sexual
Program Impacts on Intermediate Outcomes Although SSI had no statistically significant effect on knowledge or motivation, it positively and significantly affected some attitudes, intentions, and skills (Table 3). Because no formal statistical adjustments were made to account for the substantial number of secondary outcomes, we would expect some significant findings to arise purely as a result of chance. These results should therefore be viewed cautiously and interpreted in the context of the totality of evidence. SSI had a small but significant impact on attitudes toward using protection (birth control and condoms). Although both groups expressed positive attitudes toward protection, on average the treatment
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
TABLE 1—Baseline Characteristics for the Analytic Sample: Safer Sex Intervention; Minnesota, Tennessee, and Florida; July 2012–February 2014 Demographic Characteristic and Outcomes
Rangea
No.
Treatment Meanb
Control Mean
Group Differencec
P
Age
13–20
1809
17.12
17.14
–0.01
.79
Race/ethnicity, %d,e Hispanic
1809
16.80
19.90
–3.10
.09
Non-Hispanic Black
1809
35.77
35.24
0.53
.81
Non-Hispanic White
1809
33.58
31.65
1.93
.34
Non-Hispanic other
1809
13.85
13.21
0.64
.70
e
Family structure and relationships, % Lives with biological parents
1790
78.97
78.02
0.96
.63
Feels very close to and cared for by father
1603
29.58
26.07
3.51
.14
Feels very close to and cared for by mother
1779
44.25
48.33
–4.09
.10
Ever smoked cigarettes
1801
51.49
53.28
–1.79
.47
Ever drank alcohol
1799
78.54
82.10
–3.56
.07
Ever used marijuana
1799
67.29
68.03
–0.74
.75
Ever sexually activef Currently sexually active (in past 90 d)
1794 1789
93.27 82.97
94.07 83.53
–0.80 –0.56
.51 .76
Sexual intercourse in the past 90 d
1790
78.70
79.08
–0.38
.85
Oral sex in the past 90 d
1786
66.00
66.28
–0.28
.91
Anal sex in the past 90 d
1379
11.52
10.68
0.84
.64
Sexual intercourse without birth control in the past 90 d
1790
31.31
31.47
–0.15
.95
Sexual intercourse without a condom in the past 90 d
1790
59.86
59.14
0.72
.77
Oral sex without a condom in the past 90 d Anal sex without a condom in the past 90 d
1786 1379
62.36 9.58
62.15 7.91
0.22 1.68
.93 .30
Sexual intercourse with > 1 partner (lifetime)
1710
66.21
67.35
–1.15
.63
Sexual intercourse with > 5 partners (lifetime)
1710
23.60
23.37
0.24
.91
1–4
1808
3.26
3.26
0.00
.97
Knowledge of pregnancy risk
0–100
1806
68.50
70.96
–2.47
.17
Knowledge of STI transmission
0–100
1808
68.75
67.24
1.51
.25
Intentions to have sexual intercourse in the next 12 mo
1798
84.22
84.40
–0.18
.92
Intentions to have oral sex in the next 12 mo Intentions to use a condom if they were to have sexual intercourse
1794 1799
60.34 84.62
61.82 83.91
–1.47 0.72
.53 .69
Intentions to use birth control if they were to have sexual intercourse
1800
92.76
91.48
1.29
.33
Risk behavior, %e
Sexual behavior, %
e
Sexual risk, %e
Attitudesg toward protection Knowledgeh
Intentions, %e
Note. STI = sexually transmitted infection. The baseline treatment–control difference was estimated in a regression model with the same structural terms as the impact model but where the dependent variable was the baseline measure, and the only independent variables included in the model were the treatment group indicator and terms for the randomization blocks. a For continuous variables, we present the range. All other variables are dichotomous. b The treatment mean was calculated as the sum of the control group mean and the model-estimated treatment–control difference (group difference). c The group difference is the treatment–control difference. For outcomes reported as percentages, the group difference is expressed in percentage points. For scale outcomes, the group difference is expressed in the original metric of the outcome variable. Because of rounding, reported group differences may differ from differences between reported means for the treatment and control groups. d Racial/ethnic categories include Hispanic, Black non-Hispanic, White non-Hispanic, and other race non-Hispanic, where other is defined as Asian, American Indian or Alaska Native, Native Hawaiian or other Pacific Islander, multiracial, or undisclosed. e For dichotomous variables, we present the percentage of respondents who responded affirmatively. f Sexual activity is defined differently across sites. At 2 sites, sexual activity refers to sexual intercourse, oral sex, or anal sex.Youths were not asked about anal sex at 1 of the sites. g The attitudes variable is a composite scale score with higher scores indicating more positive attitudes. h Knowledge variables are composite scale scores representing the percentage of items answered correctly.
Supplement 1, 2016, Vol 106, No. S1
AJPH
Kelsey et al.
Peer Reviewed
Research
S57
AJPH RESEARCH
TABLE 2—Nine-Month Impacts on Sexual Behavior: Safer Sex Intervention; Minnesota, Tennessee, and Florida; April 2013–March 2015 Outcome
No.
Treatment, %a
Control, %
Group Differenceb
P
c
Sexual behavior Currently sexually active (in the past 90 d)d,e
1801
74.84
74.96
–0.11
.95
Sexual intercourse in the past 90 d
1801
71.29
72.18
–0.89
.66
Oral sex in the past 90 d
1801
59.32
60.39
–1.07
.63
Anal sex in the past 90 d
1389
9.13
6.13
2.99
.051
1801
22.05
27.82
–5.78
.005
Sexual intercourse without a condom in the past 90 df
1801
53.66
57.45
–3.79
.09
Oral sex without a condom in the past 90 d
1801
54.32
56.63
–2.31
.30
Anal sex without a condom in the past 90 d
1389
7.32
4.65
2.67
.06
Sexual intercourse with > 1 partner (lifetime)
1735
70.07
71.82
–1.75
.33
Sexual intercourse with > 5 partners (lifetime)
1735
26.35
28.86
–2.51
.16
Sexual risk Sexual intercourse without birth control in
sites, the treatment–control differences were not significant. The impact on intention to use condoms varied significantly by sexual experience at baseline. SSI participants who had been sexually active at baseline were 8 percentage points (10%) more likely than were control participants to express intentions to use condoms, but among those who were sexually inexperienced at baseline, we found no impacts on intentions to use condoms during sexual intercourse.
the past 90 de
Note. All outcomes are dichotomous. a The treatment group percentage is regression adjusted, calculated as the sum of the control group percentage and the regression-adjusted impact estimate (group difference). b The group difference is the treatment–control difference expressed in percentage points. Because of rounding, reported group differences may differ from the differences between reported percentages for the treatment and control groups. c Sexual activity was defined differently across sites. At 2 sites, sexual activity refers to sexual intercourse, oral sex, or anal sex. Youths were not asked about anal sex at 1 of the sites. d Currently sexually active was constructed from youths’ reports of having sexual intercourse, oral sex, and anal sex in the 90 d preceding survey administration. The measure was coded 0–1, with 0 = no sexual activity and 1 = 1 or more sexual activities in the previous 90 d. e For the 2 confirmatory outcomes, statistical significance at P < .05, P < .01, and P < .001 implies statistical significance at those levels after applying a Benjamini-Hochberg adjustment for multiple comparisons. f Sexual intercourse without a condom was constructed from youths’ reports of engaging in the behavior “even just once” in the 90 d preceding survey administration. The measure was coded on a scale ranging from 0 to 1: youths who engaged in the behavior were coded as 1, and those who did not were coded as 0.
group had slightly more positive attitudes than the control group, by three-hundredths of a point on a scale ranging from 1 to 4 (an effect size of 0.09). There were no significant impacts on attitudes toward risky sexual behavior. SSI had no significant effects on participants’ knowledge of sexual risk. In general, participants were well informed about risks of pregnancy and STIs. SSI had no significant effects on participants’ motivation to delay childbearing. Both treatment and control group members indicated the importance of delaying childbearing until personal goals have been achieved and averaged at least 3.73 on this 4-point scale. No impacts were found on intentions to be sexually active, and no effects were found on intentions to use birth control. However, there was a significant impact on intentions to use a condom during sexual intercourse. Although control group members reported strong
S58
Research
Peer Reviewed
Kelsey et al.
intentions to use condoms, treatment group members were on average 7 percentage points (8%) more likely to say they would. We found significant impacts on perceived refusal skills. Program participants were somewhat more confident than control participants that they would be able to say no to sex under different circumstances (by an average of one tenth of a point on a scale ranging from 1 to 4, an effect size of 0.17). We found no impacts on perceived ability to successfully negotiate condom use with a partner. We found a significant treatment · site interaction for intentions. At the Minnesota site, program participants were more likely, on average, than were control participants to report intentions to use condoms (11 percentage points; 14%) and intentions to use birth control (4 percentage points; 4%). At the Tennessee and Florida
DISCUSSION Findings provide important evidence about SSI’s effectiveness in changing sexual behavior and its antecedents after 9 months among female adolescents at 3 distinct locations across the United States. These results demonstrate that SSI reduced a key sexual risk-taking behavior among female adolescents, namely sexual intercourse without birth control. This impact is similar in size to the overall pooled impact for the most successful pregnancy prevention programs (multicomponent or youth development programs) found in a 2006 meta-analysis of adolescent pregnancy prevention efforts.9 This positive behavioral finding, together with the subgroup findings for Hispanic youths and those not sexually experienced at baseline, and the positive findings for intentions to use condoms and refusal skills, suggest that SSI produced meaningful changes for female adolescents in the short term. Future work will examine whether these changes are sustained over time. Beyond that, the study suggests some implications for clinical practice. Motivational interviewing, although successfully used in other clinical practices, has not been previously tested on a large scale in the field of sexual and reproductive health. It seems to be effective in actively engaging participants and in retaining them (nearly twothirds of participants attended 3 of the 4 sessions). Faced with the challenge of changing established behaviors, the technique produced changes in several of the skills and intentions that ultimately lead to the necessary actions. Although it is hard to disentangle the roles of motivational interviewing and repeated contact, it is likely that the follow-up sessions helped to reinforce
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
TABLE 3—Nine-Month Impacts on Intermediate Outcomes: Safer Sex Intervention; Minnesota, Tennessee, and Florida; April 2013–March 2015 Outcome
Rangea
No.
Treatment Meanb
Control Mean
Group Differencec
SESd
P
0.09
.050
Attitudese Attitudes toward protection
1–4
1809
3.36
3.32
0.03
0–100
1802
4.12
5.42
–1.30
.06
Knowledge of pregnancy risk
0–100
1809
78.53
78.26
0.27
.82
Knowledge of STI risk
0–100
1809
75.91
74.80
1.11
.18
1–4
1805
3.76
3.73
0.03
Sexual intercourse
1802
82.56
83.14
–0.58
Oral sex
1801
65.95
67.05
–1.10
.59
Use a condom if they were to have sexual intercourse
1804
86.31
79.74
6.57
< .001
Use birth control if they were to have sexual intercourse
1803
92.41
91.18
1.23
.36
Attitudes toward risky behavior e
Knowledge
e
Motivation to delay childbearing
0.05
.31
Intentions to engage in the following behaviors in the next 12 mo, %f
Skills
.73
e
Refusal skills
1–4
1808
3.45
3.34
0.10
0.17
.001
Condom negotiation skills
1–4
1808
3.73
3.69
0.03
0.08
.13
Note. SES = standardized effect size of the difference; STI = sexually transmitted infection. a For continuous variables, we present the range. All other variables are dichotomous. b The treatment group mean is regression adjusted, calculated as the sum of the control group mean and the regression-adjusted impact estimate (group difference). c The group difference is the treatment–control difference. For outcomes reported as percentages, the group difference is expressed in percentage points. For scale outcomes, the group difference is expressed in the original metric of the outcome variable. Because of rounding, reported group differences may differ from differences between reported means for the treatment and control groups. d For outcomes that are not dichotomous or measured on a 0–100 scale the SES is the group difference divided by the pooled standard deviation of the treatment and control groups. e Composite scale scores. f Dichotomous variables, reported as percentage of respondents who responded affirmatively.
the participants’ intentions to take action and to support the change process. Although the generalizability of the study findings is limited, this large-scale, rigorous, experimental test of SSI at 3 sites provides important information about how to address the sexual and reproductive health needs of young women that goes beyond a single replication. CONTRIBUTORS M. Kelsey and J. Layzer conceptualized and designed the study, interpreted the data, and drafted and revised the article. J. T. Walker drafted and revised the content. C. Price and R. Juras conceptualized and designed the study and analyzed and interpreted the data.
ACKNOWLEDGMENTS This study was conducted by Abt Associates as part of the Teen Pregnancy Prevention Replication Study, a national replication study of 3 evidence-based adolescent pregnancy prevention reduction strategies funded by the Office of Adolescent Health (OAH) and the Office of the Assistant Secretary for Planning and Evaluation in the US Department of Health and Human Services (HHS; contract no. HHSP23320095624WC, order no. HHSP23337011T, awarded in September 2011).
Supplement 1, 2016, Vol 106, No. S1
AJPH
This article is based on a larger report located on the HHS Web site at https://aspe.hhs.gov/teen-pregnancyprevention-tpp-replication-study. We thank each of the participating grantees for their guidance and assistance. Note. The views expressed in this report are those of the authors and do not necessarily represent the policies of HHS or the Office of Adolescent Health.
HUMAN PARTICIPANT PROTECTION The study was approved by the institutional review board at Abt Associates for all sites and by additional local institutional review boards associated with the study sites as needed.
REFERENCES 1. Kost K, Henshaw SUS. Teenage Pregnancies, Births and Abortions, 2010: National Trends by Age, Race and Ethnicity. New York, NY: Guttmacher Institute; 2014. Available at: http://www.guttmacher.org/pubs/USTPtrends10.pdf. Accessed November 5, 2015. 2. Forhan SE, Gottlieb SL, Sternberg MR, et al. Prevalence of sexually transmitted infections among female adolescents aged 14 to 19 in the United States. Pediatrics. 2009;124(6):1505–1512. 3. Kappeler EM, Farb AF. Historical context for the creation of the Office of Adolescent Health and the Teen Pregnancy Prevention Program. J Adolesc Health. 2014; 54(3, suppl):S3–S9.
4. Shrier LA, Ancheta R, Goodman E, Chiou VM, Lyden MR, Emans SJ. Randomized controlled trial of a safer sex intervention for high-risk adolescent girls. Arch Pediatr Adolesc Med. 2001;155(1):73–79. 5. Orr LL. Social Experiments: Evaluating Public Programs With Experimental Methods. Thousand Oaks, CA: Sage; 1999. 6. Puma MJ, Olsen RB, Bell SH, Price C. What to Do When Data Are Missing in Group Randomized Controlled Trials. Report no. NCEE 2009-0049. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education; 2009. 7. What Works Clearinghouse. WWC Procedures and Standards Handbook Version 3.0. Washington, DC: What Works Clearinghouse, Institute of Education Sciences, US Department of Education; 2014. Available at: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/ wwc_procedures_v3_0_standards_handbook.pdf. Accessed March 7, 2016. 8. Schochet PZ. Technical Methods Report: Guidelines for Multiple Testing in Impact Evaluations. Report no. NCEE 2008-4018. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education, 2008. 9. Scher L, Maynard R, Stagner M. Interventions intended to reduce pregnancy-related outcomes among adolescents: a systematic review. Campbell Systematic Reviews. 2006:12.
Kelsey et al.
Peer Reviewed
Research
S59
Replicating the Safer Sex Intervention: 9-Month Impact Findings of a Randomized Controlled Trial Meredith Kelsey, PhD, Jessica T. Walker, PhD, Jean Layzer, MEd, Cristofer Price, ScM, and Randall Juras, PhD Objectives. To test the effects of the Safer Sex Intervention (SSI) on female adolescents’ sexual behavior and possible antecedents of behavior such as sexual health attitudes, knowledge, motivation, intentions, and skills. Methods. A randomized controlled trial compared SSI (n = 1196) with no intervention (n = 613) among female adolescents aged 13 to 20 years at 3 sites across the United States from 2012 to 2015. Intent-to-treat impacts were estimated at 9 months after baseline, overall, and for key subgroups. Results. Compared with control participants, SSI participants were less likely to have sexual intercourse without birth control, more likely to report positive attitudes toward protection and intention to use condoms, and more confident of their ability to refuse sex. SSI did not affect sexual risk knowledge or motivation to delay childbearing. Positive impacts on sexual behavior and sexual risk were observed among key subgroups of youths who were aged 18 years or older, Hispanic, not sexually experienced at baseline, and enrolled at the Minnesota site. Conclusions. SSI produced meaningful changes in sexual behavior and sexual risk and successfully addressed some potential antecedents of sexual risk behavior. (Am J Public Health. 2016;106:S53–S59. doi:10.2105/AJPH.2016.303372) See editorials, p. S5–S31.
I
n the United States, pregnancy occurs at a rate of 57.4 per 1000 female adolescents, and 1 in 4 sexually active female adolescents has a sexually transmitted infection (STI).1,2 Both outcomes can negatively affect youths’ well-being and future prospects. For this reason, preventing pregnancy and STIs among adolescents is a top public health priority— one in which the federal government has invested heavily, specifically through the Office of Adolescent Health’s Teen Pregnancy Prevention Program.3 The Safer Sex Intervention (SSI) is an evidence-based intervention being replicated by Teen Pregnancy Prevention Program grantees, and it is intended to reduce the incidence of STIs and increase condom use among young sexually active female adolescents. Previous findings from a study of SSI, conducted by the developer in the 1990s in a Boston, Massachusetts, hospital with young women being treated for an STI, indicated that the program reduced their likelihood of having multiple sexual partners
Supplement 1, 2016, Vol 106, No. S1
AJPH
and provided evidence that SSI improved knowledge of sexual risks, attitudes toward and intentions around condom use, and use of condoms.4 In 2010, the Office of Adolescent Health awarded 5-year grants to 75 grantees to implement evidence-based programs intended to reduce adolescent pregnancy. Of the 28 evidence-based adolescent pregnancy prevention programs that grantees might choose to replicate, SSI was the only one designed for a clinic setting and one of the few that adopted an individualized approach to the challenge of changing sexual behavior among adolescents. Given the shortage of evidence-based alternatives for clinic settings and individualized programming, understanding SSI’s
effectiveness is important to both practitioners and policymakers. In this study, we examined whether SSI is effective when replicated in a larger number of clinic settings, with different and more diverse populations, and in 3 geographic locations. Although the program was designed for sexually experienced young women, program staff believed the program would also be relevant for young women who were sexually inexperienced but seeking reproductive health services, and a small number of such youths were included in the present study. We tested whether SSI has positive effects on sexual behaviors that could lead to pregnancy or STI and such antecedents of behavior as attitudes, knowledge, motivation, intentions, and skills. We present findings after 9 months, focusing on the following research questions: 1. Did SSI reduce sexual behavior and risky sexual activity? 2. Did SSI have positive effects on intermediate outcomes thought to lead to behavior change (attitudes toward protection and risky behavior, knowledge of pregnancy risk and STI risk, motivation to delay childbearing, intentions to avoid sexual activity and risky sexual behavior, refusal skills, and condom negotiation skills)?
METHODS The study features an experimental design in which female adolescents were randomly assigned to a group that received SSI or to
ABOUT THE AUTHORS Meredith Kelsey, Jessica T. Walker, Cristofer Price, and Randall Juras are with Abt Associates, Inc., Cambridge, MA. Jean Layzer is with Belmont Research Associates, Belmont, MA. Correspondence should be sent to Meredith Kelsey, PhD, 55 Wheeler Street, Cambridge, MA 02138 (e-mail: Meredith.kelsey@ abtassoc.com). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted July 4, 2016. doi: 10.2105/AJPH.2016.303372
Kelsey et al.
Peer Reviewed
Research
S53
AJPH RESEARCH
a control group that did not. The clinical trial registration number is NCT02540304.
Safer Sex Intervention SSI is a clinic-based, sexual health intervention for female adolescents delivered in one-on-one, face-to-face sessions with a female health educator. The intervention is individually tailored, includes medically accurate information and discussion about safe sexual behaviors, emphasizes skills to protect against risk, and promotes self-efficacy and selfesteem. In the 50- to 60-minute initial session, the health educator guides the participant through a sequence of topics, allowing time for role play and other activities. Topics include the consequences of unprotected sex, risk perception, preventing pregnancy and STIs, condoms, where to obtain condoms, secondary abstinence, and talking about sex. Three subsequent 15- to 30-minute booster sessions, similar in content, are delivered 1, 3, and 6 months after the initial session. The theoretical framework for the intervention draws on social cognitive theory, the transtheoretical model of behavior change, and the motivational interviewing technique.4 The health educator helps the adolescent identify her needs, motivations, and intentions; gradually identify obstacles to behavior change; and make plans to address them. Through subsequent booster sessions, the health educator tracks the participant’s progress through the stages of change. Figure 1 shows the program elements, hypothesized outcomes, and pathways by which SSI seeks to achieve these outcomes.
Setting Replication sites were selected for the study from the small number of grantees that received funding to implement SSI. Three grantees, located in Minnesota, Tennessee, and Florida, were willing to participate and able to meet the requirements of a large-scale rigorous study. Across the 3 sites, SSI was implemented in 38 clinics by trained female health educators. Health educators were given dedicated space within the clinic, and clinicians identified and referred eligible female adolescents to them.
Training, Preparation, and Fidelity Health educators and supervisory staff from all 3 sites attended a 3-day training
S54
Research
Peer Reviewed
Kelsey et al.
provided by the program developer. The training introduced staff to the intervention and how to deliver it with fidelity and focused on motivational interviewing. Supervisory staff received additional training and were certified as trainers to train replacement staff and provide refresher trainings to all staff. All 3 sites provided ongoing training and practice in motivational interviewing. Each site was required to implement the program with fidelity, and fidelity was assessed, monitored, and reported to OAH at regular intervals by program staff. To measure fidelity, OAH provided fidelity checklists and required that they be completed for every session of the program. These data indicate that each site delivered the intervention to youths with fidelity to the core elements of the program model.
Sample Eligibility criteria for the program (and study) were based on gender (female), age (13– 20 years), sexual debut (sexually active or planning to be), and pregnancy status (pregnant adolescents were not eligible but could participate postpartum). Participants were enrolled over a 19-month period, beginning in July 2012. Across all 3 sites, a total of 2108 youths consented, completed a baseline survey, and were randomized (treatment group, n = 1403; control group, n = 705).
program. To protect its integrity, the randomization process was designed and implemented by an independent study team. Participants assigned to the treatment group received SSI. Those assigned to the control group received the usual standard of care offered in the clinic, including medical services sought and contraceptive information. Both groups could receive nonprogram services and informational materials offered by the clinic or available in the community. Clinics varied in the level of contraceptive information offered to members of the control group as part of their usual standard of care.
Data Collection Participants were surveyed at baseline (July 2012–February 2014) and 9 months later (April 2013–March 2015). A Web-based system was used to capture and store survey responses. The baseline survey was completed individually at the clinic on a dedicated computer. At follow-up, participants were e-mailed a unique link to the survey, which they completed independently in any convenient location, using a computer or mobile device. Of the 2108 eligible participants, 86% (n = 1809) completed a 9-month follow-up survey, forming the analytic sample. The 9-month completion rates were very similar across sites and across the study groups (treatment group, n = 1196; control group, n = 613).
Enrollment and Random Assignment
Measures
Clinic staff used the eligibility criteria to identify potential study participants when they presented for clinic services. Youths were then referred to a health educator, who confirmed eligibility, described the study, obtained informed consent, and facilitated all study enrollment activities. Parental consent was waived for unaccompanied minors who sought sexual health services at the clinic. Once youths consented, they completed a self-administered baseline survey before being randomly assigned. Random assignment occurred independently in each of the clinics with a treatment-to-control ratio of approximately 2:1. Individuals within clinics were randomly assigned within randomization blocks on the basis of age to address potential clinic-level variation in participant characteristics and program implementation and potential age-related differences in responses to the
Participants were asked questions that allowed us to measure 2 sets of outcomes: (1) behavioral outcomes (i.e., measures of sexual behavior and sexual risk) and (2) intermediate outcomes (i.e., factors believed to be antecedents of behavioral outcomes). They were also asked about their backgrounds, including race and ethnicity. Because of welldocumented racial/ethnic disparities in rates of adolescent pregnancy and STIs and variation in race/ethnicity across the 3 sites, we examined racial/ethnic differences in program impacts. We briefly describe these measures here (for additional details, see Appendix A, available as a supplement to the online version of this article at http://www.ajph.org). Behavioral outcomes. To address the study’s key questions about the impact of SSI on sexual behavior and sexual risk, we identified 10 measures:
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
Inputs
Process
Intermediate Outcomes
Outcomes
Trained female health educator
Provide medically accurate information on sexual risk behavior and consequences
Positive attitudes toward protection Improved knowledge and understanding
Safer sexual behavior
One-on-one individualized face-to-face sessions
P Positive, trusting relationship
Motivational interview
Facilitate selfassessment of motivational status
Support for motivational change process
Improved motivation/intention to avoid sexual risk
Consistent effective condom use Consistent use of contraceptives Abstinence
Demonstrate condom use and teach negotiation/refusal skills
Role-play, practice to improve skills
Improved negotiation/refusal /condom use skills
Three “booster” sessions
Reduction in sexual activity Reduction in no. of sexual partners
Reduction in STIs Reduction in unwanted pregnancies Reduction in adolescent births
Access to condoms and safer sex info
Note. STI = sexually transmitted infection.
FIGURE 1—Logic model for the Safer Sex Intervention
1. currently sexually active (in the past 90 days), 2. sexual intercourse without birth control in the past 90 days (sexual intercourse defined on the survey as “a male putting his penis into a female’s vagina”; birth control defined as a condom or other form of contraceptive), 3. sexual intercourse in the past 90 days, 4. oral sex in the past 90 days, 5. anal sex in the past 90 days, 6. sexual intercourse without a condom in the past 90 days, 7. oral sex without a condom in the past 90 days, 8. anal sex without a condom in the past 90 days, 9. sexual intercourse with more than 1 partner (lifetime), and 10. sexual intercourse with more than 5 partners (lifetime). For the latter 2 measures, participants reported the actual number of lifetime partners, which we recoded to minimize the
Supplement 1, 2016, Vol 106, No. S1
AJPH
influence of outliers. Otherwise, questions about behaviors were framed as yes–no questions. Responses for each behavior were coded on a scale on which 0 = no and 1 = yes. Intermediate outcomes. We constructed measures to assess the intermediate outcomes (those that may precede behavioral outcomes). A list of the items composing each scale, coding details, and additional information on reliability are provided in Appendix A. Attitudes toward protection were measured on a scale ranging from 1 to 4, with higher scores indicating more positive attitudes toward the use of condoms and other contraceptive methods (12 items; a = .78). Attitudes toward risky behavior were measured on a scale ranging from 0 to 1 and reflected the proportion of statements the respondent agreed with about the acceptability of risky sexual behavior (higher scores represent greater support for risky behavior; 7 items; a = .79). Knowledge of pregnancy risk was measured on a scale ranging from 0 to
1 and reflected the proportion of items about pregnancy risk factors answered correctly (4 items; a = .54). The knowledge of STI risk scale ranged from 0 to 1 and reflected the proportion of items about STI risk factors answered correctly (12 items; a = .68). Motivation to delay childbearing was measured on a scale ranging from 1 to 4, with higher scores indicating greater motivation (3 items; a = .88). Condom negotiation skills were measured on a scale ranging from 1 to 4, with higher scores indicating stronger ability to negotiate condom use (7 items; a = .84). Refusal skills were measured on a scale ranging from 1 to 4, with higher scores indicating stronger ability to abstain from sexual behavior (6 items; a = .83). To assess intentions to engage in sexual behaviors and use protection, we examined 4 dichotomous items separately. Race/ethnicity. To measure race/ethnicity, we asked participants to identify their race from among 6 major categories and separately
Kelsey et al.
Peer Reviewed
Research
S55
AJPH RESEARCH
indicate whether they were of Hispanic descent. We then classified participants as Hispanic, non-Hispanic White, non-Hispanic Black, or non-Hispanic other race/ethnicity.
Analytic Approach To test the impact of SSI, we compared outcomes for the treatment and control groups using a regression framework. For continuous outcomes, we ran traditional regression models; for dichotomous outcomes, we ran linear probability models. We included baseline covariates (e.g., demographic variables, risk behaviors such as baseline sexual activity and substance use), preintervention measure of the outcome variable when available (available for all except attitudes toward risky behavior, refusal skills, and condom negotiation skills), and a series of random assignment block indicators to increase statistical precision of the impact estimates for a given sample size5 and reduce attrition bias from missing data.6 To maintain the integrity of the random assignment design, we conducted an intent-to-treat analysis, using assignment status as the key variable of interest. Data for the impact estimates were pooled across the 3 sites. To address the few instances of missing outcome data, we used case deletion and a dummy-variable adjustment in regression models to account for missing covariates.6 To address multiple comparisons problems (i.e., to reduce the chance of spurious findings from a large number of tests), before analyzing data we prespecified 2 primary or confirmatory outcomes in our analysis plan as the main indicators of program effectiveness: sexual activity in the past 90 days and sexual intercourse without birth control in the past 90 days. To control for having 2 confirmatory outcomes, we applied a formal Benjamini-Hochberg correction, which controls for the false positive rate by adjusting P-value thresholds.7 We considered all other outcomes to be secondary or exploratory; thus, they did not warrant a multiple comparisons adjustment.8 Our main analysis included study participants who were sexually experienced at baseline and participants who were not. As part of our exploratory analyses and to better understand what works for whom, we estimated impacts for key subgroups (site, age,
S56
Research
Peer Reviewed
Kelsey et al.
race/ethnicity, sexual experience at baseline) and tested for differences among subgroups. We implemented subgroup analyses by including subgroup indicators and treatment · subgroup interaction terms in the model and testing for significance of the interaction term. We only present impact estimates for individual subgroups when there was a statistically significant difference between subgroups, which helps guard against overinterpretation of potentially spurious findings across the large number of tests. Because subgroup analyses were exploratory, we did not adjust for multiple comparisons.
RESULTS At baseline, the study sample was on average aged 17 years and diverse in race/ ethnicity: 18% Hispanic, 36% non-Hispanic Black, 33% non-Hispanic White, and 13% non-Hispanic other race/ethnicity. Threequarters lived with 1 or both biological parents. Ninety-four percent had ever been sexually active. Three-quarters had ever used alcohol, two-thirds had ever used marijuana, and half had ever smoked cigarettes. Baseline equivalence was assessed using models with the same structural components as the impact model (i.e., randomization block indicators and treatment group indicator). At baseline, no significant treatment–control group differences were found (Table 1).
intercourse, oral sex, or anal sex, nor did it affect the likelihood of having sexual intercourse with more than 1 partner or more than 5 partners in one’s lifetime. We found some statistically significant differences in behavioral impacts between sexually experienced study participants and the relatively small group who were sexually inexperienced at baseline (Table A, available as a supplement to the online version of this article at http://www.ajph.org). SSI reduced the likelihood of having sexual intercourse with more than 1 lifetime partner by an average of 21% points (70%) for participants who were sexually inexperienced at baseline. Although the numbers affected are small (at baseline, 6% of participants reported never having been sexually active), the impact is large, meaningful, and significantly different than the negligible impact on sexually experienced youths. Among Hispanic youths, SSI significantly decreased the likelihood of having more than 1 lifetime sexual partner by 10 percentage points (13%), on average. We found no impacts on this outcome for any other racial/ethnic group. Among youths aged 18 years and older, SSI significantly decreased rates of engaging in oral sex without a condom by 7 percentage points (11%) on average. We found no impacts on this outcome for younger youths. The impacts on sexual intercourse in the past 90 days also differed significantly between the 2 age groups; however, in neither did the effect reach a conventional level of statistical significance.
Program Impacts on Behavior We observed a large positive impact on 1 of 2 primary outcomes: sexual intercourse without birth control (Table 2). On average, SSI participants were 6 percentage points (21%) less likely to report engaging in sexual intercourse without birth control than were the control group. This impact was significant even after applying a multiple comparisons correction. However, participants in the treatment group were no less likely to report being sexually active in the past 90 days than were their control group counterparts. SSI had no impact on any other measures of sexual activity. The program was not effective in reducing sexual intercourse, oral sex, or anal sex in the past 90 days. It did not affect rates of condom use during sexual
Program Impacts on Intermediate Outcomes Although SSI had no statistically significant effect on knowledge or motivation, it positively and significantly affected some attitudes, intentions, and skills (Table 3). Because no formal statistical adjustments were made to account for the substantial number of secondary outcomes, we would expect some significant findings to arise purely as a result of chance. These results should therefore be viewed cautiously and interpreted in the context of the totality of evidence. SSI had a small but significant impact on attitudes toward using protection (birth control and condoms). Although both groups expressed positive attitudes toward protection, on average the treatment
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
TABLE 1—Baseline Characteristics for the Analytic Sample: Safer Sex Intervention; Minnesota, Tennessee, and Florida; July 2012–February 2014 Demographic Characteristic and Outcomes
Rangea
No.
Treatment Meanb
Control Mean
Group Differencec
P
Age
13–20
1809
17.12
17.14
–0.01
.79
Race/ethnicity, %d,e Hispanic
1809
16.80
19.90
–3.10
.09
Non-Hispanic Black
1809
35.77
35.24
0.53
.81
Non-Hispanic White
1809
33.58
31.65
1.93
.34
Non-Hispanic other
1809
13.85
13.21
0.64
.70
e
Family structure and relationships, % Lives with biological parents
1790
78.97
78.02
0.96
.63
Feels very close to and cared for by father
1603
29.58
26.07
3.51
.14
Feels very close to and cared for by mother
1779
44.25
48.33
–4.09
.10
Ever smoked cigarettes
1801
51.49
53.28
–1.79
.47
Ever drank alcohol
1799
78.54
82.10
–3.56
.07
Ever used marijuana
1799
67.29
68.03
–0.74
.75
Ever sexually activef Currently sexually active (in past 90 d)
1794 1789
93.27 82.97
94.07 83.53
–0.80 –0.56
.51 .76
Sexual intercourse in the past 90 d
1790
78.70
79.08
–0.38
.85
Oral sex in the past 90 d
1786
66.00
66.28
–0.28
.91
Anal sex in the past 90 d
1379
11.52
10.68
0.84
.64
Sexual intercourse without birth control in the past 90 d
1790
31.31
31.47
–0.15
.95
Sexual intercourse without a condom in the past 90 d
1790
59.86
59.14
0.72
.77
Oral sex without a condom in the past 90 d Anal sex without a condom in the past 90 d
1786 1379
62.36 9.58
62.15 7.91
0.22 1.68
.93 .30
Sexual intercourse with > 1 partner (lifetime)
1710
66.21
67.35
–1.15
.63
Sexual intercourse with > 5 partners (lifetime)
1710
23.60
23.37
0.24
.91
1–4
1808
3.26
3.26
0.00
.97
Knowledge of pregnancy risk
0–100
1806
68.50
70.96
–2.47
.17
Knowledge of STI transmission
0–100
1808
68.75
67.24
1.51
.25
Intentions to have sexual intercourse in the next 12 mo
1798
84.22
84.40
–0.18
.92
Intentions to have oral sex in the next 12 mo Intentions to use a condom if they were to have sexual intercourse
1794 1799
60.34 84.62
61.82 83.91
–1.47 0.72
.53 .69
Intentions to use birth control if they were to have sexual intercourse
1800
92.76
91.48
1.29
.33
Risk behavior, %e
Sexual behavior, %
e
Sexual risk, %e
Attitudesg toward protection Knowledgeh
Intentions, %e
Note. STI = sexually transmitted infection. The baseline treatment–control difference was estimated in a regression model with the same structural terms as the impact model but where the dependent variable was the baseline measure, and the only independent variables included in the model were the treatment group indicator and terms for the randomization blocks. a For continuous variables, we present the range. All other variables are dichotomous. b The treatment mean was calculated as the sum of the control group mean and the model-estimated treatment–control difference (group difference). c The group difference is the treatment–control difference. For outcomes reported as percentages, the group difference is expressed in percentage points. For scale outcomes, the group difference is expressed in the original metric of the outcome variable. Because of rounding, reported group differences may differ from differences between reported means for the treatment and control groups. d Racial/ethnic categories include Hispanic, Black non-Hispanic, White non-Hispanic, and other race non-Hispanic, where other is defined as Asian, American Indian or Alaska Native, Native Hawaiian or other Pacific Islander, multiracial, or undisclosed. e For dichotomous variables, we present the percentage of respondents who responded affirmatively. f Sexual activity is defined differently across sites. At 2 sites, sexual activity refers to sexual intercourse, oral sex, or anal sex.Youths were not asked about anal sex at 1 of the sites. g The attitudes variable is a composite scale score with higher scores indicating more positive attitudes. h Knowledge variables are composite scale scores representing the percentage of items answered correctly.
Supplement 1, 2016, Vol 106, No. S1
AJPH
Kelsey et al.
Peer Reviewed
Research
S57
AJPH RESEARCH
TABLE 2—Nine-Month Impacts on Sexual Behavior: Safer Sex Intervention; Minnesota, Tennessee, and Florida; April 2013–March 2015 Outcome
No.
Treatment, %a
Control, %
Group Differenceb
P
c
Sexual behavior Currently sexually active (in the past 90 d)d,e
1801
74.84
74.96
–0.11
.95
Sexual intercourse in the past 90 d
1801
71.29
72.18
–0.89
.66
Oral sex in the past 90 d
1801
59.32
60.39
–1.07
.63
Anal sex in the past 90 d
1389
9.13
6.13
2.99
.051
1801
22.05
27.82
–5.78
.005
Sexual intercourse without a condom in the past 90 df
1801
53.66
57.45
–3.79
.09
Oral sex without a condom in the past 90 d
1801
54.32
56.63
–2.31
.30
Anal sex without a condom in the past 90 d
1389
7.32
4.65
2.67
.06
Sexual intercourse with > 1 partner (lifetime)
1735
70.07
71.82
–1.75
.33
Sexual intercourse with > 5 partners (lifetime)
1735
26.35
28.86
–2.51
.16
Sexual risk Sexual intercourse without birth control in
sites, the treatment–control differences were not significant. The impact on intention to use condoms varied significantly by sexual experience at baseline. SSI participants who had been sexually active at baseline were 8 percentage points (10%) more likely than were control participants to express intentions to use condoms, but among those who were sexually inexperienced at baseline, we found no impacts on intentions to use condoms during sexual intercourse.
the past 90 de
Note. All outcomes are dichotomous. a The treatment group percentage is regression adjusted, calculated as the sum of the control group percentage and the regression-adjusted impact estimate (group difference). b The group difference is the treatment–control difference expressed in percentage points. Because of rounding, reported group differences may differ from the differences between reported percentages for the treatment and control groups. c Sexual activity was defined differently across sites. At 2 sites, sexual activity refers to sexual intercourse, oral sex, or anal sex. Youths were not asked about anal sex at 1 of the sites. d Currently sexually active was constructed from youths’ reports of having sexual intercourse, oral sex, and anal sex in the 90 d preceding survey administration. The measure was coded 0–1, with 0 = no sexual activity and 1 = 1 or more sexual activities in the previous 90 d. e For the 2 confirmatory outcomes, statistical significance at P < .05, P < .01, and P < .001 implies statistical significance at those levels after applying a Benjamini-Hochberg adjustment for multiple comparisons. f Sexual intercourse without a condom was constructed from youths’ reports of engaging in the behavior “even just once” in the 90 d preceding survey administration. The measure was coded on a scale ranging from 0 to 1: youths who engaged in the behavior were coded as 1, and those who did not were coded as 0.
group had slightly more positive attitudes than the control group, by three-hundredths of a point on a scale ranging from 1 to 4 (an effect size of 0.09). There were no significant impacts on attitudes toward risky sexual behavior. SSI had no significant effects on participants’ knowledge of sexual risk. In general, participants were well informed about risks of pregnancy and STIs. SSI had no significant effects on participants’ motivation to delay childbearing. Both treatment and control group members indicated the importance of delaying childbearing until personal goals have been achieved and averaged at least 3.73 on this 4-point scale. No impacts were found on intentions to be sexually active, and no effects were found on intentions to use birth control. However, there was a significant impact on intentions to use a condom during sexual intercourse. Although control group members reported strong
S58
Research
Peer Reviewed
Kelsey et al.
intentions to use condoms, treatment group members were on average 7 percentage points (8%) more likely to say they would. We found significant impacts on perceived refusal skills. Program participants were somewhat more confident than control participants that they would be able to say no to sex under different circumstances (by an average of one tenth of a point on a scale ranging from 1 to 4, an effect size of 0.17). We found no impacts on perceived ability to successfully negotiate condom use with a partner. We found a significant treatment · site interaction for intentions. At the Minnesota site, program participants were more likely, on average, than were control participants to report intentions to use condoms (11 percentage points; 14%) and intentions to use birth control (4 percentage points; 4%). At the Tennessee and Florida
DISCUSSION Findings provide important evidence about SSI’s effectiveness in changing sexual behavior and its antecedents after 9 months among female adolescents at 3 distinct locations across the United States. These results demonstrate that SSI reduced a key sexual risk-taking behavior among female adolescents, namely sexual intercourse without birth control. This impact is similar in size to the overall pooled impact for the most successful pregnancy prevention programs (multicomponent or youth development programs) found in a 2006 meta-analysis of adolescent pregnancy prevention efforts.9 This positive behavioral finding, together with the subgroup findings for Hispanic youths and those not sexually experienced at baseline, and the positive findings for intentions to use condoms and refusal skills, suggest that SSI produced meaningful changes for female adolescents in the short term. Future work will examine whether these changes are sustained over time. Beyond that, the study suggests some implications for clinical practice. Motivational interviewing, although successfully used in other clinical practices, has not been previously tested on a large scale in the field of sexual and reproductive health. It seems to be effective in actively engaging participants and in retaining them (nearly twothirds of participants attended 3 of the 4 sessions). Faced with the challenge of changing established behaviors, the technique produced changes in several of the skills and intentions that ultimately lead to the necessary actions. Although it is hard to disentangle the roles of motivational interviewing and repeated contact, it is likely that the follow-up sessions helped to reinforce
AJPH
Supplement 1, 2016, Vol 106, No. S1
AJPH RESEARCH
TABLE 3—Nine-Month Impacts on Intermediate Outcomes: Safer Sex Intervention; Minnesota, Tennessee, and Florida; April 2013–March 2015 Outcome
Rangea
No.
Treatment Meanb
Control Mean
Group Differencec
SESd
P
0.09
.050
Attitudese Attitudes toward protection
1–4
1809
3.36
3.32
0.03
0–100
1802
4.12
5.42
–1.30
.06
Knowledge of pregnancy risk
0–100
1809
78.53
78.26
0.27
.82
Knowledge of STI risk
0–100
1809
75.91
74.80
1.11
.18
1–4
1805
3.76
3.73
0.03
Sexual intercourse
1802
82.56
83.14
–0.58
Oral sex
1801
65.95
67.05
–1.10
.59
Use a condom if they were to have sexual intercourse
1804
86.31
79.74
6.57
< .001
Use birth control if they were to have sexual intercourse
1803
92.41
91.18
1.23
.36
Attitudes toward risky behavior e
Knowledge
e
Motivation to delay childbearing
0.05
.31
Intentions to engage in the following behaviors in the next 12 mo, %f
Skills
.73
e
Refusal skills
1–4
1808
3.45
3.34
0.10
0.17
.001
Condom negotiation skills
1–4
1808
3.73
3.69
0.03
0.08
.13
Note. SES = standardized effect size of the difference; STI = sexually transmitted infection. a For continuous variables, we present the range. All other variables are dichotomous. b The treatment group mean is regression adjusted, calculated as the sum of the control group mean and the regression-adjusted impact estimate (group difference). c The group difference is the treatment–control difference. For outcomes reported as percentages, the group difference is expressed in percentage points. For scale outcomes, the group difference is expressed in the original metric of the outcome variable. Because of rounding, reported group differences may differ from differences between reported means for the treatment and control groups. d For outcomes that are not dichotomous or measured on a 0–100 scale the SES is the group difference divided by the pooled standard deviation of the treatment and control groups. e Composite scale scores. f Dichotomous variables, reported as percentage of respondents who responded affirmatively.
the participants’ intentions to take action and to support the change process. Although the generalizability of the study findings is limited, this large-scale, rigorous, experimental test of SSI at 3 sites provides important information about how to address the sexual and reproductive health needs of young women that goes beyond a single replication. CONTRIBUTORS M. Kelsey and J. Layzer conceptualized and designed the study, interpreted the data, and drafted and revised the article. J. T. Walker drafted and revised the content. C. Price and R. Juras conceptualized and designed the study and analyzed and interpreted the data.
ACKNOWLEDGMENTS This study was conducted by Abt Associates as part of the Teen Pregnancy Prevention Replication Study, a national replication study of 3 evidence-based adolescent pregnancy prevention reduction strategies funded by the Office of Adolescent Health (OAH) and the Office of the Assistant Secretary for Planning and Evaluation in the US Department of Health and Human Services (HHS; contract no. HHSP23320095624WC, order no. HHSP23337011T, awarded in September 2011).
Supplement 1, 2016, Vol 106, No. S1
AJPH
This article is based on a larger report located on the HHS Web site at https://aspe.hhs.gov/teen-pregnancyprevention-tpp-replication-study. We thank each of the participating grantees for their guidance and assistance. Note. The views expressed in this report are those of the authors and do not necessarily represent the policies of HHS or the Office of Adolescent Health.
HUMAN PARTICIPANT PROTECTION The study was approved by the institutional review board at Abt Associates for all sites and by additional local institutional review boards associated with the study sites as needed.
REFERENCES 1. Kost K, Henshaw SUS. Teenage Pregnancies, Births and Abortions, 2010: National Trends by Age, Race and Ethnicity. New York, NY: Guttmacher Institute; 2014. Available at: http://www.guttmacher.org/pubs/USTPtrends10.pdf. Accessed November 5, 2015. 2. Forhan SE, Gottlieb SL, Sternberg MR, et al. Prevalence of sexually transmitted infections among female adolescents aged 14 to 19 in the United States. Pediatrics. 2009;124(6):1505–1512. 3. Kappeler EM, Farb AF. Historical context for the creation of the Office of Adolescent Health and the Teen Pregnancy Prevention Program. J Adolesc Health. 2014; 54(3, suppl):S3–S9.
4. Shrier LA, Ancheta R, Goodman E, Chiou VM, Lyden MR, Emans SJ. Randomized controlled trial of a safer sex intervention for high-risk adolescent girls. Arch Pediatr Adolesc Med. 2001;155(1):73–79. 5. Orr LL. Social Experiments: Evaluating Public Programs With Experimental Methods. Thousand Oaks, CA: Sage; 1999. 6. Puma MJ, Olsen RB, Bell SH, Price C. What to Do When Data Are Missing in Group Randomized Controlled Trials. Report no. NCEE 2009-0049. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education; 2009. 7. What Works Clearinghouse. WWC Procedures and Standards Handbook Version 3.0. Washington, DC: What Works Clearinghouse, Institute of Education Sciences, US Department of Education; 2014. Available at: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/ wwc_procedures_v3_0_standards_handbook.pdf. Accessed March 7, 2016. 8. Schochet PZ. Technical Methods Report: Guidelines for Multiple Testing in Impact Evaluations. Report no. NCEE 2008-4018. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education, 2008. 9. Scher L, Maynard R, Stagner M. Interventions intended to reduce pregnancy-related outcomes among adolescents: a systematic review. Campbell Systematic Reviews. 2006:12.
Kelsey et al.
Peer Reviewed
Research
S59
