Research

Original Investigation

Brief Intervention for Problem Drug Use in Safety-Net Primary Care Settings

A Randomized Clinical Trial Peter Roy-Byrne, MD; Kristin Bumgardner, BS; A n to in e tte Krupski, PhD; Chris Dunn, PhD; Richard Ries, MD; Dennis Donovan, PhD; Imara I. West, MPH; Charles Maynard, PhD; David C. Atkins, PhD; M eredith C. Graves, PhD; Jutta M. Joesch, PhD; Gary A. Zarkin, PhD

s

E dito ria l page 4 8 8

IMPORTANCE Although brief intervention is effective for reducing problem alcohol use, few

_

data exist on its effectiveness for reducing problem drug use, a common issue in

“

disadvantaged populations seeking care in safety-net medical settings (hospitals and

D S up ple m en ta l c o n te n t at

community health clinics serving low-income patients with limited or no insurance).

R elated a rtic le page 502

jam a.com

OBJECTIVE To determine whether brief intervention improves drug use outcomes compared with enhanced care as usual.

DESIGN, SETTING. AND PARTICIPANTS A randomized clinical trial with blinded assessments at

baseline and at 3,6 ,9 , and 12 months conducted in 7 safety-net primary care clinics in Washington State. Of 1621 eligible patients reporting any problem drug use in the past 90 days, 868 consented and were randomized between April 2 0 09 and September 2012. Follow-up participation was more than 87% at all points. INTERVENTIONS Participants received a single brief intervention using motivational

interviewing, a handout and list o f substance abuse resources, and an attempted 10-minute telephone booster within 2 weeks (n = 435) or enhanced care as usual, which included a handout and list o f substance abuse resources (n = 433). MAIN OUTCOMES AND MEASURES The primary outcomes were self-reported days of problem

drug use in the past 30 days and Addiction Severity Index-Lite (ASI) Drug Use composite score. Secondary outcomes were admission to substance abuse treatment; ASI composite scores for medical, psychiatric, social, and legal domains; emergency department and inpatient hospital admissions, arrests, mortality, and human immunodeficiency virus risk behavior. RESULTS Mean days used of the most common problem drug at baseline were 14.40 (SD, 11.29) (brief intervention) and 13.25 (SD, 10.69) (enhanced care as usual); at 3 months

postintervention, means were 11.87 (SD, 12.13) (brief intervention) and 9.84 (SD, 10.64) (enhanced care as usual) and not significantly different (difference in differences, p = 0.89 [95% Cl, -0 .4 9 to 2.26]). Mean ASI Drug Use composite score at baseline was 0.11 (SD, 0.10) (brief intervention) and 0.11 (SD, 0.10) (enhanced care as usual) and at 3 months was 0.10 (SD, 0.09) (brief intervention) and 0.09 (SD, 0.09) (enhanced care as usual) and not significantly different (difference in differences, p = 0 .0 0 8 [95% Cl, -0 .0 0 6 to 0.021]). During the 12 months following intervention, no significant treatment differences were found for either variable. No significant differences were found for secondary outcomes.

Author Affiliations: Department o f Psychiatry and Behavioral Sciences, School o f Medicine, University of Washington, Seattle (Roy-Byrne, Bumgardner, Krupski, Dunn. Ries. Donovan. West, Atkins, Graves, Joesch); Department o f Health

CONCLUSIONS a n d r e l e v a n c e A one-time brief intervention with attempted telephone

booster had no effect on drug use in patients seen in safety-net primary care settings. This finding suggests a need for caution in promoting widespread adoption o f this intervention for drug use in primary care.

Services, School o f Public Health, University o f Washington, Seattle (Maynard); King County Office of Performance, Strategy and Budget, Seattle, Washington (Joesch); RTI international, Research Triangle Park, North Carolina (Zarkin).

TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00877331

JAMA. 2014;312(5):492-501. doi:10.1001/jama.2014.7860

492

Corresponding Author: Peter RoyByrne, MD, University o f Washington School o f Medicine, PO Box 359911, 325 Ninth Ave, Seattle, WA 98104 ([email protected]).

jama.com

Original Investigation Research

Brief Intervention for Problem Drug Use

ased on th e established efficacy of brief (1-2 sessions) interventions for hazardous alcohol use in patients seen in m edical settings , 1,2 national dissem ination pro­ grams of screening, brief intervention, and referral to treat­ m en t (SBIRT) for “alcohol and d ru g s” have been im p le­ m ented on a widespread scale .3,4 This rapid adoption of brief intervention for drugs other than alcohol has outstripped its meager evidence base. Of the 3 randomized clinical trials of brief intervention for drug abuse in non-treatm ent-seeking individuals ,5' 8 only 2 were positive ,7,8 and only l found an ef­ fect among US patients .8 Ambulatory primary care is an important setting to test the effectiveness of brief intervention because of the large vol­ um e of patients seen in that setting .9 The majority of patients with problem drug use in this setting are not accessing sub­ stance abuse treatm ent .4,10 Adding brief substance abuse in­ terventions onsite in primary care is also consistent with newly emerging “integrated care” m odels .11 Recent studies show that a disproportionate number of in­ dividuals with drug abuse or dependence are from lower so­ cioeconomic strata 12 and primarily receive care in public sec­ tor hospitals and com m unity health clinics (ie, safety-net medical settings that serve low-income patients with limited or no insurance). Although the cost of untreated drug abuse in this system is substantial (eg, mortality, crime, lost produc­ tivity, increased medical care ),13 there is limited information o n th e s e p u b lic h e a lth - r e le v a n t o u tc o m e s fo r b r ie f intervention .14,15 Policy makers need such data to create trac­ tion to drive policy changes required to improve substance abuse treatm ent. The purpose of this study was to determine w hether a brief intervention using motivational interviewing would reduce prob­ lem drug use in safety-net primary care patients and increase ad­ mission to specialist substance abuse treatment. We also sought to estimate the effects of the brief intervention on several pub­ lic health outcomes directly related to problem drug use.

B

Methods Design

A 2 -group randomized clinical trial was conducted to exam ­ ine the effects of a l-tim e brief intervention using m otiva­ tional interviewing with attem pted telephone booster com­ p are d w ith en h a n c e d care as usu al. A h ybrid efficacyeffectiveness design was chosen to enhance external validity by using a brief intervention protocol similar to that used in the Substance Abuse and Mental Health Services Administra­ tion-funded Washington State SBIRT study 16 and having the intervention delivered by social workers already working with patients in these clinics .17 The study was approved by an in­ stitutional review board and an independent data and safety monitoring board. Further details about study design and ra­ tionale have been described .18 Participants

Participants were recruited betw een April 2009 and Septem­ ber 2012 from the waiting rooms of 7 safety-net (public hospitaljama.com

associated) primary care clinics in King County, Washington. Inclusion criteria were age 18 years or older; self-reported use of an illegal drug or nonprescribed medication (ie, problem drug use) at least once in the 90 days before screening19; Eng­ lish-speaking and able to read and understand screening and consent forms (sixth-grade literacy); currently receiving and planning to continue receiving care in the clinic; and having telephone or e-mail access to facilitate scheduling follow-up assessments. Exclusion criteria were attendance in formal sub­ stance abuse treatm en t in th e past m onth (excluding selfhelp groups such as Narcotics Anonymous); high risk of im ­ m in en t suicide; life -th rea te n in g m edical illness; severe cognitive impairment; or active psychosis. All participants pro­ vided w ritten informed consent and received compensation in gift cards for completion of study assessments ($25 at base­ line and 3 months, $30 at 6 months, $35 at 9 months, $40 at 12 months, and $10 for urine samples at each follow-up). Randomization

After a brief baseline assessment, participants were random ­ ized in a 1:1 ratio (Figure) to brief intervention or enhanced care as usual using perm uted blocks stratified by clinic and by 3 fac­ tors known to affect outcome: drug use severity 20 (Drug Abuse Screening Test [DAST-1 0 ] score >3 [range, 0 -1 0 ; 10 indicates greatest severity]; the DAST-10 instrum ent and interpreta­ tion guide are available in the eAppendix in the Supplement), comorbid mental illness 21 (Addiction Severity Index-Lite [ASI] Psychiatric Status composite score >0.38 [range, 0 -1; 1 indi­ cates greatest problem severity]), and readiness to change 22 (goal of abstinence on Thoughts about Abstinence assess­ ment). Varying-sized blocks were generated prior to enroll­ m ent, and allocation was concealed in sequentially n u m ­ bered opaque envelopes opened by the research assistant at random ization .23 Intervention

To maximize the possibility of an effect while maintaining realworld feasibility, brief interventionists were recruited from so­ cial workers in participating clinics (n = l l ) .24 The interven­ tionists agreed to adhere to the study protocol, including ongoing training and supervision, and to provide w ritten in­ formed consent (because data on their fidelity to the motiva­ tional interviewing model would be used for the study). Ad­ ditional m aster’s- and bachelor’s-level interventionists not working in the clinics (n = 6 ) were later added to help m eet re­ cruitm ent goals. All interventionists were trained by 1 of the original trainers for the Substance Abuse and Mental Health Services Administration’s national SBIRT initiative 3 using a group workshop followed by individual feedback on audiotaped role-plays w ith up to 5 standardized patients over the course of 5 w eeks.25 All interventionists m et basic m otiva­ tional interviewing proficiency training goals on at least 1 prac­ tice role-play .26 Participants randomized to the intervention condition were to receive a single, approximately 30-minute brief interven­ tion. The clinical tasks of the intervention protocol were to give participants feedback about their drug use screening (DAST10 ) results, explore the pros and cons of drug use, increase parJAMA Augu5t6,2014 Volume 312, Number 5

493

Research Original Investigation

ticipant confidence in being able to change, and discuss op­ tions for change. When appropriate, interventionists used an illustrated handout depicting the participant’s DAST-10 score and its associated problem severity (low, intermediate, substantial/severe) to aid the discussion (the DAST-io feedback handout is available in the eAppendix in the Supplement) and provided a list of substance abuse treatm ent resources. A mo­ tivational interviewing approach was used to perform these tasks, with the hope that understanding and exploring am ­ bivalence about change among vulnerable, underserved par­ ticipants while eliciting their own arguments for change might empower them and bridge cultural differences. Also, as op­ posed to a “one size fits all” intervention, this tailored ap ­ proach allowed flexibility as to which or how many drugs to target, as well as in how to guide the participant (eg, specialty treatm ent, abstinence, harm reduction). The same interven­ tionist attem pted a follow-up telephone booster session within 2 weeks of th e intervention. Interventions w ere audio re ­ corded and scored by trained coders using the Motivational In­ terview ing T reatm ent Integrity (MITI) coding sy stem .27 Monthly group supervision and ongoing e-mailed feedback from the trainer (25% of recorded brief interventions) were used to maintain intervention fidelity. To maximize external valid­ ity, MITI scoring feedback was not given to interventionists for training role-plays or for real-patient supervision. Enhanced Care as Usual

Participants in the enhanced care as usual group received the same illustrated handout depicting their DAST-10 drug prob­ lem severity score and list of substance abuse treatm ent re­ sources. These were provided w ith only a quick introduction by the research assistant to minimize intervention elements in the control condition and resembled the “notification and referral” strategy that might be implemented in high-quality usual care. Assessment Measures

The assessment battery, adm inistered by trained research as­ sistants, was brief to minimize any unintended intervention effects17,28 and to maximize the chance of completing study procedures around a primary care visit. The baseline inter­ view assessed demographics, including self-reported race and ethnicity; substance abuse severity (DAST-10)20; drug use in the previous month, comorbid medical and mental illness, and social and legal outcomes related to drug use (ASI)29; goal for changing drug use (Thoughts about Abstinence assessment)22; and the HIV Risk-taking Behavior Scale.30 Urine samples for drug screens were also collected to track point prevalence of actual drug use. All measures except the demographics and DAST-10 were repeated at 3-, 6-, 9-, and 12-month follow-up assessments conducted by research assistants blinded to ran­ domization assignment. The last 12-month assessm ent was completed on September 20,2013. Primary Outcomes

The primary outcome, prespecified on ClinicalTrials.gov, was self-reported days of drug use during the past 30 days. The ASI measures drug use for individual drugs but does not include 494

JAMA August 6,2014 Volume 312, Number 5

Brief Intervention for Problem Drug Use

an omnibus category for num ber of days with any drug use. As a proxy for this, present analyses focused on the m ost fre­ quently used drug at baseline for each individual. This was supplem ented with the ASI Drug Use composite score (range, 0 -1; 1 indicates greatest severity), an integrated measure of fre­ quency of use and associated problems for all drugs used. For both primary outcomes, m easurem ent of drug use excluded use o f alcohol and prescription drugs w hen u sed as p re ­ scribed. Secondary Outcomes

Medical, psychiatric, social, and legal subscales of the ASI were used to assess functional correlates of drug use. The assess­ m ents were supplem ented w ith Washington State adm inis­ trative data, which included chemical dependency treatm ent records, inpatient hospitalizations, state patrol arrest rec­ ords, and death records. Emergency departm ent and outpa­ tient visits were also tracked using electronic medical record data from the safety-net medical center w here the study took place. Analysis

Acute treatm ent differences in primary outcomes (days used during last 30 days and drug use composite) focused on pre­ intervention to 3 m onths postintervention and used a differ­ ence in differences analysis strategy .31 Long-term treatm ent differences during 12 m onths postin terv en tio n were an a­ lyzed using generalized estim ating equations (GEEs).32 Be­ cause days of use was highly bimodal, the original count vari­ able was divided into 4 categories of days of use (0 ,1-10,1120 , 21-30) and analyzed using an ordinal GEE,33 whereas the drug use composite score was analyzed with a gaussian GEE. For both outcomes, alternative distributions (eg, negative bi­ nomial, log-normal) were examined, but all substantive con­ clusions w ere identical to th o se rep o rted . Covariates in ­ cluded baseline value of the outcome, indicator variables for the clinics where recruitm ent occurred, and baseline values of the stratification variables used in randomization. Admin­ istrative data were analyzed via logistic regression, and ASI scores for medical, psychiatric, social, and legal domains were analyzed via linear regression. Prespecified treatm ent effect modification of primary outcomes was exam ined for base­ line stratification variables of drug use severity, comorbid psy­ chiatric illness, and motivation to change drug use. As a post hoc exploratory analysis, treatm ent effect modification of se­ lect secondary outcomes was also examined. All analyses were intention-to-treat, w ith all available data from participants analyzed by random ization group. There were few missing data overall (87% of planned assess­ m ents completed), and GEE analyses m ade use of all avail­ able data. Significance was based on 2-sided P values 6 medical conditions),35 w ith notable percentages of hospitalization (25%) and emergency departm ent use (49%); 10% had received chemical depen­ dency treatm en t (Table 2). Drug use was sim ilar betw een groups (mean, 13.82 [SD, 11.00] days of the m ost frequently used drug) and was highly heterogeneous, w ith different types and patterns of use evident according to the severity of drug use measured by validated DAST-10 score categories (eTable 1 and eTable 2 in the Supplem ent). Many partici­ p an ts also used alcohol, b u t because we had no data on quantity of alcohol consumed per drinking day, determining degree of hazardous alcohol use was not possible. Supple­ m enting self-reported drug use w ith positive urine screen results only slightly increased the proportion of individuals classified as using particular drugs (eTable 3 in the Supple­ ment). A majority of participants (56%) had some comorbid m ental illness, and a m inority (37%) had the goal to try to ab stain from drugs. These 2 stratification variables were also significantly associated w ith problem drug use severity such th a t the m ost severe group was m ost likely to have a goal of abstinence and also to have more mental illness. Intervention Participation and Fidelity

Of the 435 participants randomized to the intervention, 423 (97%) received a brief intervention and 203 (47%) received a booster call. Most participants (90%) received the brief inter­ vention on the same day as the baseline assessment, but for logistical reasons, the remaining 10% had to return to the clinic for the intervention a m edian of 4 days after assessment. The brief intervention averaged 27 m inutes, and 86% of interven­ tions were recorded. Mean interventionist scores for all audiorecorded brief interventions met basic proficiency cutoffs for 4 of 5 MITI summ ary scores, suggesting adequate but not ex­ pert proficiency.26 Over the course of the next year, a small num ber of participants in both conditions (brief interven­ tion: n = 26; enhanced care as usual: n = 34) received brief in­ terventions outside the study via regular clinical services. Main O utcom es

Figure 2 depicts days of drug use during the last 30 days and drug use com posite scores over tim e in the 2 groups. The jama.com

Original Investigation Research

treatm ent means in each plot are virtually identical, change over time is minimal, and there is no evidence of an interven­ tion effect. Mean days used of th e m ost com m on problem drug at baseline were 14.40 (SD, 11.29) (brief intervention) an d 13.25 (SD, 10.69) (en h an ced care as usual) and at 3 m onths postintervention were 11.97 (SD, 12.13) (brief inter­ vention) and 9.84 (SD, 10.64) (enhanced care as usual). Mean ASI Drug Use composite score at baseline was 0.11 (SD, 0.10) (brief intervention) and 0.11 (SD, 0.10) (enhanced care as usual) and at 3 m onths was 0.10 (SD, 0.09) (brief interven­ tion) and 0.09 (SD, 0.09) (enhanced care as usual). Acute trea tm e n t differences based on difference in differences revealed no significant differences in either days of drug use (P = 0.89 [95% Cl, -0.49 to 2.26]) or ASI Drug Use composite (p = 0.008 [95% Cl, -0.006 to 0.021]). Generalized estimating equation analyses confirmed th at there were no significant treatm ent differences during 12 m onths postintervention in either days of drug use (odds ratio, 1.20 [95% Cl, 0.96 to 1.50]) or the ASI Drug Use composite (P = 0.005 [95% Cl, -0.005 to 0.016]). Table 2 reports the secondary outcome variables, includ­ ing ASI scores for medical, psychiatric, employment, social, and legal domains, proportion of individuals who accepted a re­ ferral to chemical dependency treatm ent and had an initial treatm ent contact, and relevant public health outcomes (eg, medical care use, arrests, deaths). There were no significant intervention effects on any secondary outcomes, including ad­ mission to chemical dependency treatm ent. Effect modification of primary drug use outcomes by base­ line drug use severity, psychiatric comorbidity, and motiva­ tion to change revealed no significant m oderation effects (eTable 4 and eTable 5 in the Supplement). However, in ex­ ploratory effect m odification analyses of secondary o u t­ comes, drug use severity significantly m oderated interven­ tion effects on both admission to treatm ent and emergency departm ent use. Participants with high drug problem sever­ ity receiving the intervention were more likely to enter spe­ cialist drug treatm ent and more likely to reduce their use of the emergency department (Table 3 and eTable 6 in the Supple­ ment).

Discussion In th e overall sam ple, p articip an ts w ho received a b rief intervention using proficiency-level motivational interview­ ing show ed no benefit across m ultiple dom ains, including drug use frequency, adm issions to drug treatm ent, and all other secondary m easures, including those of high public health relevance. Both groups show ed m odest and similar reductions in drug use frequency over th e first 3 m onths w ith no subsequent change, possibly suggesting a regres­ sion to th e m ean in b o th groups. A lthough our negative findings are co n siste n t w ith prior stu d ies in tre a tm e n t­ seeking patients in the United States,5’6’8 they also may be a product of our participant characteristics as well as the ways in which the intervention was conducted and the outcomes were evaluated. JAMA August 6.2014 Volume 312, Number 5

495

Research Original Investigation

Brief Intervention fo r Problem Drug Use

Figure 1. P a rticip a n t F low in th e Trial o f a B rie f In te rv e n tio n fo r P roblem D rug Use

39 062 Patients approached for screening

T 28725 Excluded 15560 Excluded at approach (did not meet inclusion criteria or met exclusion criteria)3 9952 Declined to participate 3213 Other (eg, left during approach)

10337 Screened

~r — -►

8716 Excluded 8540 Excluded at screening (did not meet inclusion criteria or met exclusion criteria)3 176 Other (eg, left during screening)

1621 Eligible

— >

______

674 Excluded 520 Declined to participate 154 Other (eg, left during consent process)

> 947 Provided consent

— >

79 Excluded 44 Excluded during baseline assessment 22 Formal chemical dependency treatment 9 Acute suicidality or psychosis 8 No drug use in past 3 mo 3 Not primary care patient 2 Insufficient contact information 32 Declined to participate 3 Other

868 Randomized

a Inclusion criteria were age 18 years or older; self-reported use o f an illegal drug or nonprescribed medication (ie, problem drug use) at least once in the 9 0 days before screening19; English-speaking and able to read and understand screening and consent forms (sixth-grade literacy); currently receiving and planning to continue care in the clinic; and having telephone or e-mail access to facilitate scheduling follow-up assessments. Exclusion criteria were attendance in formal substance abuse treatm ent in the past month (excluding self-help groups such as Narcotics Anonymous); high risk o f imm inent suicide; life-threatening medical illness; severe cognitive impairment; or active psychosis. Data for reasons o f exclusion are not available.

496

JAMA

August 6,2014

Volume 312, Number 5

jama.com

Brief Intervention for Problem Drug Use

Original Investigation Research

Table 1. Baseline Characteristics o f Randomized Participants (N = 8 6 8 )a Participants, No. (%)

Characteristic

Overall (N = 868)

Brief Intervention (n = 435)

Enhanced Care as Usual (n = 433)

47.76 (10.89)

47.50 (11.29)

48.03 (10.48)

264 (30)

127 (29)

137 (32)

Demographics Age, mean (SD), y Women Raceb White

386 (45)

190 (44)

196 (45)

Black

320 (37)

157 (36)

163 (38)

Other

150 (18)

82 (19)

68 (16)

72 (9)

33 (9)

39 (10)

Hispanic Marital status Married/living with partner

161 (19)

80 (18)

81 (19)

Divorced/separated/widowed

348 (40)

167 (39)

181 (42)

Never married

357 (41)

188 (43)

169 (39)

Education Some high school or less

166 (19)

91 (21)

75 (17)

High school graduate

254 (29)

125 (29)

129 (30)

Beyond high school

447 (52)

218(50)

229 (53)

Employment status Working

78 (9)

33 (8)

45 (10)

238(27)

120 (28)

118(27)

Disabled and unable to work

551(64)

282 (65)

269(62)

Homeless in shelter or on street >1 night in past 90 d

263 (30)

130 (30)

133 (31)

CDPS medical conditions, mean (SD)35,C

6.12 (3.56)

6.20 (3.54)

6.04 (3.58)

ASI Psychiatric Status composite score >0.3821,d

470 (54)

232 (53)

238 (55)

>1 Mental illness ICD-9 diagnosis codec

478 (56)

248 (58)

230 (55)

Unemployed/retired/in school/homemaker/other

Medical/psychiatric

a Missing values are not included in this table.

Substance use ASI days most frequently used drug, mean (SD)e ASI Drug Use composite score, mean (SD)d-e

13.82 (11.00)

14.40 (11.29)

13.25 (10.69)

0.11 (0.10)

0.11 (0.10)

0.11 (0.10)

656 (76)

333 (77)

323(75)

ASI drug use, any in past 30 df Marijuana Stimulants'

362 (42)

184 (42)

178 (41)

Cocaine

325(37)

161 (37)

164 (38)

Amphetamines Opiates' Heroin Methadone and other opiates/analgesics nonprescribed Sedatives/hypnotics/tranquilizers Other drugs'-9

63 (7)

36 (8)

27 (6)

228 (26)

105 (24)

123 (28)

59(7)

30(7)

29 (7)

208 (24)

96 (22)

112 (26)

72(8)

29 (7)

43 (10)

51(6)

23 (5)

28 (6)

2 or more drugs used in past 30 de

389 (45)

188 (43)

201 (46)

Intravenous drug use in past 30 d

72(8)

39(9)

33 (8)

Low (score 1-2)

278 (32)

141 (32)

137 (32)

328 (38)

169 (39)

159 (37)

Substantial/severe (score >6) Goal of total abstinence from drugse'h

262 (30)

125 (29)

137(32)

323 (37)

158 (36)

165 (38)

d ASI composite scores range from 0 to 1, with 1indicating greatest problem severity. e Excludes use of alcohol or nicotine. ' ASI drug use groups reported are not mutually exclusive. 8 "Other drugs" can include all other abused medications (eg, antihistamines, antidepressants) or drugs of abuse (eg, hallucinogens, inhalants) not included in the h From the Thoughts about Abstinence measure, which is used to assess one's goal for changing drug use (no goal, controlled use, occasional use, temporary abstinence, total abstinence slip is possible, total abstinence never use

ASI Alcohol Use composite score, mean (SD)d

0.15 (0.20)

0.14 (0.20)

0.16 (0.20)

ASI alcohol use, any in past 30 d

598 (69)

276 (63)

322 (74)

Nicotine use, any in past 30 d

620(72)

318 (73)

302 (70)

jama.com

b Assessed by self-report using National Institutes of Health reporting categories for federally funded clinical research. c Administrative data available for 848 participants in the 1-year preintervention period.

DAST-10 drug use severity'

Intermediate (score 3-5)

Abbreviations: ASI, Addiction Severity Index: Bl, brief intervention; CD, chemical dependency; CDPS. Chronic Illness and Disability Payment System; DAST-10, Drug Abuse Screening Test 10-item; ICD-9. International Classification of Diseases, Ninth Revision.

abstinence from drugs” includes "Total abstinence, never use again" and "Total abstinence, slip is possible."

JAMA August 6,2014 Volume 312, Number 5

497

Brief Intervention fo r Problem Drug Use

Research Original Investigation

Table 2. S elect S econdary O u tco m e s3-11 Baseline Brief Intervention (n = 426)

Outcome

1 Yearc

Enhanced Care as Usual (n = 422)

Brief Intervention (n = 426)

Enhanced Care as Usual (n = 422)

Regression Coefficient or Adjusted OR (95% Cl)de

P

Value

ASI composite scores, mean (SD)f Medical9

0.65 (0.32)

0.66 (0.35)

0.54 (0.35)

0.56 (0.36)

-0 .0 0 4 (-0 .0 5 0 to 0.042)d

.86

Psychiatric"

0.37 (0.24)

0.39 (0.24)

0.31 (0.26)

0.32 (0.26)

0.004 (-0 .0 2 6 to 0.034)d

.79

Employment'

0.79 (0.23)

0.79 (0.23)

0.78 (0.24)

0.78 (0.24)

0.006 (-0 .0 1 6 to 0.028)d

.58

Family/social1

0.17 (0.22)

0 .1 7 (0 .2 2 )

0.11 (0.18)

0.13 (0.20)

-0 .0 2 0 (-0 .0 4 6 to 0.006)d

.14

Legal"

0.06 (0.12)

0.07 (0.15)

0.04 (0.10)

0.04 (0.12)

0.000 (-0 .0 1 4 to 0.014)d

.95

Public health-relevant measures, No. (%) w ith any Washington State chemical dependency treatment admissions1

37 (9)

5 3 (1 3 )

60 (14)

57 (14)

1.16 (0.77 to 1 .7 3 )'

.48

Washington State inpatient hospitalizations"1

113 (27)

108 (26)

106 (25)

98 (23)

1.09 (0.78 to 1 .5 1 )'

.62

Safety-net hospital emergency department visits

215 (50)

213 (50)

204 (48)

198 (47)

1.04 (0.76 to 2 .0 6 )'

.77

Safety-net outpatient visits

3 8 1 (8 9 )

Felony or gross misdemeanor arrests HIV Risk-taking Behavior Scale risk factor > l n

380 (90)

402 (94)

399 (95)

1.00 (0.53 to 1.88)e

.99

3 6 (8 )

43 (10)

41 (10)

37 (9)

1.21 (0.74 to 1 .9 8 )'

.45

237 (55)

225 (53)

195 (51)

198 (51)

0.90 (0.66 to 1 .2 5 )'

.54

10 (2)

7 (2 )

1.42 (0.54 to 3 .7 8 )'

.48

NA

Death

NA

' ASI composite scores range from 0 to 1, w ith 1 indicating greatest problem

Abbreviations: ASI, Addiction Severity Index-Lite; ED, emergency department; HIV, human immunodeficiency virus; NA, not applicable; OR, odds ratio.

severity.

3 Data available fo r 848 participants unless otherwise indicated.

8 Data available fo r 776 participants.

b With the exception o f death, the preintervention value o f the outcome was used as a covariate in the model. For example, in assessing the association

h Data available fo r 764 participants. ' Data available fo r 771 participants.

between randomization and hospitalization w ithin 1 year after intervention,

j Data available for 768 participants.

the proportion hospitalized in the 1 year before intervention was used as a

k Data available for 748 participants.

covariate.

1 Excludes detoxification services.

c Results measured at 12-month assessment fo r ASI composite scores and during the 1-year postintervention period for public health-relevant measures.

"Data available through December 31,2012; 86% had complete 1-year information.

d Regression coefficients by linear regression. A positive coefficient indicates the intervention was associated w ith a higher score.

" Data available for 764 participants.

' Adjusted ORs by logistic regression. Values greater than 1 indicate the intervention was associated w ith higher odds o f the outcome.

Figure 2. Changes in P roblem D rug Use O ve r Tim e Drug use days in last 30 d for most frequently used drug

ASI drug use composite score during 12 mo

Time Since Randomization, mo

Time Since Randomization, mo

3

No. o f participants Brief intervention Enhanced care as usual

435 433

378 389

381 385

377 384

386 392

426 420

370 379

6

374 373

9

372 375

12

380 382

The Addiction Severity Index-Lite (ASI) Drug Use composite score accounts for frequency o f use and associated problems fo r all drugs used, excluding alcohol and medications taken as prescribed (range, 0-1; 1 indicates greatest severity).

498

JAMA

August 6,2014

Volume 312, Number 5

jama.com

B r ie f I n t e r v e n tio n f o r P ro b le m D ru g Use

O r ig in a l I n v e s tig a t io n

R e s e a rc h

T a b le 3 . P o t e n t ia l E f f e c t M o d i f i c a t i o n : E s t im a t e s f o r S e le c t S e c o n d a r y O u t c o m e s b y B a s e lin e D r u g U s e S e v e r it y 3 A d m is s io n s

r.

.

,„

, _ . . C h e m ic a l D e p e n d e n c y T re a tm e n t

E m e rg e n c y D e p a rtm e n t --------------------------------------------------------------------------------------------------------------------------- -— - —

(E s tim a te d L ik e lih o o d o f 2 1 A d m is s io n )

P o te n tia l E ffe c t M o d ifie r /L e v e l

B rie f In te r v e n tio n (n = 4 2 6 )

E n h a n ce d Care as U sua l (n = 4 2 2 )

P V a lu e

E s tim a te d L ik e lih o o d o f 2 1 A d m is s io n B rie f In te r v e n tio n (n = 4 2 6 )

E n h an ce d Care as U sual (n = 4 2 2 )

P V a lu e

M ean N u m b e r A m o n g A d m it te d 11 B rie f In te r v e n tio n (n = 2 0 4 )

E n h an ce d Care as U sua l (n = 1 9 8 )

V a lu e

.0 1

P

B a se lin e d ru g use s e v e rity L o w (D A S T -1 0 sco re 1 -2 )

0 .0 4

0 .0 9

4 /1 3 7

9 /1 3 5

In te rm e d ia te (D A S T -1 0 sco re 3 - 5 )

0 .1 3

0 .1 3

N o. e x p e rie n c in g e v e n t/s a m p le size o r m ean (S D )C

2 0 /1 6 7

1 9 /1 5 6

0 .2 6

0 .1 6

3 6 /1 2 2

2 9 /1 3 1

No. e x p e rie n c in g

.1 4

0 .7 9

0 .9 0

5 3 /1 3 7

5 4 /1 3 5

0 .7 8

0 .7 6

8 1 /1 6 7

6 9 /1 5 6

0 .8 4

0 .7 7

7 0 /1 2 2

7 5 /1 3 1

.8 6

2 .3 1

1 .4 4

2 .4 9 (2 .3 3 )

1 .6 5 ( 1 .0 5 )

2 .8 0

2 .9 3

2 .9 4 (2 .9 3 )

2 .8 3 ( 4 .7 3 )

1 .9 3

3 .6 0

e v e n t/s a m p le size o r m e a n (S D )C

S u b s ta n tia l/s e v e re (D A S T -1 0 sco re 2 6 ) No. e x p e rie n c in g

.95

.0 4

.4 5

.9 0

2 .6 3 (2 .8 1 )

.8 6

.01

4 .0 3 ( 5 .2 9 )

e v e n t/s a m p le size o r m e a n (S D )C A b b re v ia tio n s : ASI, A d d ic t io n S e v e rity In d e x - L ite : DAST-10, D ru g A b u s e S c re e n in g T e s t 1 0 -ite m . 3 O u tc o m e e s tim a te s a p p ly t o 1 -year p o s t in t e r v e n tio n p e rio d , c o n d itio n a l o n

s tr a tifie d b y le v e ls o f DAST-10 s c o re s re v e a le d n o s ig n ific a n t t r e a tm e n t d iffe re n c e s . c U n a d ju s te d re s u lts m e a s u re d o v e r 1 -year p o s t in t e r v e n tio n p e rio d f o r n u m b e r

b a s e lin e d r u g u s e s e v e r ity a n d a t m e a n v a lu e s o f p r e in te r v e n t io n le v e l o f

o f p a r tic ip a n ts e x p e rie n c in g 1 o r m o re c h e m ic a l d e p e n d e n c y tr e a tm e n t

o u tc o m e s , b a s e lin e ASI P s y c h ia tric c o m p o s ite s c o re , a n d a b s tin e n c e g o a l o n

a d m is s io n s , n u m b e r o f p a r tic ip a n ts e x p e rie n c in g 1 o r m o re e m e rg e n c y

t h e T h o u g h ts a b o u t A b s tin e n c e a s s e s s m e n t a t b a s e lin e . b T r e a tm e n t d iffe r e n c e s f o r e m e rg e n c y d e p a r tm e n t c o s ts w e r e a ls o e x a m in e d

d e p a r tm e n t a d m is s io n s , a n d m e a n n u m b e r o f e m e rg e n c y d e p a r tm e n t a d m is s io n s a m o n g th o s e w it h 1 o r m o re a d m is s io n s .

a n d c o n ta in e d a n u m b e r o f e x tr e m e o u tlie rs . N o n p a ra m e tric ra n k s ta tis tic s

First and m ost im portant, drug use in th e sam ple was v ery h etero g en e o u s, sp an n in g casual m ariju an a u se to severe opiate and stim ulant abuse. Although the DAST-10 score, w hich m easures severity and is strongly related to type of drug used, did not moderate intervention effects on freq u en cy o f drug use, it did m o d erate b o th tre a tm e n t u p ta k e an d em ergency d e p a rtm e n t use in ex p lo ra to ry analyses of secondary outcom es. Participants in th e brief intervention group with the m ost severe drug use problems, w ith a greater pattern of stim ulant and opiate abuse, were more likely to pursue specialty treatm ent and had reduced use of the em ergency d epartm ent relative to those in the enhanced care as usual group. This is consistent w ith the 1 positive study of brief intervention for cocaine and opiate users by Bernstein et al,7 although no effects were found on treatm ent uptake. In contrast, no benefit of the intervention was found for those in the lowest severity of drug use prob­ lems, predom inantly marijuana users. Although legalization of m arijuana use in W ashington State did not occur until well after enrollm ent, th e legalization clim ate m ay have served to norm alize m arijuana use, often n o t associated w ith work or personal problems, and reduce motivation for change for this substance. Second, the sample was severely socioeconomically disadvantaged. However, studies of alco­ hol brief intervention have not indicated that disadvantage/ poverty m oderates brief intervention effect,36 and the study by Bernstein et al drew participants from homeless and lowincome clinics. Third, the sample had a high rate of comorbid m ental illness, w hich is know n to be associated w ith poorer substance use outcom es.37 These last 2 aspects of ja m a .c o m

our population (poverty and m ental illness), along w ith the fact th at patients in this study received compensation, may limit the generalizability of our results. Other factors may have influenced the effectiveness of the brief intervention. First, the majority of participants had a single brief intervention contact, with only 47% receiving a fol­ low-up booster call. Although more frequent contact may in­ crease efficacy,1some studies have failed to show this.38 Sec­ ond, the participant’s primary care physician did not deliver th e intervention, nor did a research-confirm ed expert. Al­ though the current use of a physician-extender to deliver the intervention is consistent w ith extant integrated care models and has been shown effective in other contexts,39 the physi­ cian might have m ore influence with a patient.15 Third, despite efforts to simplify the assessment battery, all participants received up to 5 assessments, in which infor­ mation about drug use was discussed, as well as DAST-iO feed­ back. It is possible that this may have inadvertently served to produce an intervention effect in the com parison group.40 Fourth, measuring frequency b u t not quantity of drug use is a limited m easure of outcome. Similarly, our measure of alco­ hol use did not include quantity consumed, limiting our abil­ ity to use alcohol as either a predictor or outcome variable. Cur­ rently there is no gold standard for quantifying problem drug use. Researchers measuring problem drug use outcomes m ust find a way to measure quantity, as well as frequency, of use. It is possible th a t our exploratory finding of reduced em er­ gency departm ent utilization could reflect changes in quan­ tity but not frequency of drug use, although the finding may also be spurious. JA M A

A u g u s t 6 ,2 0 1 4

V o lu m e 312. N u m b e r 5

499

Research Original Investigation

Brief Intervention fo r Problem Drug Use

Despite these limitations, further research to identify sub­ groups responsive to this intervention, as well as the role of more intensive interventions, appears to be warranted. For ex­ ample, targeting intervention efforts toward individuals with severe drug abuse, many of whom use stim ulants and opiates and may be at higher risk of overdose and other harmful con­ sequences, might increase the uptake of specialty treatm ent and reduce emergency departm ent utilization.

Conclusions A one-tim e b rief in terv en tio n w ith attem p ted telephone booster call had no effect on drug use in patients seen in safetynet primary care settings. This finding suggests a need for cau­ tion in promoting widespread adoption of this intervention for drug use in primary care.

ARTICLE INFORMATION

enhanced brief intervention and then to provide

Involvement Screening Test (ASSIST) in clients

Author Contributions: Dr Roy-Byrne had full

study interventions to participating clinic patients; and the expert research staff employed by the

recruited from primary health-care settings in four

access to all o f the data in the study and takes responsibility for the integrity o f the data and the accuracy o f the data analysis.

grant fo r study coordination, data collection

Study concept and design: Roy-Byrne, Bumgardner,

and coding o f intervention recordings. We also

resulting in more than 87% follow-up participation,

9. Saitz R, Alford DP, Bernstein J, Cheng DM, Samet J, Palfai T. Screening and brief intervention for unhealthy drug use in primary care settings:

Krupski, Dunn, Ries, Donovan, Joesch.

gratefully acknowledge the expert oversight o f our

Acquisition, analysis, or interpretation o f data: All authors.

data and safety monitoring board (Katharine A.

Drafting o f the manuscript: Roy-Byrne, Bumgardner, Dunn, Ries, Atkins.

Institute, Seattle, Washington]; John M. Roll, PhD [Professor and Senior Vice Chancellor, Washington

intervention. SubstAbus. 2007;28(3):1-2.

Critical revision o f the manuscript for im portant

State University, Spokane]; and Andrew J. Saxon,

11. Chan YF, Huang H, Sieu N, Unutzer J. Substance

randomized clinical trials are needed .J Addict Med. 2010;4(3):123-130.

Bradley, MD, MPH [Group Health Research 10. Saitz R. Introduction: screening and brief

intellectual content: Roy-Byrne, Bumgardner,

MD [Director, Center o f Excellence in Substance

screening and referral for substance abuse

Krupski, Ries, Donovan, West, Maynard, Atkins, Graves, Joesch, Zarkin. Statistical analysis: West, Maynard, Atkins, Graves,

Abuse Treatment and Education, VA Puget Sound

treatm ent in an integrated mental health care

Joesch, Zarkin.

Washington, Seattle]), and the support o f the late

Obtained funding: Roy-Byrne, Bumgardner, Krupski, Dunn, Ries, Donovan, Joesch. Zarkin.

Richard A. Denisco, MD, MPH, our program official at the National institute on Drug Abuse. Dr Denisco

Administrative, technical, or m aterial support:

and the members o f the data and safety monitoring

Roy-Byrne, Bumgardner, Krupski.

board did not receive compensation from this grant.

Study supervision: Roy-Byrne, Bumgardner, Dunn, Ries.

Health Care System; Professor, Department of Psychiatry & Behavioral Sciences, University o f

REFERENCES Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for

Effectiveness o f brief alcohol interventions in

Roy-Byrne reported receiving financial support as the editor in chief o f Depression and Anxiety, Journal Watch Psychiatry, and UpToDate Psychiatry

primary care populations. Cochrane Database Syst Rev. 2007;(2):CD004148.

and receiving stock options fo r consultation to

subm itted work. Dr Ries reported receiving financial support from Janssen Pharmaceuticals Inc, Alkermes, and Reckitt Benckiser Pharamaceutical Inc, outside the submitted work. No other authors reported disclosures. Funding/Support: This study was funded by National Institute on Drug Abuse grant R01 DA026014 (Dr Roy-Byrne). Role of the Sponsor: The National Institute on Drug Abuse had no role in the design and conduct o f the study; the collection, management, analysis, and interpretation o f the data: the preparation, review, or approval o f the manuscript: or the decision to submit the manuscript fo r publication. Disclaimer: The views expressed reflect those o f the authors and do not necessarily reflect the official views o f the National Institute on Drug Abuse or the National Institutes o f Health. Additional Contributions: We gratefully acknowledge the participation and support o f many people w itho u t whom this study could never have been completed. These include the patients and the administrative and clinical staff, including medical directors, o f the primary care clinics that

2. Bertholet N, Daeppen JB. Wietlisbach V, Fleming M, Burnand B. Reduction o f alcohol consumption

JAMA

August 6,2014

Volume 312, Number 5

Prevalence, correlates, disability, and com orbidity o f DSM-IVdrug abuse and dependence in the United States: results from the national epidemiologic survey on alcohol and related conditions. Arch Gen Psychiatry. 2007;64(5):566576. 13. Wickizer TM, Campbell K, Krupski A, Stark K. treated fo r substance abuse in Washington State. Milbank Q. 2000;78(4):585-608. 14. Solberg LI, Maciosek MV, Edwards NM. Primary care intervention to reduce alcohol misuse: ranking its health impact and cost effectiveness. Am JPrev Med. 2008;34(2):143-152.

by brief alcohol intervention in primary care: systematic review and meta-analysis. Arch Intern Med. 2005;165(9):986-995.

15. Fleming MF, Barry KL, Manwell LB, Johnson K, London R. Brief physician advice fo r problem alcohol drinkers: a randomized controlled trial in

3. Substance Abuse and Mental Health Services Administration (SAMHSA). Screening, Brief

community-based primary care practices. JAMA. 1997;277(13):1039-1045.

Intervention, Referral and Treatment (SBIRT). SAMHSA website, http://beta.samhsa.gov/sbirt. April 2014. Accessed July 2,2014. 4. BaborTF, McRee BG, Kassebaum PA, Grimaldi PL, Ahmed K, Bray J. Screening, Brief Intervention, and Referral to Treatment (SBIRT): toward a public health approach to the management o f substance abuse. SubstAbus. 2007;28(3):7-30.

16. Krupski A, Sears JM, Joesch JM, et al. Impact o f brief interventions and brief treatm ent on admissions to chemical dependency treatment. Drug Alcohol Depend. 2010;110(1-2):126-136. 17. Carroll KM, Rounsaville BJ. Bridging the gap: a hybrid model to link efficacy and effectiveness research in substance abuse treatm ent. Psychiatr Serv. 2003;54(3):333-339.

5. O tto C, Crackau B, Lohrmann I, et al. Brief intervention in general hospital fo r problematic prescription drug use: 12-month outcome. Drug

18. Krupski A, Joesch JM, Dunn C, et al. Testing the effects o f brief intervention in primary care for

Alcohol Depend. 2009;105(3):221-226.

problem drug use in a randomized controlled trial: rationale, design, and methods. Addict Sci Clin Pract.

6. Zahradnik A, O tto C, Crackau B. et al. Randomized controlled trial o f a brief intervention

2012;7(1):27.

fo r problematic prescription drug use in non-treatment-seeking patients. Addiction. 2 00 9 ;

19. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test fo r drug

104(1):109-117.

use in primary care. Arch Intern Med. 2010:170(13): 1155-1160.

7. Bernstein J, Bernstein E, Tassiopoulos K. Heeren T, Levenson S, Hingson R. Brief motivational intervention at a clinic visit reduces cocaine and

20. Yudko E, Lozhkina 0 , Fouts A. A comprehensive review o f the psychometric

heroin use. Drug Alcohol Depend. 2005;77(l):49-59.

properties o f the Drug Abuse Screening Test. JSubst Abuse Treat. 2007;32(2):189-198.

8. Humeniuk R, Ali R, Babor T, et al. A randomized controlled trial o f a brief intervention for illicit drugs

21. Alterman Al, McLellan AT, Shifman RB. Do

linked to the Alcohol, Smoking and Substance

substance abuse patients w ith more

participated in the study: the interventionists receiving grant support w ho spent tim e to train and gain proficiency in the motivational interviewing-

program. PsychiatrServ. 2013:64(0:88-90. 12. Compton WM, Thomas YF, Stinson FS, Grant BF.

Employment outcomes among AFDC recipients 1. Kaner EF, Beyer F, Dickinson HO, et al.

Disclosure o f Potential Conflicts o f Interest. Dr

Valant Medical Solutions (behavioral health electronic medical record company), outside the

500

countries. Addiction. 2012;107(5):957-966.

jama.com

Brief Intervention fo r Problem Drug Use

Original Investigation Research

psychopathology receive more treatment? J Nerv

28. Kypri K. Methodological issues in alcohol

Ment Dis. 1993;181(9):576-582.

screening and brief intervention research. Subst Abus.

22. Hall SM, Havassy BE, Wasserman DA.

2007;28(3):31-42.

(5):540-545.

Com m itm ent to abstinence and acute stress in

29. Cacciola JS. Alterman Al, McLellan AT, Lin YT,

37. Krupski A, Sears JM, Joesch JM, et al.

relapse to alcohol, opiates, and nicotine. J Consult

Lynch KG. Initial evidence for the reliability and validity o f a "Lite" version o f the Addiction Severity

Self-reported alcohol and drug use six months after

Index. Drug Alcohol Depend. 2007:87(2-3):297-302.

Clin Psychol. 1990;58(2):175-181.

outcome of, screening and brief interventions for alcohol misuse: a review. Alcohol Alcohol. 2006:41

brief intervention: do changes in reported use vary

23. Doig GS, Simpson F. Randomization and allocation concealment: a practical guide for

30. Darke S, Hall W, Heather N, Ward J, Wodak A.

by mental-health status? Addict Sci Clin Pract. 2012; 7(1):24.

researchers. J Crit Care. 2005;20(2):187-193.

The reliability and validity o f a scale to measure HIV

38. D'Onofrio G, Fiellin DA, Pantalon MV, et al. A

24. Roche AM, Freeman T. Brief interventions:

risk-taking behaviour among intravenous drug users. AIDS. 1991;5(2):181-185.

brief intervention reduces hazardous and harmful

good in theory but weak in practice. Drug Alcohol Rev. 2004;23(1):11-18.

31. Ashenfelter 0, Card D. Using the longitudinal

drinking in emergency departm ent patients. Ann Emerg Med. 2012;60(2):181-192.

25. Baer JS, Rosengren DB, Dunn CW, Wells EA,

structure o f earnings to estimate the effects of

39. Sullivan LE, Tetrault JM, Braithwaite RS, Turner

Ogle RL, Hartzler B. An evaluation o f workshop

training programs. Rev Econ Stat. 1985;67:648-660.

BJ, Fiellin DA. A meta-analysis o f the efficacy o f

training in motivational interviewing for addiction

32. Hardin J. Hildbe J. Generalized Estimating

nonphysician brief interventions for unhealthy alcohol use: implications for the patient-centered medical home. Am J Addict. 2011;20(4):343-356.

and mental health clinicians. Drug Alcohol Depend.

Equations. 2nd ed. Boca Raton, FL: CRC Press; 2013.

2004;73(1):99-106.

33. Touloumis A, Agresti A, Kateri M. GEE for

26. Moyers TB, Martin T, Manuel JK, Hendrickson

multinomial responses using a local odds ratios

40. Neighbors C, Lewis MA, Atkins DC, et al.

SM, Miller WR. Assessing competence in the use of motivational interviewing. JSubst Abuse Treat.

parameterization. Biometrics. 2013;69(3):633-640.

Efficacy o f web-based personalized normative feedback: a two-year randomized controlled trial.

2005;28(1):19-26.

34. Team RCR. A Language and Environment for

27. Moyers TB, Martin T, Manuel JK, Miller WR,

for Statistical Computing; 2013.

Ernst D. Revised Global Scales: Motivational Interview Treatment Integrity (MITI 3.1). University

35. Kronick R, Gilmer T, Dreyfus T. Lee L. Improving

o f New Mexico Center on Alcoholism, Substance

J Consult Clin Psychol. 2010;78(6):898-911.

Statistical Computing. Vienna, Austria: R Foundation

health-based payment for Medicaid beneficiaries: CDPS. Health Care FinancRev. 2000;21(3):29-64.

Abuse, and Addictions (CASAA) website. http://casaa .unm.edu/download/MITI3J.pdf. January 22,2010. Accessed July 16,2014.

jama.com

36. Littlejohn C. Does socio-economic status influence the acceptability of, attendance for, and

JAMA August 6,2014 Volume 312, Number 5

501

Copyright 2014 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.