HHS Public Access Author manuscript Author Manuscript
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28. Published in final edited form as: Psychol Addict Behav. 2016 February ; 30(1): 1–11. doi:10.1037/adb0000128.
A Mixed-methods Evaluation of the Feasibility, Acceptability and Preliminary Efficacy of a Mobile Intervention for Methadone Maintenance Clients Honoria Guarino, Ph.D.1, Michelle Acosta, Ph.D.1, Lisa A. Marsch, Ph.D.2, Haiyi Xie, Ph.D.2, and Yesenia Aponte-Melendez, M.A.1
Author Manuscript
1
Center for Technology and Health, National Development and Research Institutes, New York, NY
2
Center for Technology and Behavioral Health, Dartmouth Center for Psychiatric Research, Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH
Abstract
Author Manuscript Author Manuscript
Despite the recent explosion of behavioral health interventions delivered on mobile devices, little is known about factors that make such applications practical, engaging and useful to their target audience. This study reports on the feasibility, acceptability and preliminary efficacy of a prototype of a novel, interactive mobile psychosocial intervention to reduce problematic drug use among clients in methadone maintenance treatment (MMT). A mixed-methods pilot study with new MMT clients (n=25) indicated that the mobile intervention approach was feasible, and that participants found the intervention highly acceptable and useful. On 100-point visual analog scale (VAS) items, participants reported high levels of liking the program (M=75.6), and endorsed it as useful (M=77.5), easy to use (M=80.7), and containing a significant amount of new information (M=74.8). When compared with 25 study participants who received standard MMT alone, pilot participants rated their treatment significantly higher in interestingness and usefulness, and were significantly more satisfied with their treatment. In qualitative interviews, participants reported using the mobile intervention in a range of settings, including during times of heightened risk for substance use, and finding it helpful in managing drug cravings. Additionally, pilot participants showed evidence of increased treatment retention and abstinence from illicit opioids (in terms of effect size) over a 3-month period relative to those in standard MMT, suggesting the application’s potential to enhance treatment outcomes. These promising findings suggest that an evidence-based mobile therapeutic tool addressing substance use may appeal to drug treatment clients and have clinical utility as an adjunct to formal treatment.
Corresponding Author: Honoria Guarino, Ph.D., Principal Investigator, National Development and Research Institutes, Inc., 71 West 23rd Street, 4th Floor, New York, NY 10010, Ph: (212) 845-4540, Fax: (917) 438-0894, [email protected]. Authors Note: In addition to her academic affiliation, Dr. Marsch is affiliated with HealthSim, LLC, the health-promotion software development organization that developed the technology-based intervention referenced in this manuscript. Dr. Marsch has worked extensively with her institutions to manage any potential conflict of interest. All research data collection, data management, and statistical analyses were conducted by individuals with no affiliation to HealthSim, LLC.
Guarino et al.
Page 2
Author Manuscript
Keywords technology-based interventions; mobile health; intervention acceptability; intervention engagement; psychosocial interventions for SUD
Author Manuscript Author Manuscript
Approximately 681,000 Americans reported using heroin in 2013, according to the latest National Survey on Drug Use and Health, a 69% increase from the 2002 rate (SAMHSA, 2014a), while more than 11 million people misused prescription opioids in 2013 (SAMHSA, 2014b). Given the recent increase in the number of Americans reporting nonmedical use of prescription opioids (SAMHSA, 2014a) – and emerging evidence that a growing proportion of nonmedical prescription opioid users are transitioning to heroin use (Frank, MateuGelabert, Guarino, Bennett, Wendel, & Teper, 2014; Jones, 2013; Mars, Bourgois, Karandinos, Montero, & Ciccarone, 2013; Mateu-Gelabert, Guarino, Jessell, & Teper, A., 2014) – the demand for effective treatment for opioid use disorders is high and likely to intensify in the near future. Yet the vast majority of those who could benefit from treatment do not receive it. This may be due to the limited availability of existing treatment programs and to a range of societal and individual-level barriers, including the stigma associated with identifying oneself as a substance user and difficulty attending a treatment program on a regular basis (Campbell, Muench & Nunes, in press; Cunningham, Sobel & Chow, 1993; Rapp, Xu, Carr, Lane, Wang, & Carlson, 2006). In addition, drug treatment programs face numerous barriers to providing comprehensive, evidence-based psychosocial treatment to their clients, including limited financial and staffing resources, high staff turnover and highvolume clinician-patient caseloads (D’Aunno & Vaughn, 1995; Kraft, Rothbard, Hadley, McLellan, & Asch, 1997). Although methadone maintenance treatment (MMT) has been demonstrated to be one of the most efficacious treatment modalities for opioid-dependence, MMT programs offer the lowest amount of psychosocial treatment and have the highest client-to-staff ratio relative to other types of drug treatment, such as outpatient counseling, inpatient rehabilitation and residential programs (Etheridge, Craddock, Dunteman, & Hubbard, 1995).
Author Manuscript
Technology-based interventions have strong potential to help address these barriers and to expand access to evidence-based psychosocial treatment both within MMT and in drug treatment more generally. Relative to traditional, face-to-face treatment modalities, technology-based interventions can offer greater privacy, convenience, reach and costeffectiveness Cucciare, Weingardt, Greene, & Hoffman, 2012; Cucciare, Weingardt & Humphreys, 2009; (Marsch, Carroll & Kiluk, 2014). These benefits may be especially relevant in MMT, where patient attendance burden is high and staff and financial resources are strained. In addition, technology-based therapeutic tools, especially those utilizing mobile technologies, have the potential to deliver personalized, on-demand treatment to individuals with problematic substance use, providing support in real-world contexts, at times and in situations when it may be most needed (Marsch, 2012). There is a nascent, but growing, evidence base demonstrating the efficacy of technologybased interventions for substance use disorders (SUDs), especially when they are used as adjuncts to traditional, face-to-face treatment modalities. Meta-analyses and systematic
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 3
Author Manuscript
reviews of technology-based interventions for SUDs have consistently found that such tools are associated with significantly better outcomes than minimal or no-treatment controls and may be equivalent in efficacy to other evidence-based psychosocial treatments, such as clinician-delivered Cognitive Behavioral Therapy and Motivation Enhancement Therapy (Campbell, Muench & Nunes, in press; Litvin, Abrantes & Brown, 2013; Moore, Fazzino, Garnet, Cutter, & Barry, 2011; Quanbeck, Chih, Isham, Johnson, & Gustafson, 2014). However, the preponderance of research in this area has focused on web-delivered interventions for problem drinking and smoking cessation, while fewer technology-based interventions have been developed and evaluated for other SUDs, including opioid dependence. The research literature is also sparser for web-based tools that are accessible on mobile devices as compared to desktop-based web applications (Moore et al., 2011).
Author Manuscript Author Manuscript
Multiple studies and reviews of technology-based tools for a range of behavioral health conditions, including SUDs, have generally found significant correlations between use of, or “engagement” with, an intervention and positive outcomes, with individuals who display higher rates of intervention use being most likely to evince successful treatment outcomes (Bennett & Glasgow, 2009; Freyne, Saunders, Brindal, Berkovsky, & Smith, 2012; Gainsbury & Blaszczynski, 2011; Glasgow, Christiansen, Kurz, King, Woolley, Faber et al., 2011; Gustafson, McTavish, Chih, Atwood, Johnson, Boyle et al., 2014; Litvin, Abrantes, & Brown, 2013; Strecher, McClure, Alexander, Chakraborty, Nair, Konkel et al., 2008). However, effectively motivating members of a target population to engage with technologybased tools, and particularly to sustain engagement over time, remains a challenge. Similar to standard face-to-face treatments, user attrition has been a salient problem in research evaluations of technology-based interventions, with a 50% attrition rate in the active intervention period a common finding (Donkin & Glozier, 2012; Freyne et al., 2012; Kelders, Kok, Ossebaard, & Van Gemert-Pijnen, 2012; Litvin, Abrantes & Brown, 2013). Study retention and intervention adherence rates have generally been worse in purely technology-based trials that include no face-to-face interaction with research or clinical staff. While this may indicate that factors beyond the acceptability of the intervention itself may be involved, it does suggest the important role of human/provider contact in the effective implementation of technology-based interventions. Together, these findings underscore the importance of systematically examining the acceptability and appeal of an intervention from the end user’s perspective, as individuals are unlikely to use tools they do not like (KayLambkin, Baker, Lewin, & Carr, 2011). In-depth investigations of users’ experiences with technology-based behavioral health tools may help identify program features that reliably promote sustained engagement, thus facilitating the development of tools that are maximally appealing, engaging and effective.
Author Manuscript
Initial research in this area has identified certain features of technology-based behavioral health tools and their implementation that are associated with increased engagement, including individual-level personalization of content, “push reminders” (e.g., automated text messages or emails) to promote use of a tool, provision of incentives for use of a tool, and integration of human/social support and encouragement (Mohr, Cuijpers & Lehman, 2011; Strecher et al., 2008). However, extant process research on participants’ experiences with technology-based interventions for SUDs is quite limited, and studies applying qualitative or mixed methods to this issue are particularly scarce. A qualitative or mixed-methods Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 4
Author Manuscript
approach to process research is especially well-suited for gaining insight into participants’ experiences and perspectives, understanding how local contexts influence these experiences, and uncovering salient themes that may be unanticipated at the start of an investigation. More research of this type is needed to deepen our understanding of the specific intervention features and components that best promote acceptability and sustained engagement with technology-based interventions for SUDs.
Author Manuscript
The current study was conducted within the context of a larger randomized, controlled trial to evaluate the effectiveness of a web-based psychosocial program called the Therapeutic Education System (TES; Bickel, Marsch, Buchalter, & Badger, 2008) when delivered to opioid-dependent individuals in MMT. TES is an interactive, customized program that is theoretically grounded in the evidence-based behavior therapy program for substance abuse known as the Community Reinforcement Approach. This approach aims to train individuals in skills and behaviors to help reduce problematic substance use and HIV/HCV risk behaviors, improve employment status and family/social relations, and increase nonsubstance-related recreational activities (Budney & Higgins, 1998). TES has been demonstrated to significantly improve treatment outcomes when delivered to individuals in a range of community-based substance abuse treatment settings, including MMT (Bickel, Marsch, Buchalter & Badger, 2008; Brooks, Ryder, Carise, & Kirby, 2010; Marsch, Grabinski, Bickel, Desrosiers, Guarino, Muehlbach et al., 2011; Marsch, Guarino, Acosta, Aponte-Melendez, Cleland, Grabinski et al., 2014; Campbell, Nunes, Matthews, Stitzer, Miele, Polsky et al., 2015).
Author Manuscript
As a supplement to this parent trial (Marsch et al., 2014), our team developed a prototype version of an interactive, mobile phone-based psychosocial intervention, based on key elements of the Community Reinforcement Approach and Cognitive Behavior Therapy, to promote skills acquisition and reduce illicit drug use among MMT clients. Designed to be theoretically compatible with TES in its focus on practical skills-acquisition and selfmanagement of drug use, this mobile application, called the Check-In Program, was conceptualized as a “treatment extender”, enabling provision of flexible, on-demand therapeutic support to participants in settings and at times of their choosing.
Author Manuscript
The present mixed-methods pilot study reports on the feasibility, acceptability and preliminary efficacy of this novel application when delivered to a sample of participants in conjunction with web-based TES. We hoped to learn whether MMT clients would be interested in and readily able to use the mobile application, whether they would be able to retain their study phones in working condition for the duration of the intervention period and whether they would find the tool helpful in reducing or abstaining from illicit opioid use. To evaluate the application’s potential to enhance treatment outcomes, we also assessed participants’ opioid use (via urine toxicology) and treatment retention relative to a group receiving standard MMT alone.
The Check-In Program The prototype version of the Check-In Program evaluated in this study consisted of two skills-based modules, a Functional Analysis module which aimed to help individuals
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 5
Author Manuscript
identify their patterns of problematic substance use and the specific triggers to substance use they face in their day-to-day lives, and a Self-Management module which assisted individuals in developing a plan to manage the triggers to substance use identified in the Functional Analysis module. Each module contained explanatory text defining key concepts (e.g., “triggers”, “self-management plan”), followed by an interactive exercise. A standardized, daily text message prompt (“Don’t forget to use the Check-In Program to help you stay on track.”) was sent to each program user’s mobile phone to encourage use of the program.
Author Manuscript
Upon first accessing the program, users were required to complete both modules in their entirety at least once. Progress through the application at each subsequent use of the program was tailored based on responses to recurring queries presented at each login asking if the individual had used any opioids (besides his/her daily methadone dose) or been exposed to any high-risk situations that could lead to opioid use since his/her last login. Affirmative responses led the user into the exercise portion of the Functional Analysis or Self-Management module, respectively, while negative responses allowed the user to either log out of the program or electively complete either module’s exercise. At any time, users could elect to review the introductory didactic modular content.
Author Manuscript
In the Functional Analysis exercise, participants were asked to recall their most recent episode of non-MMT opioid use and then to identify their most salient triggers, or emotional, cognitive or environmental cues that increase the likelihood of engaging in problematic opioid use, including both internal triggers (e.g., certain thoughts or feelings) and external triggers (e.g., certain people, places or situations). Participants were then asked to identify the short-term and long-term consequences of their opioid use, both positive and negative. For all interactive items within both modules, users had the option of selecting a response from a checklist provided or selecting “other” and typing in their own response using the phone’s touchscreen keyboard.
Author Manuscript
In the Self-Management exercise, participants were asked to select one of their selfidentified triggers and were guided through successive steps to develop a plan for managing this trigger in the future without resorting to opioid use. Self-management plans could involve strategies such as re-arranging one’s daily routine or environment to avoid the trigger or identifying alternative responses to the trigger that do not involve using opioids. In order to help motivate behavior change and prepare participants for potential challenges in managing triggers, participants were then asked to identify the likely positive and negative consequences of implementing their chosen plan. Finally, participants were asked to rate the difficulty of their plan, using a scrollbar to select a difficulty rating ranging from 1 (least difficult) to 10 (most difficult).
Method Parent Study Procedures In the parent trial, opioid-dependent adults (n=160) within their first 30 days of entering outpatient MMT at the study site, a large New York City MMT program, were randomized to receive either 12 months of 1) standard treatment or 2) a reduced schedule of standard
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 6
Author Manuscript
treatment plus biweekly sessions using TES. Eligible participants had to be at least 18 years of age, meet DSM criteria for opioid dependence (required for entry into MMT), have sufficient English language ability to understand the study assessments and intervention content, and be enrolled during the first month of their current MMT episode. Details of the parent trial have been published elsewhere (Marsch et al., 2014; Acosta, Marsch, Xie, Guarino, & Aponte-Melendez, 2013).
Author Manuscript
Standard treatment included daily administration of a therapeutic dose of methadone (typically ranging from 80-120 mg/day). MMT was initiated immediately at treatment entry and usually required clinic attendance six days per week. Standard treatment also included the routine substance abuse counseling provided by the MMT program, i.e., meeting with one’s assigned counselor for 30-60-minute individual sessions once weekly during the first month of treatment and once monthly thereafter. Participants randomized to the reducedstandard treatment-plus-TES condition received the same methadone treatment plus standard counseling as those in the standard condition, except that one half of each counseling session was spent with their counselor, while the other half was spent using TES in an on-site computer lab. Pilot Study Procedures At the conclusion of participant enrollment in the parent trial, an additional sample of MMT clients (n=25) meeting the above eligibility criteria were enrolled and received reducedstandard treatment-plus-TES, as well as access to the mobile Check-In program, for the 12week pilot study intervention period.
Author Manuscript
As a supplement to their biweekly computer-based TES sessions, each pilot study participant was provided with a Nokia feature phone preloaded with the Check-In application for use throughout the intervention period. Participants were given an initial, one-on-one tutorial (approximately 30 minutes long) in which research staff explained how to use key features of the Nokia phone and how to access, securely login to and complete the Check-In program.
Author Manuscript
Participants were encouraged to use the Check-In program at any time, as frequently as they wished, particularly in settings outside of their treatment program. Because the prototype Check-In program was designed as a native (not web-based) application, no wireless connection was necessary to use the mobile intervention. After the initial tutorial, participants were asked to meet with research staff on a weekly basis. During these sessions (about 15 minutes long, on average): Check-In program usage data were downloaded from the phones onto a central server (as there was no viable way to remotely extract data from the native application); prepaid minutes were added to mobile phone accounts; technical issues were resolved; and booster tutorials were provided, as needed. Research staff members were also readily available, by phone and in-person, to provide ongoing technical support and troubleshooting assistance to participants outside of regular weekly meetings, and detailed logs of these contacts were maintained. Participants were offered a $50 incentive for returning their original study phones at the end of the study period. (This allowed research staff to reset the returned phones and distribute
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 7
Author Manuscript
them to newly enrolled participants, thereby helping to contain costs.) Because an aim of the pilot study was to assess the feasibility of the mobile phone-based intervention delivery approach, a lenient phone replacement policy was adopted such that participants were not penalized for lost, stolen or broken phones, and missing phones would be replaced – multiple times, if necessary – as determined on a case-by-case basis. All study procedures, for both the parent trial and the pilot study, were approved by the appropriate IRB, and all participants provided written informed consent prior to study enrollment. Quantitative and Qualitative Data Collection
Author Manuscript
Participants’ demographics, as well as their drug use and drug treatment histories, were assessed at baseline with the Addiction Severity Index, a reliable, valid and widely-used measure of substance misuse and associated domains (ASI; McLellan, Luborsky, Cacciola, Griffith, Evans, Barr et al., 1985). An assessment battery including a structured Feedback Survey was administered at monthly follow-up time points for the duration of the 12-week study period (for a total of 3 Feedback Surveys per participant). Participants were compensated $50 for completing each assessment battery. Check-In program usage data were automatically saved in the mobile application’s back-end tracking system and manually downloaded onto the server during weekly research visits.
Author Manuscript
The Feedback Survey included 11 visual analog scale (VAS) items using a 0-100 scale in which 0 represented “not at all” and 100 represented “a great deal”. This measure was developed by our team from a version used in multiple prior studies evaluating technologybased interventions with substance-using populations. Six items asked both pilot study and parent trial participants how much they thought their assigned intervention: 1) was interesting; 2) was useful; 3) conveyed new information; 4) clarified any misunderstandings; 5) was easy to understand; and 6) was satisfactory. Five items asked pilot study participants only to rate the mobile Check-in program in terms of its: 1) ease of use; 2) likeability; 3) potential usefulness if more content were included; 4) helpfulness in reducing drug cravings; and 5) helpfulness in reducing the likelihood they would use drugs. (All items are summarized in Table 2). In addition, three open-ended items asked participants for their general comments on the Check-In program, suggestions for improvement and additional content areas for future expansions of the application.
Author Manuscript
Participants provided weekly urine samples to research staff, with sample collection randomly observed by a research associate of the same sex. Using point-of-care qualitative urine test cups, each urine sample was screened for: THC, cocaine; barbiturates; benzodiazepines; methamphetamine; opiates; methadone; propoxyphene; and oxycodone (Drug Check Drug Test Cup, Drug Test Systems, Dover, NH). Only the urine toxicology results for opioids were used for the primary outcome analyses reported here. Participants were compensated $10 for each urine sample provided. Qualitative feedback was systematically elicited from pilot study participants at each of the three monthly follow-up time points and during weekly study visits. These brief, semistructured interviews consisted of open-ended probes designed to elicit participants’
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 8
Author Manuscript
experiences using the application, including: general impressions of the program’s usefulness and appeal; the settings in which they had used the program; barriers and facilitators to use of the program; technical difficulties encountered; program features they liked the most and least; the extent to which they found the program helpful in managing their substance use; and suggestions for future versions of the program. Detailed notes summarizing this feedback, including, to the extent possible, direct quotations, were recorded by research staff during and/or immediately following interviews. Data Analysis
Author Manuscript
Descriptive statistics summarizing participants’ demographic, drug use and drug treatment characteristics, program usage metrics and responses to individual Feedback Survey VAS items (aggregated across all three time points) were prepared in Excel and SPSS. Because the Feedback Survey is not a validated and scored scale, each item was examined individually.
Author Manuscript
Key outcomes from the 25 pilot study participants were compared to the first 12 weeks of outcomes (a comparable time window) from the last 25 participants randomized to the standard condition of the parent trial (who were enrolled closest in time to pilot participants). For the 6 VAS items that were administered to both parent trial and pilot study participants, differences in mean VAS scores across conditions were assessed with t-tests conducted in SAS using the Mixed Procedure. In SPSS, the percent of participants retained in the study for twelve weeks was compared across the two conditions with the chi square technique, while a t-test was used to assess the difference in mean number of weeks of opioid abstinence. An opioid abstinent toxicology result was defined as a result negative for opiates, propoxyphene and oxycodone. For this analysis, missed urine tests were coded as opioid-positive. Since the study was underpowered for significance testing, as is typical of pilot studies, an effect size index (d) was also calculated for the opioid abstinence results, using the approach outlined by Cohen (1988), and a medium or large effect size was considered to indicate the potential efficacy of the combined computer TES-plus-mobile Check-In intervention.
Author Manuscript
Data from open-ended Feedback Survey items and field notes from brief semi-structured interviews were inductively coded for key themes, using an integrated code list to merge the two qualitative data sets. To help ensure the validity of the analysis, two team members with expertise in qualitative research methods (HG and YAM) developed the code list in collaboration, and then independently coded the entire dataset; the few discrepancies that emerged were resolved in consensus sessions. The number of times each theme was articulated by a different participant was calculated, allowing the themes to be arranged in rank order according to frequency.
Results Participants Pilot study participants (n=25) ranged in age from 21 to 61 (M=42.24, SD=12.54), and most were male (80%). They identified their race as Black/African American (44%), White
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 9
Author Manuscript
(40%), or Other (16%), with 20% endorsing Hispanic/Latino ethnicity. Over a third (36%) reported having hepatitis C and 8% reported being positive for HIV. The demographic characteristics of the last 25 participants assigned to the standard (MMTonly) condition of the parent trial were similar. Standard condition participants were 40.64 years old on average (R=22-63, SD=9.82) and 84% male. Forty percent identified their race as White, 36% as Black/African American, and 24% as Other, while 26% endorsed Hispanic/Latino ethnicity. Twenty percent reported being antibody-positive for hepatitis C, while 4% reported being HIV-positive.
Author Manuscript
Pilot study participants reported a mean of 17.28 years (range = 1-41; SD = 12.47) of regular use of their primary opioid, which, for the vast majority (92%), was heroin. (N.B. One year of “regular” use is defined, per the ASI, as use at least 3 times/week for at least 6 months). Poly-substance use was prevalent. In the 30 days prior to baseline, 44% reported using marijuana, 40% cocaine or crack and 36% sedatives, hypnotics or tranquilizers (largely benzodiazepines), while 16% reported using alcohol to intoxication. Participants reported a mean of 11.6 prior episodes of substance abuse treatment (range = 1-113; SD = 21.97), including all formal treatment modalities from inpatient detoxification and outpatient counseling to residential and medically-assisted treatment (MAT). All participants reported at least one previous episode of MMT, and 4 (16%) had previously been treated with buprenorphine. The mean number of prior MAT episodes was 7.92 (range = 1-61; SD = 11.9).
Author Manuscript
Participants varied in their baseline level of proficiency with mobile technologies. Virtually all participants owned mobile phones, but in many cases, these were traditional phones, not feature phones or smart phones similar to those used in this trial. Based on qualitative interview data, 9 participants had no prior experience with feature/smart phones, while 7 participants (generally those under age 35) were highly proficient in their use. Feasibility of the Mobile Phone-based Intervention Approach
Author Manuscript
Findings suggest that delivering intervention content to MMT clients via a mobile phone is a feasible approach. Almost all participants, including those of older age and those with limited technological experience, were interested in and willing to try the mobile intervention. Ninety-two-percent (23/25) used the Check-In program at least once, with the majority using the program far more frequently (as detailed below). Of the two participants who did not use the Check-In program, one, whose work schedule made it difficult for him to meet with study staff, never received a study mobile phone and was discharged from MMT less than one month after enrolling in the study; the second spent much of the study period in inpatient drug treatment. Nine participants (36%) required additional technical assistance (beyond the standard initial tutorial and brief weekly data download/check-in meetings) in order to successfully use the Nokia phone and/or the Check-In program. These 9 participants engaged with staff for supplementary coaching purposes for an average of 4.6 contacts (range = 2-10) over the 12week study (the vast majority of these contacts were in-person, given participants’ daily clinic attendance). These contacts were generally brief; over the life of the trial, this subset
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 10
Author Manuscript
of participants utilized an average of 45 minutes (range = 10-90) of staff time for technical support. The total staff time required for provision of technical support (beyond initial instruction and weekly maintenance activities) in this trial was 6 hours and 45 minutes (across two staff members).
Author Manuscript
At the end of the study period, 75% of participants returned the mobile phones that were provided to them for use during the study, with 44% returning their original study phone and the remainder returning a replacement phone. Despite our liberal phone replacement policy, the majority of participants (88%, 21/24) received only one or two study phones, providing further evidence of the feasibility of a mobile phone-based delivery method in an MMT population. It should be noted, however, that two participants had considerable difficulty retaining their study phones in operational condition and received multiple replacements for phones that had been lost, stolen or broken. Two additional participants who were living in “three-quarter” houses for individuals in substance abuse treatment reported that theft of personal property was a routine occurrence in these facilities which made it challenging to use the Check-In program at their residence without attracting unwanted attention. Participants’ Use of the Check-In Program
Author Manuscript
Usage metrics automatically tracked by the mobile intervention indicate that participants used the Check-In application fairly frequently, averaging 29.8 total module completions (across the two program modules combined) over the 12-week study, with an average of 8.8 completions for the Functional Analysis module and 21.1 completions for the SelfManagement module (see Table 1). The fact that participants used the Self-management module more frequently than the Functional Analysis module is, at least in part, an artifact of the application’s navigational structure such that, if participants responded negatively to the initial query asking if they had used opioids since their last login, but indicated that they had faced high-risk situations that might increase their likelihood of using opioids, they were automatically directed to the Self-Management module, bypassing the Functional Analysis module. Rates of program use were relatively stable during the three months of the study, averaging 8.9 (SD=6.9) total module completions in month one, 11.8 (SD=13.7) in month two and 9.2 (SD=14.1) in month three. Acceptability of the Mobile Check-In Program – Quantitative Findings
Author Manuscript
As presented in Table 2, pilot participants’ responses to Feedback Survey VAS items indicate that the mobile intervention was acceptable to, and perceived as useful by, participants. Participants reported high levels of satisfaction with (M=77.2; SD=17.8) and liking (M=75.6, SD=18.6) the program, and rated it positively on measures of usefulness (M=77.5, SD=14.4) and ease of use (M=80.7; SD=16.6). Participants also reported that the intervention contained a significant amount of new information (M=74.8; SD=18.4) and helped clarify misconceptions they had had about topics addressed in the program (M=65; SD=20.9). The program received moderate, but somewhat positive, scores for ease of understanding (M=58.5; SD=31.4). Moreover, participants felt that using the Check-In program helped reduce both their drug cravings (M=63.2; SD=27.8) and the likelihood they would use drugs (M=61.8; SD=27.6), and strongly agreed that the program would be useful if it offered additional topic areas of skills-building and support (M=82.3; SD=14.5).
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 11
Author Manuscript
Comparing pilot participants’ VAS ratings with those of the last 25 participants enrolled in the standard condition of the parent trial reveals that participants rated the reduced-standard treatment-plus-TES-and-Check-In program condition significantly better than standard MMT counseling on a number of indicators (see Table 2). Specifically, pilot participants’ VAS ratings were significantly higher for interestingness (t=4.48; p
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28. Published in final edited form as: Psychol Addict Behav. 2016 February ; 30(1): 1–11. doi:10.1037/adb0000128.
A Mixed-methods Evaluation of the Feasibility, Acceptability and Preliminary Efficacy of a Mobile Intervention for Methadone Maintenance Clients Honoria Guarino, Ph.D.1, Michelle Acosta, Ph.D.1, Lisa A. Marsch, Ph.D.2, Haiyi Xie, Ph.D.2, and Yesenia Aponte-Melendez, M.A.1
Author Manuscript
1
Center for Technology and Health, National Development and Research Institutes, New York, NY
2
Center for Technology and Behavioral Health, Dartmouth Center for Psychiatric Research, Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH
Abstract
Author Manuscript Author Manuscript
Despite the recent explosion of behavioral health interventions delivered on mobile devices, little is known about factors that make such applications practical, engaging and useful to their target audience. This study reports on the feasibility, acceptability and preliminary efficacy of a prototype of a novel, interactive mobile psychosocial intervention to reduce problematic drug use among clients in methadone maintenance treatment (MMT). A mixed-methods pilot study with new MMT clients (n=25) indicated that the mobile intervention approach was feasible, and that participants found the intervention highly acceptable and useful. On 100-point visual analog scale (VAS) items, participants reported high levels of liking the program (M=75.6), and endorsed it as useful (M=77.5), easy to use (M=80.7), and containing a significant amount of new information (M=74.8). When compared with 25 study participants who received standard MMT alone, pilot participants rated their treatment significantly higher in interestingness and usefulness, and were significantly more satisfied with their treatment. In qualitative interviews, participants reported using the mobile intervention in a range of settings, including during times of heightened risk for substance use, and finding it helpful in managing drug cravings. Additionally, pilot participants showed evidence of increased treatment retention and abstinence from illicit opioids (in terms of effect size) over a 3-month period relative to those in standard MMT, suggesting the application’s potential to enhance treatment outcomes. These promising findings suggest that an evidence-based mobile therapeutic tool addressing substance use may appeal to drug treatment clients and have clinical utility as an adjunct to formal treatment.
Corresponding Author: Honoria Guarino, Ph.D., Principal Investigator, National Development and Research Institutes, Inc., 71 West 23rd Street, 4th Floor, New York, NY 10010, Ph: (212) 845-4540, Fax: (917) 438-0894, [email protected]. Authors Note: In addition to her academic affiliation, Dr. Marsch is affiliated with HealthSim, LLC, the health-promotion software development organization that developed the technology-based intervention referenced in this manuscript. Dr. Marsch has worked extensively with her institutions to manage any potential conflict of interest. All research data collection, data management, and statistical analyses were conducted by individuals with no affiliation to HealthSim, LLC.
Guarino et al.
Page 2
Author Manuscript
Keywords technology-based interventions; mobile health; intervention acceptability; intervention engagement; psychosocial interventions for SUD
Author Manuscript Author Manuscript
Approximately 681,000 Americans reported using heroin in 2013, according to the latest National Survey on Drug Use and Health, a 69% increase from the 2002 rate (SAMHSA, 2014a), while more than 11 million people misused prescription opioids in 2013 (SAMHSA, 2014b). Given the recent increase in the number of Americans reporting nonmedical use of prescription opioids (SAMHSA, 2014a) – and emerging evidence that a growing proportion of nonmedical prescription opioid users are transitioning to heroin use (Frank, MateuGelabert, Guarino, Bennett, Wendel, & Teper, 2014; Jones, 2013; Mars, Bourgois, Karandinos, Montero, & Ciccarone, 2013; Mateu-Gelabert, Guarino, Jessell, & Teper, A., 2014) – the demand for effective treatment for opioid use disorders is high and likely to intensify in the near future. Yet the vast majority of those who could benefit from treatment do not receive it. This may be due to the limited availability of existing treatment programs and to a range of societal and individual-level barriers, including the stigma associated with identifying oneself as a substance user and difficulty attending a treatment program on a regular basis (Campbell, Muench & Nunes, in press; Cunningham, Sobel & Chow, 1993; Rapp, Xu, Carr, Lane, Wang, & Carlson, 2006). In addition, drug treatment programs face numerous barriers to providing comprehensive, evidence-based psychosocial treatment to their clients, including limited financial and staffing resources, high staff turnover and highvolume clinician-patient caseloads (D’Aunno & Vaughn, 1995; Kraft, Rothbard, Hadley, McLellan, & Asch, 1997). Although methadone maintenance treatment (MMT) has been demonstrated to be one of the most efficacious treatment modalities for opioid-dependence, MMT programs offer the lowest amount of psychosocial treatment and have the highest client-to-staff ratio relative to other types of drug treatment, such as outpatient counseling, inpatient rehabilitation and residential programs (Etheridge, Craddock, Dunteman, & Hubbard, 1995).
Author Manuscript
Technology-based interventions have strong potential to help address these barriers and to expand access to evidence-based psychosocial treatment both within MMT and in drug treatment more generally. Relative to traditional, face-to-face treatment modalities, technology-based interventions can offer greater privacy, convenience, reach and costeffectiveness Cucciare, Weingardt, Greene, & Hoffman, 2012; Cucciare, Weingardt & Humphreys, 2009; (Marsch, Carroll & Kiluk, 2014). These benefits may be especially relevant in MMT, where patient attendance burden is high and staff and financial resources are strained. In addition, technology-based therapeutic tools, especially those utilizing mobile technologies, have the potential to deliver personalized, on-demand treatment to individuals with problematic substance use, providing support in real-world contexts, at times and in situations when it may be most needed (Marsch, 2012). There is a nascent, but growing, evidence base demonstrating the efficacy of technologybased interventions for substance use disorders (SUDs), especially when they are used as adjuncts to traditional, face-to-face treatment modalities. Meta-analyses and systematic
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 3
Author Manuscript
reviews of technology-based interventions for SUDs have consistently found that such tools are associated with significantly better outcomes than minimal or no-treatment controls and may be equivalent in efficacy to other evidence-based psychosocial treatments, such as clinician-delivered Cognitive Behavioral Therapy and Motivation Enhancement Therapy (Campbell, Muench & Nunes, in press; Litvin, Abrantes & Brown, 2013; Moore, Fazzino, Garnet, Cutter, & Barry, 2011; Quanbeck, Chih, Isham, Johnson, & Gustafson, 2014). However, the preponderance of research in this area has focused on web-delivered interventions for problem drinking and smoking cessation, while fewer technology-based interventions have been developed and evaluated for other SUDs, including opioid dependence. The research literature is also sparser for web-based tools that are accessible on mobile devices as compared to desktop-based web applications (Moore et al., 2011).
Author Manuscript Author Manuscript
Multiple studies and reviews of technology-based tools for a range of behavioral health conditions, including SUDs, have generally found significant correlations between use of, or “engagement” with, an intervention and positive outcomes, with individuals who display higher rates of intervention use being most likely to evince successful treatment outcomes (Bennett & Glasgow, 2009; Freyne, Saunders, Brindal, Berkovsky, & Smith, 2012; Gainsbury & Blaszczynski, 2011; Glasgow, Christiansen, Kurz, King, Woolley, Faber et al., 2011; Gustafson, McTavish, Chih, Atwood, Johnson, Boyle et al., 2014; Litvin, Abrantes, & Brown, 2013; Strecher, McClure, Alexander, Chakraborty, Nair, Konkel et al., 2008). However, effectively motivating members of a target population to engage with technologybased tools, and particularly to sustain engagement over time, remains a challenge. Similar to standard face-to-face treatments, user attrition has been a salient problem in research evaluations of technology-based interventions, with a 50% attrition rate in the active intervention period a common finding (Donkin & Glozier, 2012; Freyne et al., 2012; Kelders, Kok, Ossebaard, & Van Gemert-Pijnen, 2012; Litvin, Abrantes & Brown, 2013). Study retention and intervention adherence rates have generally been worse in purely technology-based trials that include no face-to-face interaction with research or clinical staff. While this may indicate that factors beyond the acceptability of the intervention itself may be involved, it does suggest the important role of human/provider contact in the effective implementation of technology-based interventions. Together, these findings underscore the importance of systematically examining the acceptability and appeal of an intervention from the end user’s perspective, as individuals are unlikely to use tools they do not like (KayLambkin, Baker, Lewin, & Carr, 2011). In-depth investigations of users’ experiences with technology-based behavioral health tools may help identify program features that reliably promote sustained engagement, thus facilitating the development of tools that are maximally appealing, engaging and effective.
Author Manuscript
Initial research in this area has identified certain features of technology-based behavioral health tools and their implementation that are associated with increased engagement, including individual-level personalization of content, “push reminders” (e.g., automated text messages or emails) to promote use of a tool, provision of incentives for use of a tool, and integration of human/social support and encouragement (Mohr, Cuijpers & Lehman, 2011; Strecher et al., 2008). However, extant process research on participants’ experiences with technology-based interventions for SUDs is quite limited, and studies applying qualitative or mixed methods to this issue are particularly scarce. A qualitative or mixed-methods Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 4
Author Manuscript
approach to process research is especially well-suited for gaining insight into participants’ experiences and perspectives, understanding how local contexts influence these experiences, and uncovering salient themes that may be unanticipated at the start of an investigation. More research of this type is needed to deepen our understanding of the specific intervention features and components that best promote acceptability and sustained engagement with technology-based interventions for SUDs.
Author Manuscript
The current study was conducted within the context of a larger randomized, controlled trial to evaluate the effectiveness of a web-based psychosocial program called the Therapeutic Education System (TES; Bickel, Marsch, Buchalter, & Badger, 2008) when delivered to opioid-dependent individuals in MMT. TES is an interactive, customized program that is theoretically grounded in the evidence-based behavior therapy program for substance abuse known as the Community Reinforcement Approach. This approach aims to train individuals in skills and behaviors to help reduce problematic substance use and HIV/HCV risk behaviors, improve employment status and family/social relations, and increase nonsubstance-related recreational activities (Budney & Higgins, 1998). TES has been demonstrated to significantly improve treatment outcomes when delivered to individuals in a range of community-based substance abuse treatment settings, including MMT (Bickel, Marsch, Buchalter & Badger, 2008; Brooks, Ryder, Carise, & Kirby, 2010; Marsch, Grabinski, Bickel, Desrosiers, Guarino, Muehlbach et al., 2011; Marsch, Guarino, Acosta, Aponte-Melendez, Cleland, Grabinski et al., 2014; Campbell, Nunes, Matthews, Stitzer, Miele, Polsky et al., 2015).
Author Manuscript
As a supplement to this parent trial (Marsch et al., 2014), our team developed a prototype version of an interactive, mobile phone-based psychosocial intervention, based on key elements of the Community Reinforcement Approach and Cognitive Behavior Therapy, to promote skills acquisition and reduce illicit drug use among MMT clients. Designed to be theoretically compatible with TES in its focus on practical skills-acquisition and selfmanagement of drug use, this mobile application, called the Check-In Program, was conceptualized as a “treatment extender”, enabling provision of flexible, on-demand therapeutic support to participants in settings and at times of their choosing.
Author Manuscript
The present mixed-methods pilot study reports on the feasibility, acceptability and preliminary efficacy of this novel application when delivered to a sample of participants in conjunction with web-based TES. We hoped to learn whether MMT clients would be interested in and readily able to use the mobile application, whether they would be able to retain their study phones in working condition for the duration of the intervention period and whether they would find the tool helpful in reducing or abstaining from illicit opioid use. To evaluate the application’s potential to enhance treatment outcomes, we also assessed participants’ opioid use (via urine toxicology) and treatment retention relative to a group receiving standard MMT alone.
The Check-In Program The prototype version of the Check-In Program evaluated in this study consisted of two skills-based modules, a Functional Analysis module which aimed to help individuals
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 5
Author Manuscript
identify their patterns of problematic substance use and the specific triggers to substance use they face in their day-to-day lives, and a Self-Management module which assisted individuals in developing a plan to manage the triggers to substance use identified in the Functional Analysis module. Each module contained explanatory text defining key concepts (e.g., “triggers”, “self-management plan”), followed by an interactive exercise. A standardized, daily text message prompt (“Don’t forget to use the Check-In Program to help you stay on track.”) was sent to each program user’s mobile phone to encourage use of the program.
Author Manuscript
Upon first accessing the program, users were required to complete both modules in their entirety at least once. Progress through the application at each subsequent use of the program was tailored based on responses to recurring queries presented at each login asking if the individual had used any opioids (besides his/her daily methadone dose) or been exposed to any high-risk situations that could lead to opioid use since his/her last login. Affirmative responses led the user into the exercise portion of the Functional Analysis or Self-Management module, respectively, while negative responses allowed the user to either log out of the program or electively complete either module’s exercise. At any time, users could elect to review the introductory didactic modular content.
Author Manuscript
In the Functional Analysis exercise, participants were asked to recall their most recent episode of non-MMT opioid use and then to identify their most salient triggers, or emotional, cognitive or environmental cues that increase the likelihood of engaging in problematic opioid use, including both internal triggers (e.g., certain thoughts or feelings) and external triggers (e.g., certain people, places or situations). Participants were then asked to identify the short-term and long-term consequences of their opioid use, both positive and negative. For all interactive items within both modules, users had the option of selecting a response from a checklist provided or selecting “other” and typing in their own response using the phone’s touchscreen keyboard.
Author Manuscript
In the Self-Management exercise, participants were asked to select one of their selfidentified triggers and were guided through successive steps to develop a plan for managing this trigger in the future without resorting to opioid use. Self-management plans could involve strategies such as re-arranging one’s daily routine or environment to avoid the trigger or identifying alternative responses to the trigger that do not involve using opioids. In order to help motivate behavior change and prepare participants for potential challenges in managing triggers, participants were then asked to identify the likely positive and negative consequences of implementing their chosen plan. Finally, participants were asked to rate the difficulty of their plan, using a scrollbar to select a difficulty rating ranging from 1 (least difficult) to 10 (most difficult).
Method Parent Study Procedures In the parent trial, opioid-dependent adults (n=160) within their first 30 days of entering outpatient MMT at the study site, a large New York City MMT program, were randomized to receive either 12 months of 1) standard treatment or 2) a reduced schedule of standard
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 6
Author Manuscript
treatment plus biweekly sessions using TES. Eligible participants had to be at least 18 years of age, meet DSM criteria for opioid dependence (required for entry into MMT), have sufficient English language ability to understand the study assessments and intervention content, and be enrolled during the first month of their current MMT episode. Details of the parent trial have been published elsewhere (Marsch et al., 2014; Acosta, Marsch, Xie, Guarino, & Aponte-Melendez, 2013).
Author Manuscript
Standard treatment included daily administration of a therapeutic dose of methadone (typically ranging from 80-120 mg/day). MMT was initiated immediately at treatment entry and usually required clinic attendance six days per week. Standard treatment also included the routine substance abuse counseling provided by the MMT program, i.e., meeting with one’s assigned counselor for 30-60-minute individual sessions once weekly during the first month of treatment and once monthly thereafter. Participants randomized to the reducedstandard treatment-plus-TES condition received the same methadone treatment plus standard counseling as those in the standard condition, except that one half of each counseling session was spent with their counselor, while the other half was spent using TES in an on-site computer lab. Pilot Study Procedures At the conclusion of participant enrollment in the parent trial, an additional sample of MMT clients (n=25) meeting the above eligibility criteria were enrolled and received reducedstandard treatment-plus-TES, as well as access to the mobile Check-In program, for the 12week pilot study intervention period.
Author Manuscript
As a supplement to their biweekly computer-based TES sessions, each pilot study participant was provided with a Nokia feature phone preloaded with the Check-In application for use throughout the intervention period. Participants were given an initial, one-on-one tutorial (approximately 30 minutes long) in which research staff explained how to use key features of the Nokia phone and how to access, securely login to and complete the Check-In program.
Author Manuscript
Participants were encouraged to use the Check-In program at any time, as frequently as they wished, particularly in settings outside of their treatment program. Because the prototype Check-In program was designed as a native (not web-based) application, no wireless connection was necessary to use the mobile intervention. After the initial tutorial, participants were asked to meet with research staff on a weekly basis. During these sessions (about 15 minutes long, on average): Check-In program usage data were downloaded from the phones onto a central server (as there was no viable way to remotely extract data from the native application); prepaid minutes were added to mobile phone accounts; technical issues were resolved; and booster tutorials were provided, as needed. Research staff members were also readily available, by phone and in-person, to provide ongoing technical support and troubleshooting assistance to participants outside of regular weekly meetings, and detailed logs of these contacts were maintained. Participants were offered a $50 incentive for returning their original study phones at the end of the study period. (This allowed research staff to reset the returned phones and distribute
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 7
Author Manuscript
them to newly enrolled participants, thereby helping to contain costs.) Because an aim of the pilot study was to assess the feasibility of the mobile phone-based intervention delivery approach, a lenient phone replacement policy was adopted such that participants were not penalized for lost, stolen or broken phones, and missing phones would be replaced – multiple times, if necessary – as determined on a case-by-case basis. All study procedures, for both the parent trial and the pilot study, were approved by the appropriate IRB, and all participants provided written informed consent prior to study enrollment. Quantitative and Qualitative Data Collection
Author Manuscript
Participants’ demographics, as well as their drug use and drug treatment histories, were assessed at baseline with the Addiction Severity Index, a reliable, valid and widely-used measure of substance misuse and associated domains (ASI; McLellan, Luborsky, Cacciola, Griffith, Evans, Barr et al., 1985). An assessment battery including a structured Feedback Survey was administered at monthly follow-up time points for the duration of the 12-week study period (for a total of 3 Feedback Surveys per participant). Participants were compensated $50 for completing each assessment battery. Check-In program usage data were automatically saved in the mobile application’s back-end tracking system and manually downloaded onto the server during weekly research visits.
Author Manuscript
The Feedback Survey included 11 visual analog scale (VAS) items using a 0-100 scale in which 0 represented “not at all” and 100 represented “a great deal”. This measure was developed by our team from a version used in multiple prior studies evaluating technologybased interventions with substance-using populations. Six items asked both pilot study and parent trial participants how much they thought their assigned intervention: 1) was interesting; 2) was useful; 3) conveyed new information; 4) clarified any misunderstandings; 5) was easy to understand; and 6) was satisfactory. Five items asked pilot study participants only to rate the mobile Check-in program in terms of its: 1) ease of use; 2) likeability; 3) potential usefulness if more content were included; 4) helpfulness in reducing drug cravings; and 5) helpfulness in reducing the likelihood they would use drugs. (All items are summarized in Table 2). In addition, three open-ended items asked participants for their general comments on the Check-In program, suggestions for improvement and additional content areas for future expansions of the application.
Author Manuscript
Participants provided weekly urine samples to research staff, with sample collection randomly observed by a research associate of the same sex. Using point-of-care qualitative urine test cups, each urine sample was screened for: THC, cocaine; barbiturates; benzodiazepines; methamphetamine; opiates; methadone; propoxyphene; and oxycodone (Drug Check Drug Test Cup, Drug Test Systems, Dover, NH). Only the urine toxicology results for opioids were used for the primary outcome analyses reported here. Participants were compensated $10 for each urine sample provided. Qualitative feedback was systematically elicited from pilot study participants at each of the three monthly follow-up time points and during weekly study visits. These brief, semistructured interviews consisted of open-ended probes designed to elicit participants’
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 8
Author Manuscript
experiences using the application, including: general impressions of the program’s usefulness and appeal; the settings in which they had used the program; barriers and facilitators to use of the program; technical difficulties encountered; program features they liked the most and least; the extent to which they found the program helpful in managing their substance use; and suggestions for future versions of the program. Detailed notes summarizing this feedback, including, to the extent possible, direct quotations, were recorded by research staff during and/or immediately following interviews. Data Analysis
Author Manuscript
Descriptive statistics summarizing participants’ demographic, drug use and drug treatment characteristics, program usage metrics and responses to individual Feedback Survey VAS items (aggregated across all three time points) were prepared in Excel and SPSS. Because the Feedback Survey is not a validated and scored scale, each item was examined individually.
Author Manuscript
Key outcomes from the 25 pilot study participants were compared to the first 12 weeks of outcomes (a comparable time window) from the last 25 participants randomized to the standard condition of the parent trial (who were enrolled closest in time to pilot participants). For the 6 VAS items that were administered to both parent trial and pilot study participants, differences in mean VAS scores across conditions were assessed with t-tests conducted in SAS using the Mixed Procedure. In SPSS, the percent of participants retained in the study for twelve weeks was compared across the two conditions with the chi square technique, while a t-test was used to assess the difference in mean number of weeks of opioid abstinence. An opioid abstinent toxicology result was defined as a result negative for opiates, propoxyphene and oxycodone. For this analysis, missed urine tests were coded as opioid-positive. Since the study was underpowered for significance testing, as is typical of pilot studies, an effect size index (d) was also calculated for the opioid abstinence results, using the approach outlined by Cohen (1988), and a medium or large effect size was considered to indicate the potential efficacy of the combined computer TES-plus-mobile Check-In intervention.
Author Manuscript
Data from open-ended Feedback Survey items and field notes from brief semi-structured interviews were inductively coded for key themes, using an integrated code list to merge the two qualitative data sets. To help ensure the validity of the analysis, two team members with expertise in qualitative research methods (HG and YAM) developed the code list in collaboration, and then independently coded the entire dataset; the few discrepancies that emerged were resolved in consensus sessions. The number of times each theme was articulated by a different participant was calculated, allowing the themes to be arranged in rank order according to frequency.
Results Participants Pilot study participants (n=25) ranged in age from 21 to 61 (M=42.24, SD=12.54), and most were male (80%). They identified their race as Black/African American (44%), White
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 9
Author Manuscript
(40%), or Other (16%), with 20% endorsing Hispanic/Latino ethnicity. Over a third (36%) reported having hepatitis C and 8% reported being positive for HIV. The demographic characteristics of the last 25 participants assigned to the standard (MMTonly) condition of the parent trial were similar. Standard condition participants were 40.64 years old on average (R=22-63, SD=9.82) and 84% male. Forty percent identified their race as White, 36% as Black/African American, and 24% as Other, while 26% endorsed Hispanic/Latino ethnicity. Twenty percent reported being antibody-positive for hepatitis C, while 4% reported being HIV-positive.
Author Manuscript
Pilot study participants reported a mean of 17.28 years (range = 1-41; SD = 12.47) of regular use of their primary opioid, which, for the vast majority (92%), was heroin. (N.B. One year of “regular” use is defined, per the ASI, as use at least 3 times/week for at least 6 months). Poly-substance use was prevalent. In the 30 days prior to baseline, 44% reported using marijuana, 40% cocaine or crack and 36% sedatives, hypnotics or tranquilizers (largely benzodiazepines), while 16% reported using alcohol to intoxication. Participants reported a mean of 11.6 prior episodes of substance abuse treatment (range = 1-113; SD = 21.97), including all formal treatment modalities from inpatient detoxification and outpatient counseling to residential and medically-assisted treatment (MAT). All participants reported at least one previous episode of MMT, and 4 (16%) had previously been treated with buprenorphine. The mean number of prior MAT episodes was 7.92 (range = 1-61; SD = 11.9).
Author Manuscript
Participants varied in their baseline level of proficiency with mobile technologies. Virtually all participants owned mobile phones, but in many cases, these were traditional phones, not feature phones or smart phones similar to those used in this trial. Based on qualitative interview data, 9 participants had no prior experience with feature/smart phones, while 7 participants (generally those under age 35) were highly proficient in their use. Feasibility of the Mobile Phone-based Intervention Approach
Author Manuscript
Findings suggest that delivering intervention content to MMT clients via a mobile phone is a feasible approach. Almost all participants, including those of older age and those with limited technological experience, were interested in and willing to try the mobile intervention. Ninety-two-percent (23/25) used the Check-In program at least once, with the majority using the program far more frequently (as detailed below). Of the two participants who did not use the Check-In program, one, whose work schedule made it difficult for him to meet with study staff, never received a study mobile phone and was discharged from MMT less than one month after enrolling in the study; the second spent much of the study period in inpatient drug treatment. Nine participants (36%) required additional technical assistance (beyond the standard initial tutorial and brief weekly data download/check-in meetings) in order to successfully use the Nokia phone and/or the Check-In program. These 9 participants engaged with staff for supplementary coaching purposes for an average of 4.6 contacts (range = 2-10) over the 12week study (the vast majority of these contacts were in-person, given participants’ daily clinic attendance). These contacts were generally brief; over the life of the trial, this subset
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 10
Author Manuscript
of participants utilized an average of 45 minutes (range = 10-90) of staff time for technical support. The total staff time required for provision of technical support (beyond initial instruction and weekly maintenance activities) in this trial was 6 hours and 45 minutes (across two staff members).
Author Manuscript
At the end of the study period, 75% of participants returned the mobile phones that were provided to them for use during the study, with 44% returning their original study phone and the remainder returning a replacement phone. Despite our liberal phone replacement policy, the majority of participants (88%, 21/24) received only one or two study phones, providing further evidence of the feasibility of a mobile phone-based delivery method in an MMT population. It should be noted, however, that two participants had considerable difficulty retaining their study phones in operational condition and received multiple replacements for phones that had been lost, stolen or broken. Two additional participants who were living in “three-quarter” houses for individuals in substance abuse treatment reported that theft of personal property was a routine occurrence in these facilities which made it challenging to use the Check-In program at their residence without attracting unwanted attention. Participants’ Use of the Check-In Program
Author Manuscript
Usage metrics automatically tracked by the mobile intervention indicate that participants used the Check-In application fairly frequently, averaging 29.8 total module completions (across the two program modules combined) over the 12-week study, with an average of 8.8 completions for the Functional Analysis module and 21.1 completions for the SelfManagement module (see Table 1). The fact that participants used the Self-management module more frequently than the Functional Analysis module is, at least in part, an artifact of the application’s navigational structure such that, if participants responded negatively to the initial query asking if they had used opioids since their last login, but indicated that they had faced high-risk situations that might increase their likelihood of using opioids, they were automatically directed to the Self-Management module, bypassing the Functional Analysis module. Rates of program use were relatively stable during the three months of the study, averaging 8.9 (SD=6.9) total module completions in month one, 11.8 (SD=13.7) in month two and 9.2 (SD=14.1) in month three. Acceptability of the Mobile Check-In Program – Quantitative Findings
Author Manuscript
As presented in Table 2, pilot participants’ responses to Feedback Survey VAS items indicate that the mobile intervention was acceptable to, and perceived as useful by, participants. Participants reported high levels of satisfaction with (M=77.2; SD=17.8) and liking (M=75.6, SD=18.6) the program, and rated it positively on measures of usefulness (M=77.5, SD=14.4) and ease of use (M=80.7; SD=16.6). Participants also reported that the intervention contained a significant amount of new information (M=74.8; SD=18.4) and helped clarify misconceptions they had had about topics addressed in the program (M=65; SD=20.9). The program received moderate, but somewhat positive, scores for ease of understanding (M=58.5; SD=31.4). Moreover, participants felt that using the Check-In program helped reduce both their drug cravings (M=63.2; SD=27.8) and the likelihood they would use drugs (M=61.8; SD=27.6), and strongly agreed that the program would be useful if it offered additional topic areas of skills-building and support (M=82.3; SD=14.5).
Psychol Addict Behav. Author manuscript; available in PMC 2016 June 28.
Guarino et al.
Page 11
Author Manuscript
Comparing pilot participants’ VAS ratings with those of the last 25 participants enrolled in the standard condition of the parent trial reveals that participants rated the reduced-standard treatment-plus-TES-and-Check-In program condition significantly better than standard MMT counseling on a number of indicators (see Table 2). Specifically, pilot participants’ VAS ratings were significantly higher for interestingness (t=4.48; p
