Issues in Mental Health Nursing, 35:517–525, 2014 Copyright © 2014 Informa Healthcare USA, Inc. ISSN: 0161-2840 print / 1096-4673 online DOI: 10.3109/01612840.2014.888601
Challenges of Implementing a Medication Management Evidence-Based Practice in a Community Mental Health Setting: Results of a Qualitative Study Peggy El-Mallakh, RN, PhD University of Kentucky, College of Nursing, Lexington, Kentucky, USA
Patricia B. Howard, PhD, RN, FAAN University of Kentucky, College of Nursing, Lexington, Kentucky, USA
Gary R. Bond, PhD Dartmouth College, Geisel School of Medicine, Hanover, New Hampshire, USA
Autumn P. Roque, BSN, RN University of Kentucky, College of Nursing, Lexington, Kentucky, USA
The Medication Management Approaches in Psychiatry (MedMAP) is a medication management evidence-based practice (EBP) to guide the use of psychotropic medications in the treatment of schizophrenia. This qualitative study examined facilitators and barriers to implementing MedMAP in community mental health treatment settings. Audio-taped qualitative interviews were conducted with practitioners and administrators involved in a MedMAP implementation project conducted in six community mental health centers. Data analysis was conducted using thematic analysis of transcribed interviews. Findings indicate that facilitators to MedMAP implementation included practitioner recognition of the value of MedMAP, consumer involvement, collaboration, continuity of care, and fidelity assessments. Barriers to MedMAP implementation included problematic technology, work flow issues, lack of flexibility in prescribers’ ability to implement MedMAP guidelines, regulatory and financial barriers, and consumer insurance status. Recommendations for improving future implementation efforts of MedMAP emphasize technological readiness, development of innovative models of care delivery, an emphasis on treatment guided by outcomes, and active leadership to promote EBPs within organizations and academic settings.
Behavioral health administrators, program managers, and researchers are giving increased attention to factors influencing the implementation of evidence-based practices (EBPs) to facilitate the timely translation of research into practice. One major study, the National Implementing Evidence-Based Practices Project (NIEBP) (McHugo et al., 2007), was an ambitious Address correspondence to Peggy El-Mallakh, University of Kentucky, College of Nursing, 547 College of Nursing, Lexington, KY 40536-0232 USA. E-mail: [email protected]
project to evaluate the use of systematic strategies to promote high quality implementation of EBPs. Funded in part by the Substance Abuse and Mental Health Services Administration (SAMHSA), this project involved implementation efforts for five psychosocial EBPs: (1) Illness Management and Recovery; (2) Supported Employment; (3) Integrated Dual Diagnosis Treatment; (4) Family Psychoeducation; and (5) Assertive Community Treatment. These were implemented in 53 community mental health treatment settings in eight states across the United States (Bond, McHugo, Becker, Rapp, & Whitley, 2008; McHugo et al., 2007). In a companion study to the NIEBP, a medication management EBP, the Medication Management Approaches in Psychiatry (MedMAP), was implemented in six community mental health treatment settings in one state (Howard et al., 2009). Recognizing the lack of user-friendly materials available to agencies seeking to implement an EBP, the NIEBP team created toolkits for each EBP consisting of materials to facilitate implementation. The toolkits included practitioner workbooks, research articles, introductory and instructional videotapes, and PowerPoint lectures. The toolkits also included program-level fidelity scales to measure how closely programs adhered to the EBP program models (McHugo et al., 2007). A key working hypothesis for the NIEBP was that systematic monitoring of fidelity promotes higher quality implementation (Bond, Drake, McHugo, Rapp, & Whitley, 2009). Efforts to disseminate and implement the SAMHSA EBPs in mental health service settings have produced a wide variety of results. The Supported Employment and Assertive Community Treatment EBPs were implemented with high fidelity;
517
518
P. EL-MALLAKH ET AL.
however, more moderate levels of fidelity were seen in implementation efforts for Illness Management, Integrated Dual Diagnosis Treatment (McHugo et al., 2007), and MedMAP (Howard et al., 2009). In addition, at a six-year follow-up of the 53 sites participating in the NIEBP Project, 47% continued to use the EBPs, 16% discontinued use and restarted the EBPs at a later date, and 37% of the implementing sites discontinued the use of these EBPs within the six years following full implementation (Bond et al., 2014). These inconsistent levels of adoption and sustainability have prompted researchers to examine contextual factors that influenced implementation of these EBPs within the participating sites (Brunette et al., 2008; Mancini et al., 2009; Marshall, Rapp, Becker, & Bond, 2008; Whitley, Gingerich, Lutz, & Mueser, 2009). The purpose of this article is to describe findings from a qualitative study that investigated facilitators and barriers to successful implementation of MedMAP and discuss implications of the findings for future implementation efforts.
LITERATURE REVIEW The Texas Medication Algorithm Project (TMAP), the precursor to MedMAP, was originally developed in 1995 in a joint partnership of the University of Texas and the Texas Department of Mental Health and Mental Retardation Services to facilitate the use of medication management best practices in routine clinical care (Chiles et al., 1999; Howard et al., 2009). The foundation of MedMAP consists of medication algorithms and guidelines that provide prescribers with strategies and tactics in the use of antipsychotic medications (Gilbert et al., 1998; Miller et al., 1999). Principles of medication management are incorporated into seven domains in MedMAP. These domains include (1) adequate information about diagnosis and treatment, (2) measurement and use of outcomes to guide decision making about medication regimens, (3) reduction of medication burden and side effects, (4) use of appropriate dosing and side effect monitoring, (5) identification of treatment refractory patients, (6) patient and family involvement in treatment planning, and (7) coordination of the treatment team (Howard et al., 2009). In addition, MedMAP stresses that serious medical illnesses, such as schizophrenia, are chronic and life-long, and acknowledges that consumers may have several prescribers treating them over the course of their illnesses. Therefore, optimal medication management requires prescriber access to thorough and comprehensive documentation on illness and medication history; rationales for medication management strategies; and information on medication safety, effectiveness, and tolerability of all medications that have been prescribed throughout a consumer’s illness. The MedMAP EBP guidelines were consolidated into an Implementation Toolkit that included a wide variety of resources, such as workbooks; educational resources for prescribers, consumers, and family members; and supporting documentation of research evidence on medication management. In addition, the Implementation Toolkit included symptom rating scales and
scales for measuring fidelity at the prescriber and organizational levels. MedMAP developers also provided training and consultation via telephone conferences for staff at implementing sites. MedMAP has been adapted over time by the original developers and is currently referred to as the Medication Treatment, Evaluation, and Management (MedTEAM) EBP (SAMHSA, 2011). PROJECT OVERVIEW A mixed-methods study design was used to implement MedMAP in six community mental health centers (CMHCs) in a south-central state of the US. Four CMHCs were located in an urban area of the state, and served 1,154 mental health consumers diagnosed with schizophrenia. The two remaining CMHCs were located in a rural area of the state, and served 226 mental health consumers diagnosed with schizophrenia. The participating CMHCs offered several specialized services in the treatment of schizophrenia, including medication monitoring, case management, individual therapy, and illness management/dual diagnosis group counseling. MedMAP Fidelity Scale Adherence to MedMAP is measured with two fidelity scales, one to measure organizational practices and the other to measure prescriber practices (Taylor et al., 2009). The latter scale is a 23-item scale based exclusively on chart reviews. The original psychometric study found large gaps in routine practice, including poor documentation of medication history and infrequent monitoring of symptoms and side effects. Fidelity assessments to measure prescriber adherence to the EBP were an integral aspect of the EBP implementation protocol (Howard et al., 2009). A Fidelity Assessment Team was responsible for conducting medical records reviews to measure prescriber fidelity to MedMAP. Thirty randomly-selected medical records were reviewed at each site visit at approximately four-month intervals, for a total of 900 medical records throughout the three-year duration of the project. Detailed findings related to fidelity outcomes have been reported elsewhere (Howard et al., 2009). To summarize, overall prescriber fidelity to MedMAP implementation was moderate, although it increased significantly between baseline and posttraining fidelity assessments. Baseline, pre-training fidelity had a mean score of 2.93 (SD = .35), and post-training fidelity mean scores ranged from 3.12–3.43 in subsequent follow up fidelity assessments. These scores were based on a 5-point behaviorally anchored scale, with scores of 4 or more representing high fidelity, 3–4 representing moderate fidelity, and below 3 representing low fidelity (McHugo et al., 2007). Findings also indicated that there was much variability in fidelity to the individual critical ingredients of MedMAP. For example, fidelity was consistently high in the areas of documentation of current practice, timely documentation of practice, use of appropriate dose ranges, and patient and family involvement in treatment
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE
planning (Howard et al., 2009). In contrast, consistently low fidelity was found in documentation of treatment outcomes, treatment guided by outcomes, side effect monitoring, and availability of adequate information about patient illness and medication history. A further MedMAP study has examined how the process of collecting and storing medication records affects the efficiency and completeness of MedMAP fidelity reviews. This study compared electronic medical records to paper records and found that electronic medical records were 40% more complete and 20% faster to retrieve (Tsai & Bond, 2008). METHODS A qualitative study was conducted at the conclusion of the MedMAP implementation study to examine the perceptions of key stakeholders on the project about the facilitators and barriers to implementation. Sample The sample consisted of eight practitioners and administrators who participated in the MedMAP implementation project. Inclusion criteria were being employed as a prescriber, program director, or administrator on the project and being willing to participate in audio-taped interviews. All participants provided informed consent prior to data collection. The Medical Institutional Review Board at the sponsoring university approved all study procedures. Procedures to ensure participant confidentiality were followed throughout the study. Data Collection Semi-structured interviews were conducted with participants to determine their perspectives on barriers and facilitators to MedMAP implementation at the participating sites. Members of the Fidelity Assessment Team conducted the interviews; interviews were audio-taped and transcribed for the purposes of data analysis, and lasted approximately 45–60 minutes. Data analysis and interpretation were conducted using classic guidelines described by Miles and Huberman (1994). Data analysis was based on three linked subprocesses, including data reduction, data display, and drawing/verifying conclusions (Miles & Huberman, 1994). In this study, data reduction was addressed by clustering data from the narrative accounts of participants and identifying preliminary themes. Data were then displayed in a matrix with narrative accounts linked to each preliminary theme. Themes were refined as additional data were compiled and the attributes and characteristics of each theme were identified. Finally, the conclusion drawing/verification phase consisted of condensing the essential themes to depict the participants’ experiences of MedMAP implementation. Data Trustworthiness Multiple strategies recommended by Lincoln and Guba (1985) were used to ensure data trustworthiness and to pro-
519
mote credibility of the study findings. For example, triangulation of data sources was paramount; quantitative data were obtained from fidelity assessments on 900 medical records over the course of three years to measure prescriber adherence to MedMAP. In addition, prolonged engagement was addressed with the researcher’s immersion in the study sites for the threeyear duration of the study. This immersion in the sites also facilitated persistent observation of the EBP implementation activities and issues. Debriefing among members of the research team was regularly conducted. In addition, peer review was a consistent activity of the research team during the process of data analysis and interpretation.
FINDINGS The interviews provided information about the barriers and facilitators that prescribers and organizations face when implementing complex practice changes that are typical of EBPs. In general, participants recognized the need for the practice change, yet simultaneously identified several individual and system-level factors that interfered with implementation. Factors that facilitated implementation were the perceived value of the MedMAP among prescribes at the participating sites, consumer involvement in MedMAP implementation, collaboration, and fidelity assessments. Participants also reported a number of factors that interfered with MedMAP implementation at the participating sites. These were technology, work flow, patient barriers, regulatory barriers, lack of flexibility, and sustainability issues. Perceived Value MedMAP was intended to promote consistency in the delivery and documentation of care, and clinicians valued the approach of “getting everyone on the same page” when prescribing medications. Participants reported that they recognized the benefits of using a formal best practice guideline for medication management in schizophrenia, which promoted their willingness to become proficient in the use of MedMAP. They also supported the use of MedMAP because it “made good clinical sense,” was based on the best available evidence, and had “immediate application” in practice. As one respondent said, “It gave structure to the approach that many of us thought made sense to begin with.” Participants particularly valued having guidelines for treatment refractory patients, which recommended the use of clozapine “instead of polypharmacy.” Another participant reported that MedMAP greatly improved documentation, making it “more consistent and thorough.” The emphasis on outcomes was viewed as clinically beneficial: “Doing an intervention and then getting objective data to see the effect of your intervention, and letting that guide your future intervention . . . that’s a perfectly sensible approach that’s hard to argue with.” Participants reported increased support for MedMAP when they observed clinical improvements in their
520
P. EL-MALLAKH ET AL.
patients: “When prescribers saw good outcomes, they became champions of MedMAP.” Consumer Involvement Participants reported that their enthusiasm for using MedMAP was enhanced when they observed consumer involvement in the implementation process. The implementation process included group sessions that focused on educating consumers about use of MedMAP in clinical care. One participant reported: They [consumers] were very excited about it and wanted to know more. Some consumers have become peer support specialists because of their participation in the MedMAP groups. They were willing to go out and share what they learned with others, and the consumers groups just got larger.
Participants also reported that consumers became more involved in their own care due to MedMAP’s strong emphasis on consumer education. Some of [the consumers] really picked up on this and they were more inclined to come in and say, “I’m having this side effect” . . . or, “could this be a side effect?” It’s an improvement, they were better educated—you were asking them to rate themselves, and they became more involved . . . they felt like they were more a part of the treatment team, rather than being treated.
Collaboration Prescribers found that success in MedMAP implementation in outpatient sites facilitated communication and collaboration between outpatient and inpatient practitioners, thus improving continuity of medication management. Collaboration was particularly evident in rural areas. For example, a rural hospital adopted MedMAP on its inpatient units due to observed improvements in the clinical status of consumers enrolled in MedMAP in their outpatient treatment facility: I thought we saw some good outcomes . . . we saw our local hospital come on board . . . so if we send someone to the hospital, the inpatient and outpatient prescribers talk to each other before making a medication change.”
Fidelity Assessments Participants indicated that the “scrutiny” provided by the fidelity assessment team was an essential factor in implementing MedMAP. Fidelity assessments provided objective data on progress in implementation, which was regularly reported to program managers. Feedback regarding progress in implementation also reinforced pre-implementation training and provided the opportunity for “hands-on” education. “When the fidelity team met with the prescribers, they did change . . . they scheduled patients to monitor side effects in the designated time frames—so they met that goal.” Participants reported that efficient communication among the fidelity assessment team and prescribers and program directors
at the participating CMHCs was essential in promoting implementation of MedMAP. One program director noted, “it was helpful that the [fidelity assessment team] could hear from the sites about what they thought was important, what they didn’t understand, or what they didn’t think was important.” This feedback from prescribers was routinely reported to MedMAP researchers for the purpose of adapting some elements to improve the “fit” of MedMAP within the work environments of the participating CMHCs. Technology The majority of participants reported that the biggest barrier to MedMAP implementation was problematic technology. The foundation of the MedMAP implementation project was a webbased application where prescribers documented their use of MedMAP in clinical practice. Participants recognized the value of technology in enhancing continuity of care between treatment settings: “The concept of a web-based application is meritorious . . . having a single repository of disease state management that can be accessed by any site when they have a need to know . . .” However, the technology for the web-based application had been in development for a year prior to the grant application; when the funding was received, the technology needed to be implemented immediately, which some participants believed was premature. Training in the use of the algorithm was particularly problematic due to issues with technology. One participant observed, “This is what people were trained on . . . people’s first experience with MedMAP was this bulky, crashing, buggy program. The computer program became synonymous with MedMAP, rather than the algorithm.” Participants at the clinical sites further reported that the technology was inconvenient, not user-friendly, and extremely frustrating to use. Comments from participants included: The web-based application did not connect to the electronic medical record they were already using, so the prescribers had to do double entries when documenting for MedMAP . . . it was double work with the dual entry systems, and that was a major, major barrier. [Prescribers] had such tight work schedules; they didn’t have the time to spend to become really comfortable with it. I never could get on it . . . I just quit trying. It would take me 20 or 25 minutes to fail to enroll a patient into the system.
Due to these technical problems, the web-based application was discontinued early in the study, and prescribers documented their use of MedMAP in the routinely used paper medical records. Work Flow A number of participants noted that some aspects of the MedMAP EBP did not fit into daily work flow at the participating sites. As one participant noted,
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE . . . any time you try to change a pattern of behavior, if the new system is markedly different and time consuming, and doesn’t fit well into the program after a short period of transition, it’s not likely to be adopted over the long term.
Another participant observed that if the EBP is added to the existing duties of a clinician, rather than replacing duties, it “eventually gets dropped.” In particular, participants reported that outcomes measurement did not fit well into work flow, primarily due to the frequency in which MedMAP required prescribers to monitor symptoms and side effects. In addition, competing practice initiatives within the participating sites interfered with MedMAP implementation by impeding work flow. Unbeknownst to the MedMAP implementation team, the participating CMHCs also were involved in a second, unrelated, best practice implementation project to improve metabolic monitoring in patients with schizophrenia. Participants reported that the requirements of MedMAP and the metabolic monitoring project did not flow well together, which added to the time demands on prescribers. Patient Barriers Several consumer-related factors interfered with full implementation of MedMAP, particularly insurance status. The MedMAP algorithm emphasized the use of very costly secondgeneration medications, and these were available only for patients who were insured by Medicaid. Although some uninsured patients relied on a Patient Assistance Program to obtain medications at no cost, many consumers were unable to afford costly medications recommended by the algorithm. One participant pointed out, . . . if they didn’t have insurance, and you didn’t have samples, you really couldn’t go by the algorithm. You had to prescribe what was available, what they could afford, or whatever samples you had . . . and if the medication required labs, you had to forget that medication and go with another one . . . these are poor counties and the resources are just not there.
Similarly, the monitoring required by the use of some medications was problematic for some patients. “Patients had to work harder” when clozapine was prescribed, due to the frequency of required laboratory monitoring, the need to obtain weekly transportation to the lab, and the cost of lab work. Patients also found that the requirement of weekly symptom monitoring for one month following a medication change was problematic. According to one participant: . . . for our patients, if you told them that they needed to come back in a week [for symptom assessment], they wouldn’t come back. It was a strain on their infrastructure resources for them to come for the current appointment, and it will take them a month to figure out how they are going to get back for their next appointment.
Regulatory Barriers Participants also observed that fidelity to implementation of some MedMAP elements was often in the moderate to low range
521
because state funding was not available to reimburse clinicians for the high intensity of services that MedMAP required. As mentioned above, MedMAP required patient symptoms to be assessed using a symptom rating scale at every medication visit. In addition, MedMAP required prescribers to monitor symptoms weekly for one month following a change in either the type or dose of a medication. This requirement was not possible due to Medicare reimbursement policies. As one prescriber stated: The idea was that the positive and negative symptom rating scales would be administered by a clinician in close chronological proximity to the physician visit . . . clearly, having the physician do these rating scales, with the patient loads we carry, was a non-starter. So the next logical person to do this was a nurse . . . but because of the way our agency deals with Medicare, we don’t do health bundle billing, and Medicare does not pay for nursing services. So we looked next at the idea of another clinician doing it, which meant anyone from a case manager to a therapist . . . but Medicare won’t pay for two visits on the same day because they consider it to be a duplication of services to some degree. So we had patients coming in [for symptom rating] on one day and then potentially come back another day to see their doctor. . . . having them come in twice for services that could easily be rendered on a single day . . . didn’t make it more likely to happen.
Furthermore, participants reported that adherence to the algorithm was difficult at times due to state regulations that imposed some barriers on the use of certain atypical antipsychotic medications. One participant stated that the paperwork required by the state to obtain pre-authorization for use of some medications was a notable barrier: “It took about four or six weeks to get approval, so in the meantime I would start the patient on another medication. By the time the medication was approved, the patient would not want to change.” Although some paperwork was required for all pre-authorizations, the paperwork was more extensive if the clinician was attempting to obtain pre-authorization for clozapine and injectable atypical antipsychotic medications. Other state regulations that participants perceived to interfere with the availability of medications in the MedMAP algorithm included a limit on the number of pills that could be dispensed in a prescription. Participants also mentioned a “3-brand rule,” which allowed the patient to be prescribed a maximum of three brand-name medications; all other medications were required to be in generic form. In addition, “limited use” of these medications was encouraged. Lack of Flexibility Although some modifications were made throughout the study to enhance flexibility in MedMAP implementation, the academicians and clinicians who originally developed MedMAP viewed certain elements as foundational to the philosophy of best practice in psychiatry; therefore these elements were not modified. For example, one element of MedMAP that was not modified was outcome monitoring; MedMAP required prescribers to use a validated rating scale to measure symptom severity at each medication visit. However, fidelity scores for outcome monitoring were low throughout the project, indicating
522
P. EL-MALLAKH ET AL.
that prescribers measured outcomes on less than 10% of their patients (Howard et al., 2009). Although participants recognized the value of treatment guided by outcomes, they indicated that the frequency in which MedMAP required outcome measurement was not practical and often not necessary. They reported that it slowed down the flow of work, and that patients found it tedious: We have 700 clients here, most of whom are pretty chronic and pretty stable, and have had their illnesses for many years. So any frequency you draw up [for using symptom rating scales] is going to seem a little bit arbitrary . . . they weren’t changing, the medications hadn’t been changed in years, they were as stable as they could be. [The patients] would ask, “Why are you asking me this again? Nothing has changed.”
In addition, participants observed that the lack of flexibility in implementing MedMAP elements interfered with their ability to use their clinical judgment when tailoring treatment to individual patients: It was not tailored . . . the inability to fit this to individual patients was a barrier—you know, individualized treatment planning is something we are always talking about, but then you have this “one size fits all” protocol.
Sustainability Several participants reported that a notable barrier to implementation was related to lack of long-term commitment to MedMAP at the clinical sites. “There was so much enthusiasm in the beginning . . . it just seemed to die out.” Associated problems with sustainability included staff turnover and inconsistent training of new employees, lack of continuing education for staff who received initial training at the beginning of the grant, and lack of continued support from the administration at the participating sites. RECOMMENDATIONS Participants were asked to comment on “lessons learned” and to discuss their recommendations for improving implementation in future efforts to incorporate MedMAP into routine clinical care. They offered a number of specific recommendations related to technologic readiness, innovations in care delivery, leadership, and a continued emphasis on monitoring patient outcomes. Ensure Technologic Readiness Prior to Implementation Participants emphasized that for implementation studies that utilize electronic or web-based documentation, the technology must be user friendly for clinicians and compatible with work flow. Comments from participants included: “If technology is going to be useful, you need to be able to get the patient entered into the system in under five minutes consistently” and “It needs to be consistent with an electronic medical record, and have some intelligence to it, like remembering information that was previously entered.” Other participants recommended
that the computer programs should build on the programs that clinicians are already accustomed to using: “Starting the computer program from scratch was not a good idea. An extraction model would have been better—in which the system currently in use is modified.” Participants also recommended that clinicians work very closely with program developers throughout the process of development, with the ultimate goal of developing a user-friendly and convenient program. Develop Innovative Models for Care Delivery Several participants expressed the opinion that the development of new models for care delivery can address many of the problems that were encountered during implementation of MedMAP. For example, one participant endorsed interprofessional collaboration to enhance the availability of resources within a treatment setting: . . . one of the problems we have had in mental health is that we have practitioners who want to provide services in the same way without looking at different service delivery models. They don’t use other resources, such as physician assistants or advanced practice nurses . . . and for us to continue to deliver health care the same way we’ve been doing it, when we have more people to see and less resources to do it, it is clearly something that can’t go forward. So, if we continue to think we’re going to use the same tools to combat new problems, we’re going to lose, we just can’t do it. We’re going to have to look for efficient [service delivery] models.
Some participants pointed out the need for flexibility in requirements for Medicare reimbursement, particularly the ability to bill for more than one service on the same day when the additional service is required by an EBP protocol. In addition, participants noted that the use of EBPs in routine clinical care need to be embedded in new service delivery models, with the use of performance-based contracting to link the use of EBPs with reimbursement for services: The contract needs to have very specific language and deliverables that are step-wise approximating the ultimate goal . . . some agencies are rewriting contracts to include evidence-based practices. So if you are a psychiatrist or an advanced practice nurse in that hospital, then you’re going to have to provide treatment for a disease state through evidence-based practice.
Emphasize Patient Outcomes Despite the difficulties participants experienced in routinely monitoring patient outcomes, they emphasized the value and necessity of outcome assessment in clinical practice. One participant stated, “If you don’t have outcomes, you don’t have anything, really.” According to one participant, thorough documentation of services and the outcomes resulting from services, is the primary strategy for reviewing service utilization and efficiency in the delivery of care. An emphasis on outcomes also was viewed as an important strategy in avoiding ineffective and wasteful services: We need to look at redundant services, like when we’ve put someone on lithium carbonate for the 14th time, you know it is a great
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE
523
medication and probably the only true mood stabilizer that we have . . . but when we redundantly do something that didn’t work before, then we have wasted services . . . that’s where evidence-based practices can make one of the greatest impacts . . . through documentation and through the efficient measurements of what truly is effective and what is not effective.
Participants held the opinion that “infused leadership” should also encompass clinical training in academic settings. One participant pointed out that inter-professional training in the use of EBPs is essential to ultimately foster change in the practice setting:
Participants also suggested that standardized procedures for the use of symptom rating scales in outcome monitoring can facilitate effective communication between prescribers regarding a patient’s status, which addresses MedMAP’s goal of promoting continuity of care:
. . . there needs to be a paradigm shift in how we train physicians and nurses. There needs to be endowed chairs in evidence-based practices in nursing, medicine, and pharmacy. . . . once a clinician knows that this is what I do, evidence-based practice, then that becomes their identity . . . obviously that’s going to take a generation to build. It has to be mandated from the Dean’s offices down, and they’re going to have to show accountability that they’re actually teaching evidencebased practices. It has to be a core part of residents’ practice. The lack of standards and the lack of outcomes are going to have to be a core part of what academic centers can address immediately.
. . . someone will say, “I just saw Mr. Smith and he is very, very, very psychotic,” and I respond by saying, “I really, really, really don’t know what that means.” [We use] the same nonspecific nomenclature on a daily basis, it’s much more useful to say, “This person is in the severe range of suspiciousness and unusual thought content.” Then I know what that means.
Foster Leadership at All Levels of an Organization Participants viewed leadership as an essential aspect of successful EBP implementation. For successful implementation to occur, leadership that supports the philosophy and routine use of EBPs needs to be “infused” into all levels of an organization, including administration, direct care providers, support staff, patients, and family members. Commitment is essential from the leaders at the upper levels of administration: “There’s nothing like a remindful mandate that we have to do this because it’s the right thing to do.” Leadership also was viewed as essential to foster organizational readiness both at the administrative and clinical levels: “Implementing this type of [EBP] is just a [monumental] task, and there has to be tremendous amount of organizational readiness to do that because there are so many components and it takes a lot of buy-in to change the inertia of an organization.” Effective leadership should solicit input from all stakeholders in the process of successful implementation, and thereby promote a sense of ownership of the EBP: “Work from the bottom up and get input from the people who really have to own it . . . [if] it’s planned from the top down, they’ll never own it.” Furthermore, leadership needs to foster unanimous support for use of the EBP: “Buy-in needs to be complete—every single clinician needs to be on board.” Leadership was viewed as essential at the individual clinician level as well. Participants recommended that clinical sites have a staff member whose sole job is management of the EBP at the site. This person was envisioned as an expert who is thoroughly trained in use of the EBP, is able to manage the EBP efficiently, provide clinicians with ongoing feedback on progress in implementation, and serve as a resource to clinicians who have questions about implementation issues. Participants also noted that the role would be most effective if the EBP support person served as a champion who would keep the momentum and enthusiasm going, and thereby promote long-term sustainability.
DISCUSSION The current study shows several important parallels with the existing implementation literature. These include commonalities in implementation that are true regardless of the practice being implemented (Damschroder & Hagedorn, 2011). For example, the importance of clinician buy-in to the adoption of a new practice has been noted by many others (McGovern, McHugo, Drake, Bond, & Merrens, 2013). In addition, one barrier to implementing and sustaining MedMAP related to what implementation researchers refer to as “relative advantage” (Rogers, 2003). Although the participants endorsed the value of outcomebased medicine, the costs for adopting MedMAP proved to be too time-consuming and, therefore, too impractical for use in routine clinical practice. Other similarities include recognition of the importance of leadership at multiple levels of an organization and the value of fidelity assessments and feedback to document progress in implementation. Torrey et al. (2012) identified several factors positively associated with high fidelity to the EBP, including active leadership and ongoing measurement and feedback to reinforce implementation. Torrey and colleagues also found that careful attention to redesigning the flow of work to facilitate the compatibility of the EBP with usual work routines was a key factor in successful implementation. However, participants in this study observed that work flow redesign in the participating CMHCs for the purpose of integrating MedMAP into routine care was difficult to accomplish. It is possible that some of the MedMAP standards are simply unrealistic in routine practice, especially in rural, under-resourced, communities. The difficulties experienced in the project with routine data collection of symptom measures are not at all unique to this study (Cradock, Young, & Sullivan, 2001). The point participants made about not needing to measure outcomes so often when patients are stable is a good one. However, some consumers continue to receive unnecessary medications year after year; therefore, a balance may need to be found between adherence to EBP guideline recommendations and prescriber
524
P. EL-MALLAKH ET AL.
flexibility in clinical judgment related to frequency of monitoring symptoms severity. Findings indicated that the web-based application was the most significant barrier to initial adoption of MedMAP. This finding is consistent with previous research on barriers to implementation of health information technology in clinical practice settings; typical barriers include resistance from physicians, inadequate computer skills among staff, and the tendency of electronic medical records to increase the amount of time practitioners devote to documentation (Goldzweig, Towfigh, Maglione, & Shekelle, 2009). As participants noted, use of existing and routinely-used electronic medical records can facilitate the integration of technology into implementation efforts. Additional findings suggest that financial constraints were a significant barrier to MedMAP implementation, a finding well-documented among other NIEBP implementation studies (Mancini et al., 2009). It is clear that a practice cannot be widely sustained if it lacks adequate funding (Bond et al., 2014). Unfortunately, implementation of MedMAP, which was intended to improve medication prescribing practices, occurred simultaneously with state-led efforts to control Medicaid spending through restrictions on medication usage. Researchers and policymakers have called for regulatory and financial systems to facilitate rather than obstruct care (Isett et al., 2007; Sederer, 2009; Torrey, Finnerty, Evans, & Wyzik, 2003). Findings about facilitators of implementation are very encouraging. For example, some mental health consumers expressed enthusiasm about their involvement in MedMAP implementation efforts and became more involved in partnering with their providers in their illness management. This enhanced consumer involvement achieved the goal of shared decisionmaking, a hallmark of recovery-oriented, patient-centered services. This finding, more than any other, supports the recommendations of others who emphasize the need to find ways to improve implementation of EBPs when faced with less than optimal implementation efforts, and continue to devote time and effort towards developing strategies to support the successful long-term EBP implementation (Brunette et al., 2008; Drake & Essock, 2009; Sederer, 2009).
Implications for Nursing Practice, Education, and Research Bridging the gap between research and practice remains a significant challenge in the mental health service system. Although mental health practitioners are increasingly encouraged to use EBPs, success stories in the dissemination, adoption, and sustainability of EBPs in mental health treatment are few (Sederer, 2009). Factors attributed to failure are now well-known. In particular, organizational structures that are resistant to change; time and work flow constraints; and lack of money have been identified as key factors that contribute to less than optimal EBP implementation and sustainability. The response to these factors
must be innovative to offset Sederer’s observation that “more of the same has no future” (p. 714). The authors support the recommendations from participants in this study as a guide to shaping the future of successful EBP implementation. In addition, nursing researchers and policymakers need to continue with efforts to determine the most effective ways to incorporate EBPs into practice. In this study, prescribers could not rely on the help of other team members because of reimbursement issues that limited the scope of practice of team members. Changes in policy and funding, so that the use of nurse practitioners and licensed clinical social workers when implementing EBPs isn’t limited, are crucial. In addition, professional nursing organizations, practitioners, and educators can advocate for increased availability of medications through participation in the legislative processes that regulate the budgets of public-funded state mental health systems. Similarly, changes are needed in nursing education. As one participant noted, a paradigm shift is needed, which can start in the education of clinicians. In addition to including content on EBPs in graduate nursing education, academic settings must prioritize the training and development of leaders who are able to foster change, promote interdisciplinary approaches to treatment, and influence the policy arena in ways that promote, rather than prohibit, the use of EBPs. Curricula also need to teach and allow students to apply theories that address organizational culture and change within organizations and health care systems. Clearly, a new wave of implementation research is warranted utilizing the “lessons learned” from the NIEBP Project to guide the refinement of strategies for EBP dissemination and implementation. Specifically, studies are needed to determine how successful sites create practice environments that promote, rather than prohibit, adoption and sustainability of EBPs. Building on the lessons learned from our study and that of others (Bond et al., 2008; Mancini et al., 2009; Brunette et al., 2008; Marshall et al., 2008; Whitley et al., 2009) can perhaps lead to new or improved models of care that result in improved satisfaction among providers and better outcomes for consumers of mental health services. Declaration of Interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
REFERENCES Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41, 228–236. Bond, G. R., Drake, R. E., McHugo, G. J., Rapp, C. A., & Whitley, R. (2009). Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice, 19, 569–581. Bond, G. R., McHugo, G. J., Becker, D. R., Rapp, C. A., & Whitley, R. (2008). Fidelity of supported employment: Lessons learned from the
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE National Evidence-Based Practice Project. Psychiatric Rehabilitation Journal, 31, 300–305. Brunette, M. F., Asher, D., Whitley, R., Lutz, W. J., Wieder, B. L., Jones, A. M., & McHugo, G. J. (2008). Implementation of integrated dual disorders treatment: A qualitative analysis of facilitators and barriers. Psychiatric Services, 59, 989–995. Chiles, J. A., Miller, A. L., Crismon, M. L., Rush, A. J., Krasnoff, A. S., & Shon, S. S. (1999). The Texas medication algorithm project: Development and implementation of the schizophrenia algorithm. Psychiatric Services, 50, 69–74. Cradock, J., Young, A. S., & Sullivan, G. (2001). The accuracy of medical record documentation in schizophrenia. Journal of Behavioral Health Services & Research, 28, 456–465. Damschroder, L. J., & Hagedorn, H. J. (2011). A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors, 25, 194–205. Drake, R. E., & Essock, S. M. (2009). The science-to-service gap in realworld schizophrenia treatment: The 95% problem. Schizophrenia Bulletin, 35, 677–678. Gilbert, D. A., Altshuler, K. Z., Rago, W. V., Shon, S. P., Crismon, M. L., Toprac, M. G., & Rush, A. J. (1998). Texas medication algorithm project: Definitions, rationale, and methods to develop medication algorithms. Journal of Clinical Psychiatry, 59, 345–351. Goldzweig, C. L., Towfigh, A., Maglione, M., & Shekelle, P. G. (2009). Costs and benefits of health information technology: New trends from the literature. Health Affairs, 28, w282–w293. Howard, P. B., El-Mallakh, P., Miller, A. L., Rayens, M. K., Bond, G. R., Henderson, K., & Cooley, A. T. (2009). Prescriber fidelity to a medication management evidence-based practice in the treatment of schizophrenia. Psychiatric Services, 60, 929–935. Isett, K. R., Burnam, M. A., Coleman-Beattie, B., Hyde, P. S., Morrissey, J. P., . . . Goldman, H. H. (2007). The state policy context of implementation issues for evidence-based practices in mental health. Psychiatric Services, 58, 914–921. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage. Mancini, A. D., Moser, L. L., Whitley, R., McHugo, G. J., Bond, G. R., Finnerty, M. T., & Burns, B. J. (2009). Assertive community treatment: Facilitators and barriers to implementation in routine mental health settings. Psychiatric Services, 60, 189–195.
525
Marshall, T., Rapp, C. A., Becker, D. R., & Bond, G. R. (2008). Key factors for implementing supported employment. Psychiatric Services, 59, 886–892. Mazade, N. A., & Glover, R. W. (2007). State mental health policy: Critical priorities confronting state mental health agencies. Psychiatric Services, 58, 1148–1150. McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., . . . Finnerty, M. T. (2007). Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services, 58, 1279–1284. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage. Miller, A. L., Chiles, J. A., Chiles, J. K., Crismon, M. L., Rush, A. J., & Shon, S. P. (1999). The Texas medication algorithm project (TMAP) schizophrenia algorithms. Journal of Clinical Psychiatry, 60, 649–657. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Sederer, L. I. (2009). Science to practice: Making what we know what we actually do. Schizophrenia Bulletin, 35, 714–718. Substance Abuse and Mental Health Services Administration. (2011). MedTEAM (Medication Treatment, Evaluation, and Management) EvidenceBased Practices (EBP) Kit (Publication No. SMA10-4549). Washington, DC: Author. Taylor, A. C., Bond, G. R., Tsai, J., Howard, P. B., El-Mallakh, P., Finnerty, M., . . . Miller, A. L. (2009). Scales to evaluate quality of medication management: Development and psychometric properties. Administration and Policy in Mental Health and Mental Health Services Research, 36, 247–254. Torrey, W. C., Bond, G. R., McHugo, G. J., & Swain, K. (2012). Evidencebased practice implementation in community mental health settings: The relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research, 39, 353–364. Torrey, W. C., Finnerty, M., Evans, A., & Wyzik, P. (2003). Strategies for leading the implementation of evidence-based practices. Psychiatric Clinics of North America, 26, 883–897. Tsai, J., & Bond, G. R. (2008). A comparison of electronic medical records to paper records in mental health centers. International Journal for Quality in Health Care, 20, 136–143. Whitley, R., Gingerich, S., Lutz, W. J., & Mueser, K. T. (2009). Implementing the illness management and recovery program in community mental health settings: Facilitators and barriers. Psychiatric Services, 60, 202–209.
Copyright of Issues in Mental Health Nursing is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Challenges of Implementing a Medication Management Evidence-Based Practice in a Community Mental Health Setting: Results of a Qualitative Study Peggy El-Mallakh, RN, PhD University of Kentucky, College of Nursing, Lexington, Kentucky, USA
Patricia B. Howard, PhD, RN, FAAN University of Kentucky, College of Nursing, Lexington, Kentucky, USA
Gary R. Bond, PhD Dartmouth College, Geisel School of Medicine, Hanover, New Hampshire, USA
Autumn P. Roque, BSN, RN University of Kentucky, College of Nursing, Lexington, Kentucky, USA
The Medication Management Approaches in Psychiatry (MedMAP) is a medication management evidence-based practice (EBP) to guide the use of psychotropic medications in the treatment of schizophrenia. This qualitative study examined facilitators and barriers to implementing MedMAP in community mental health treatment settings. Audio-taped qualitative interviews were conducted with practitioners and administrators involved in a MedMAP implementation project conducted in six community mental health centers. Data analysis was conducted using thematic analysis of transcribed interviews. Findings indicate that facilitators to MedMAP implementation included practitioner recognition of the value of MedMAP, consumer involvement, collaboration, continuity of care, and fidelity assessments. Barriers to MedMAP implementation included problematic technology, work flow issues, lack of flexibility in prescribers’ ability to implement MedMAP guidelines, regulatory and financial barriers, and consumer insurance status. Recommendations for improving future implementation efforts of MedMAP emphasize technological readiness, development of innovative models of care delivery, an emphasis on treatment guided by outcomes, and active leadership to promote EBPs within organizations and academic settings.
Behavioral health administrators, program managers, and researchers are giving increased attention to factors influencing the implementation of evidence-based practices (EBPs) to facilitate the timely translation of research into practice. One major study, the National Implementing Evidence-Based Practices Project (NIEBP) (McHugo et al., 2007), was an ambitious Address correspondence to Peggy El-Mallakh, University of Kentucky, College of Nursing, 547 College of Nursing, Lexington, KY 40536-0232 USA. E-mail: [email protected]
project to evaluate the use of systematic strategies to promote high quality implementation of EBPs. Funded in part by the Substance Abuse and Mental Health Services Administration (SAMHSA), this project involved implementation efforts for five psychosocial EBPs: (1) Illness Management and Recovery; (2) Supported Employment; (3) Integrated Dual Diagnosis Treatment; (4) Family Psychoeducation; and (5) Assertive Community Treatment. These were implemented in 53 community mental health treatment settings in eight states across the United States (Bond, McHugo, Becker, Rapp, & Whitley, 2008; McHugo et al., 2007). In a companion study to the NIEBP, a medication management EBP, the Medication Management Approaches in Psychiatry (MedMAP), was implemented in six community mental health treatment settings in one state (Howard et al., 2009). Recognizing the lack of user-friendly materials available to agencies seeking to implement an EBP, the NIEBP team created toolkits for each EBP consisting of materials to facilitate implementation. The toolkits included practitioner workbooks, research articles, introductory and instructional videotapes, and PowerPoint lectures. The toolkits also included program-level fidelity scales to measure how closely programs adhered to the EBP program models (McHugo et al., 2007). A key working hypothesis for the NIEBP was that systematic monitoring of fidelity promotes higher quality implementation (Bond, Drake, McHugo, Rapp, & Whitley, 2009). Efforts to disseminate and implement the SAMHSA EBPs in mental health service settings have produced a wide variety of results. The Supported Employment and Assertive Community Treatment EBPs were implemented with high fidelity;
517
518
P. EL-MALLAKH ET AL.
however, more moderate levels of fidelity were seen in implementation efforts for Illness Management, Integrated Dual Diagnosis Treatment (McHugo et al., 2007), and MedMAP (Howard et al., 2009). In addition, at a six-year follow-up of the 53 sites participating in the NIEBP Project, 47% continued to use the EBPs, 16% discontinued use and restarted the EBPs at a later date, and 37% of the implementing sites discontinued the use of these EBPs within the six years following full implementation (Bond et al., 2014). These inconsistent levels of adoption and sustainability have prompted researchers to examine contextual factors that influenced implementation of these EBPs within the participating sites (Brunette et al., 2008; Mancini et al., 2009; Marshall, Rapp, Becker, & Bond, 2008; Whitley, Gingerich, Lutz, & Mueser, 2009). The purpose of this article is to describe findings from a qualitative study that investigated facilitators and barriers to successful implementation of MedMAP and discuss implications of the findings for future implementation efforts.
LITERATURE REVIEW The Texas Medication Algorithm Project (TMAP), the precursor to MedMAP, was originally developed in 1995 in a joint partnership of the University of Texas and the Texas Department of Mental Health and Mental Retardation Services to facilitate the use of medication management best practices in routine clinical care (Chiles et al., 1999; Howard et al., 2009). The foundation of MedMAP consists of medication algorithms and guidelines that provide prescribers with strategies and tactics in the use of antipsychotic medications (Gilbert et al., 1998; Miller et al., 1999). Principles of medication management are incorporated into seven domains in MedMAP. These domains include (1) adequate information about diagnosis and treatment, (2) measurement and use of outcomes to guide decision making about medication regimens, (3) reduction of medication burden and side effects, (4) use of appropriate dosing and side effect monitoring, (5) identification of treatment refractory patients, (6) patient and family involvement in treatment planning, and (7) coordination of the treatment team (Howard et al., 2009). In addition, MedMAP stresses that serious medical illnesses, such as schizophrenia, are chronic and life-long, and acknowledges that consumers may have several prescribers treating them over the course of their illnesses. Therefore, optimal medication management requires prescriber access to thorough and comprehensive documentation on illness and medication history; rationales for medication management strategies; and information on medication safety, effectiveness, and tolerability of all medications that have been prescribed throughout a consumer’s illness. The MedMAP EBP guidelines were consolidated into an Implementation Toolkit that included a wide variety of resources, such as workbooks; educational resources for prescribers, consumers, and family members; and supporting documentation of research evidence on medication management. In addition, the Implementation Toolkit included symptom rating scales and
scales for measuring fidelity at the prescriber and organizational levels. MedMAP developers also provided training and consultation via telephone conferences for staff at implementing sites. MedMAP has been adapted over time by the original developers and is currently referred to as the Medication Treatment, Evaluation, and Management (MedTEAM) EBP (SAMHSA, 2011). PROJECT OVERVIEW A mixed-methods study design was used to implement MedMAP in six community mental health centers (CMHCs) in a south-central state of the US. Four CMHCs were located in an urban area of the state, and served 1,154 mental health consumers diagnosed with schizophrenia. The two remaining CMHCs were located in a rural area of the state, and served 226 mental health consumers diagnosed with schizophrenia. The participating CMHCs offered several specialized services in the treatment of schizophrenia, including medication monitoring, case management, individual therapy, and illness management/dual diagnosis group counseling. MedMAP Fidelity Scale Adherence to MedMAP is measured with two fidelity scales, one to measure organizational practices and the other to measure prescriber practices (Taylor et al., 2009). The latter scale is a 23-item scale based exclusively on chart reviews. The original psychometric study found large gaps in routine practice, including poor documentation of medication history and infrequent monitoring of symptoms and side effects. Fidelity assessments to measure prescriber adherence to the EBP were an integral aspect of the EBP implementation protocol (Howard et al., 2009). A Fidelity Assessment Team was responsible for conducting medical records reviews to measure prescriber fidelity to MedMAP. Thirty randomly-selected medical records were reviewed at each site visit at approximately four-month intervals, for a total of 900 medical records throughout the three-year duration of the project. Detailed findings related to fidelity outcomes have been reported elsewhere (Howard et al., 2009). To summarize, overall prescriber fidelity to MedMAP implementation was moderate, although it increased significantly between baseline and posttraining fidelity assessments. Baseline, pre-training fidelity had a mean score of 2.93 (SD = .35), and post-training fidelity mean scores ranged from 3.12–3.43 in subsequent follow up fidelity assessments. These scores were based on a 5-point behaviorally anchored scale, with scores of 4 or more representing high fidelity, 3–4 representing moderate fidelity, and below 3 representing low fidelity (McHugo et al., 2007). Findings also indicated that there was much variability in fidelity to the individual critical ingredients of MedMAP. For example, fidelity was consistently high in the areas of documentation of current practice, timely documentation of practice, use of appropriate dose ranges, and patient and family involvement in treatment
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE
planning (Howard et al., 2009). In contrast, consistently low fidelity was found in documentation of treatment outcomes, treatment guided by outcomes, side effect monitoring, and availability of adequate information about patient illness and medication history. A further MedMAP study has examined how the process of collecting and storing medication records affects the efficiency and completeness of MedMAP fidelity reviews. This study compared electronic medical records to paper records and found that electronic medical records were 40% more complete and 20% faster to retrieve (Tsai & Bond, 2008). METHODS A qualitative study was conducted at the conclusion of the MedMAP implementation study to examine the perceptions of key stakeholders on the project about the facilitators and barriers to implementation. Sample The sample consisted of eight practitioners and administrators who participated in the MedMAP implementation project. Inclusion criteria were being employed as a prescriber, program director, or administrator on the project and being willing to participate in audio-taped interviews. All participants provided informed consent prior to data collection. The Medical Institutional Review Board at the sponsoring university approved all study procedures. Procedures to ensure participant confidentiality were followed throughout the study. Data Collection Semi-structured interviews were conducted with participants to determine their perspectives on barriers and facilitators to MedMAP implementation at the participating sites. Members of the Fidelity Assessment Team conducted the interviews; interviews were audio-taped and transcribed for the purposes of data analysis, and lasted approximately 45–60 minutes. Data analysis and interpretation were conducted using classic guidelines described by Miles and Huberman (1994). Data analysis was based on three linked subprocesses, including data reduction, data display, and drawing/verifying conclusions (Miles & Huberman, 1994). In this study, data reduction was addressed by clustering data from the narrative accounts of participants and identifying preliminary themes. Data were then displayed in a matrix with narrative accounts linked to each preliminary theme. Themes were refined as additional data were compiled and the attributes and characteristics of each theme were identified. Finally, the conclusion drawing/verification phase consisted of condensing the essential themes to depict the participants’ experiences of MedMAP implementation. Data Trustworthiness Multiple strategies recommended by Lincoln and Guba (1985) were used to ensure data trustworthiness and to pro-
519
mote credibility of the study findings. For example, triangulation of data sources was paramount; quantitative data were obtained from fidelity assessments on 900 medical records over the course of three years to measure prescriber adherence to MedMAP. In addition, prolonged engagement was addressed with the researcher’s immersion in the study sites for the threeyear duration of the study. This immersion in the sites also facilitated persistent observation of the EBP implementation activities and issues. Debriefing among members of the research team was regularly conducted. In addition, peer review was a consistent activity of the research team during the process of data analysis and interpretation.
FINDINGS The interviews provided information about the barriers and facilitators that prescribers and organizations face when implementing complex practice changes that are typical of EBPs. In general, participants recognized the need for the practice change, yet simultaneously identified several individual and system-level factors that interfered with implementation. Factors that facilitated implementation were the perceived value of the MedMAP among prescribes at the participating sites, consumer involvement in MedMAP implementation, collaboration, and fidelity assessments. Participants also reported a number of factors that interfered with MedMAP implementation at the participating sites. These were technology, work flow, patient barriers, regulatory barriers, lack of flexibility, and sustainability issues. Perceived Value MedMAP was intended to promote consistency in the delivery and documentation of care, and clinicians valued the approach of “getting everyone on the same page” when prescribing medications. Participants reported that they recognized the benefits of using a formal best practice guideline for medication management in schizophrenia, which promoted their willingness to become proficient in the use of MedMAP. They also supported the use of MedMAP because it “made good clinical sense,” was based on the best available evidence, and had “immediate application” in practice. As one respondent said, “It gave structure to the approach that many of us thought made sense to begin with.” Participants particularly valued having guidelines for treatment refractory patients, which recommended the use of clozapine “instead of polypharmacy.” Another participant reported that MedMAP greatly improved documentation, making it “more consistent and thorough.” The emphasis on outcomes was viewed as clinically beneficial: “Doing an intervention and then getting objective data to see the effect of your intervention, and letting that guide your future intervention . . . that’s a perfectly sensible approach that’s hard to argue with.” Participants reported increased support for MedMAP when they observed clinical improvements in their
520
P. EL-MALLAKH ET AL.
patients: “When prescribers saw good outcomes, they became champions of MedMAP.” Consumer Involvement Participants reported that their enthusiasm for using MedMAP was enhanced when they observed consumer involvement in the implementation process. The implementation process included group sessions that focused on educating consumers about use of MedMAP in clinical care. One participant reported: They [consumers] were very excited about it and wanted to know more. Some consumers have become peer support specialists because of their participation in the MedMAP groups. They were willing to go out and share what they learned with others, and the consumers groups just got larger.
Participants also reported that consumers became more involved in their own care due to MedMAP’s strong emphasis on consumer education. Some of [the consumers] really picked up on this and they were more inclined to come in and say, “I’m having this side effect” . . . or, “could this be a side effect?” It’s an improvement, they were better educated—you were asking them to rate themselves, and they became more involved . . . they felt like they were more a part of the treatment team, rather than being treated.
Collaboration Prescribers found that success in MedMAP implementation in outpatient sites facilitated communication and collaboration between outpatient and inpatient practitioners, thus improving continuity of medication management. Collaboration was particularly evident in rural areas. For example, a rural hospital adopted MedMAP on its inpatient units due to observed improvements in the clinical status of consumers enrolled in MedMAP in their outpatient treatment facility: I thought we saw some good outcomes . . . we saw our local hospital come on board . . . so if we send someone to the hospital, the inpatient and outpatient prescribers talk to each other before making a medication change.”
Fidelity Assessments Participants indicated that the “scrutiny” provided by the fidelity assessment team was an essential factor in implementing MedMAP. Fidelity assessments provided objective data on progress in implementation, which was regularly reported to program managers. Feedback regarding progress in implementation also reinforced pre-implementation training and provided the opportunity for “hands-on” education. “When the fidelity team met with the prescribers, they did change . . . they scheduled patients to monitor side effects in the designated time frames—so they met that goal.” Participants reported that efficient communication among the fidelity assessment team and prescribers and program directors
at the participating CMHCs was essential in promoting implementation of MedMAP. One program director noted, “it was helpful that the [fidelity assessment team] could hear from the sites about what they thought was important, what they didn’t understand, or what they didn’t think was important.” This feedback from prescribers was routinely reported to MedMAP researchers for the purpose of adapting some elements to improve the “fit” of MedMAP within the work environments of the participating CMHCs. Technology The majority of participants reported that the biggest barrier to MedMAP implementation was problematic technology. The foundation of the MedMAP implementation project was a webbased application where prescribers documented their use of MedMAP in clinical practice. Participants recognized the value of technology in enhancing continuity of care between treatment settings: “The concept of a web-based application is meritorious . . . having a single repository of disease state management that can be accessed by any site when they have a need to know . . .” However, the technology for the web-based application had been in development for a year prior to the grant application; when the funding was received, the technology needed to be implemented immediately, which some participants believed was premature. Training in the use of the algorithm was particularly problematic due to issues with technology. One participant observed, “This is what people were trained on . . . people’s first experience with MedMAP was this bulky, crashing, buggy program. The computer program became synonymous with MedMAP, rather than the algorithm.” Participants at the clinical sites further reported that the technology was inconvenient, not user-friendly, and extremely frustrating to use. Comments from participants included: The web-based application did not connect to the electronic medical record they were already using, so the prescribers had to do double entries when documenting for MedMAP . . . it was double work with the dual entry systems, and that was a major, major barrier. [Prescribers] had such tight work schedules; they didn’t have the time to spend to become really comfortable with it. I never could get on it . . . I just quit trying. It would take me 20 or 25 minutes to fail to enroll a patient into the system.
Due to these technical problems, the web-based application was discontinued early in the study, and prescribers documented their use of MedMAP in the routinely used paper medical records. Work Flow A number of participants noted that some aspects of the MedMAP EBP did not fit into daily work flow at the participating sites. As one participant noted,
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE . . . any time you try to change a pattern of behavior, if the new system is markedly different and time consuming, and doesn’t fit well into the program after a short period of transition, it’s not likely to be adopted over the long term.
Another participant observed that if the EBP is added to the existing duties of a clinician, rather than replacing duties, it “eventually gets dropped.” In particular, participants reported that outcomes measurement did not fit well into work flow, primarily due to the frequency in which MedMAP required prescribers to monitor symptoms and side effects. In addition, competing practice initiatives within the participating sites interfered with MedMAP implementation by impeding work flow. Unbeknownst to the MedMAP implementation team, the participating CMHCs also were involved in a second, unrelated, best practice implementation project to improve metabolic monitoring in patients with schizophrenia. Participants reported that the requirements of MedMAP and the metabolic monitoring project did not flow well together, which added to the time demands on prescribers. Patient Barriers Several consumer-related factors interfered with full implementation of MedMAP, particularly insurance status. The MedMAP algorithm emphasized the use of very costly secondgeneration medications, and these were available only for patients who were insured by Medicaid. Although some uninsured patients relied on a Patient Assistance Program to obtain medications at no cost, many consumers were unable to afford costly medications recommended by the algorithm. One participant pointed out, . . . if they didn’t have insurance, and you didn’t have samples, you really couldn’t go by the algorithm. You had to prescribe what was available, what they could afford, or whatever samples you had . . . and if the medication required labs, you had to forget that medication and go with another one . . . these are poor counties and the resources are just not there.
Similarly, the monitoring required by the use of some medications was problematic for some patients. “Patients had to work harder” when clozapine was prescribed, due to the frequency of required laboratory monitoring, the need to obtain weekly transportation to the lab, and the cost of lab work. Patients also found that the requirement of weekly symptom monitoring for one month following a medication change was problematic. According to one participant: . . . for our patients, if you told them that they needed to come back in a week [for symptom assessment], they wouldn’t come back. It was a strain on their infrastructure resources for them to come for the current appointment, and it will take them a month to figure out how they are going to get back for their next appointment.
Regulatory Barriers Participants also observed that fidelity to implementation of some MedMAP elements was often in the moderate to low range
521
because state funding was not available to reimburse clinicians for the high intensity of services that MedMAP required. As mentioned above, MedMAP required patient symptoms to be assessed using a symptom rating scale at every medication visit. In addition, MedMAP required prescribers to monitor symptoms weekly for one month following a change in either the type or dose of a medication. This requirement was not possible due to Medicare reimbursement policies. As one prescriber stated: The idea was that the positive and negative symptom rating scales would be administered by a clinician in close chronological proximity to the physician visit . . . clearly, having the physician do these rating scales, with the patient loads we carry, was a non-starter. So the next logical person to do this was a nurse . . . but because of the way our agency deals with Medicare, we don’t do health bundle billing, and Medicare does not pay for nursing services. So we looked next at the idea of another clinician doing it, which meant anyone from a case manager to a therapist . . . but Medicare won’t pay for two visits on the same day because they consider it to be a duplication of services to some degree. So we had patients coming in [for symptom rating] on one day and then potentially come back another day to see their doctor. . . . having them come in twice for services that could easily be rendered on a single day . . . didn’t make it more likely to happen.
Furthermore, participants reported that adherence to the algorithm was difficult at times due to state regulations that imposed some barriers on the use of certain atypical antipsychotic medications. One participant stated that the paperwork required by the state to obtain pre-authorization for use of some medications was a notable barrier: “It took about four or six weeks to get approval, so in the meantime I would start the patient on another medication. By the time the medication was approved, the patient would not want to change.” Although some paperwork was required for all pre-authorizations, the paperwork was more extensive if the clinician was attempting to obtain pre-authorization for clozapine and injectable atypical antipsychotic medications. Other state regulations that participants perceived to interfere with the availability of medications in the MedMAP algorithm included a limit on the number of pills that could be dispensed in a prescription. Participants also mentioned a “3-brand rule,” which allowed the patient to be prescribed a maximum of three brand-name medications; all other medications were required to be in generic form. In addition, “limited use” of these medications was encouraged. Lack of Flexibility Although some modifications were made throughout the study to enhance flexibility in MedMAP implementation, the academicians and clinicians who originally developed MedMAP viewed certain elements as foundational to the philosophy of best practice in psychiatry; therefore these elements were not modified. For example, one element of MedMAP that was not modified was outcome monitoring; MedMAP required prescribers to use a validated rating scale to measure symptom severity at each medication visit. However, fidelity scores for outcome monitoring were low throughout the project, indicating
522
P. EL-MALLAKH ET AL.
that prescribers measured outcomes on less than 10% of their patients (Howard et al., 2009). Although participants recognized the value of treatment guided by outcomes, they indicated that the frequency in which MedMAP required outcome measurement was not practical and often not necessary. They reported that it slowed down the flow of work, and that patients found it tedious: We have 700 clients here, most of whom are pretty chronic and pretty stable, and have had their illnesses for many years. So any frequency you draw up [for using symptom rating scales] is going to seem a little bit arbitrary . . . they weren’t changing, the medications hadn’t been changed in years, they were as stable as they could be. [The patients] would ask, “Why are you asking me this again? Nothing has changed.”
In addition, participants observed that the lack of flexibility in implementing MedMAP elements interfered with their ability to use their clinical judgment when tailoring treatment to individual patients: It was not tailored . . . the inability to fit this to individual patients was a barrier—you know, individualized treatment planning is something we are always talking about, but then you have this “one size fits all” protocol.
Sustainability Several participants reported that a notable barrier to implementation was related to lack of long-term commitment to MedMAP at the clinical sites. “There was so much enthusiasm in the beginning . . . it just seemed to die out.” Associated problems with sustainability included staff turnover and inconsistent training of new employees, lack of continuing education for staff who received initial training at the beginning of the grant, and lack of continued support from the administration at the participating sites. RECOMMENDATIONS Participants were asked to comment on “lessons learned” and to discuss their recommendations for improving implementation in future efforts to incorporate MedMAP into routine clinical care. They offered a number of specific recommendations related to technologic readiness, innovations in care delivery, leadership, and a continued emphasis on monitoring patient outcomes. Ensure Technologic Readiness Prior to Implementation Participants emphasized that for implementation studies that utilize electronic or web-based documentation, the technology must be user friendly for clinicians and compatible with work flow. Comments from participants included: “If technology is going to be useful, you need to be able to get the patient entered into the system in under five minutes consistently” and “It needs to be consistent with an electronic medical record, and have some intelligence to it, like remembering information that was previously entered.” Other participants recommended
that the computer programs should build on the programs that clinicians are already accustomed to using: “Starting the computer program from scratch was not a good idea. An extraction model would have been better—in which the system currently in use is modified.” Participants also recommended that clinicians work very closely with program developers throughout the process of development, with the ultimate goal of developing a user-friendly and convenient program. Develop Innovative Models for Care Delivery Several participants expressed the opinion that the development of new models for care delivery can address many of the problems that were encountered during implementation of MedMAP. For example, one participant endorsed interprofessional collaboration to enhance the availability of resources within a treatment setting: . . . one of the problems we have had in mental health is that we have practitioners who want to provide services in the same way without looking at different service delivery models. They don’t use other resources, such as physician assistants or advanced practice nurses . . . and for us to continue to deliver health care the same way we’ve been doing it, when we have more people to see and less resources to do it, it is clearly something that can’t go forward. So, if we continue to think we’re going to use the same tools to combat new problems, we’re going to lose, we just can’t do it. We’re going to have to look for efficient [service delivery] models.
Some participants pointed out the need for flexibility in requirements for Medicare reimbursement, particularly the ability to bill for more than one service on the same day when the additional service is required by an EBP protocol. In addition, participants noted that the use of EBPs in routine clinical care need to be embedded in new service delivery models, with the use of performance-based contracting to link the use of EBPs with reimbursement for services: The contract needs to have very specific language and deliverables that are step-wise approximating the ultimate goal . . . some agencies are rewriting contracts to include evidence-based practices. So if you are a psychiatrist or an advanced practice nurse in that hospital, then you’re going to have to provide treatment for a disease state through evidence-based practice.
Emphasize Patient Outcomes Despite the difficulties participants experienced in routinely monitoring patient outcomes, they emphasized the value and necessity of outcome assessment in clinical practice. One participant stated, “If you don’t have outcomes, you don’t have anything, really.” According to one participant, thorough documentation of services and the outcomes resulting from services, is the primary strategy for reviewing service utilization and efficiency in the delivery of care. An emphasis on outcomes also was viewed as an important strategy in avoiding ineffective and wasteful services: We need to look at redundant services, like when we’ve put someone on lithium carbonate for the 14th time, you know it is a great
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE
523
medication and probably the only true mood stabilizer that we have . . . but when we redundantly do something that didn’t work before, then we have wasted services . . . that’s where evidence-based practices can make one of the greatest impacts . . . through documentation and through the efficient measurements of what truly is effective and what is not effective.
Participants held the opinion that “infused leadership” should also encompass clinical training in academic settings. One participant pointed out that inter-professional training in the use of EBPs is essential to ultimately foster change in the practice setting:
Participants also suggested that standardized procedures for the use of symptom rating scales in outcome monitoring can facilitate effective communication between prescribers regarding a patient’s status, which addresses MedMAP’s goal of promoting continuity of care:
. . . there needs to be a paradigm shift in how we train physicians and nurses. There needs to be endowed chairs in evidence-based practices in nursing, medicine, and pharmacy. . . . once a clinician knows that this is what I do, evidence-based practice, then that becomes their identity . . . obviously that’s going to take a generation to build. It has to be mandated from the Dean’s offices down, and they’re going to have to show accountability that they’re actually teaching evidencebased practices. It has to be a core part of residents’ practice. The lack of standards and the lack of outcomes are going to have to be a core part of what academic centers can address immediately.
. . . someone will say, “I just saw Mr. Smith and he is very, very, very psychotic,” and I respond by saying, “I really, really, really don’t know what that means.” [We use] the same nonspecific nomenclature on a daily basis, it’s much more useful to say, “This person is in the severe range of suspiciousness and unusual thought content.” Then I know what that means.
Foster Leadership at All Levels of an Organization Participants viewed leadership as an essential aspect of successful EBP implementation. For successful implementation to occur, leadership that supports the philosophy and routine use of EBPs needs to be “infused” into all levels of an organization, including administration, direct care providers, support staff, patients, and family members. Commitment is essential from the leaders at the upper levels of administration: “There’s nothing like a remindful mandate that we have to do this because it’s the right thing to do.” Leadership also was viewed as essential to foster organizational readiness both at the administrative and clinical levels: “Implementing this type of [EBP] is just a [monumental] task, and there has to be tremendous amount of organizational readiness to do that because there are so many components and it takes a lot of buy-in to change the inertia of an organization.” Effective leadership should solicit input from all stakeholders in the process of successful implementation, and thereby promote a sense of ownership of the EBP: “Work from the bottom up and get input from the people who really have to own it . . . [if] it’s planned from the top down, they’ll never own it.” Furthermore, leadership needs to foster unanimous support for use of the EBP: “Buy-in needs to be complete—every single clinician needs to be on board.” Leadership was viewed as essential at the individual clinician level as well. Participants recommended that clinical sites have a staff member whose sole job is management of the EBP at the site. This person was envisioned as an expert who is thoroughly trained in use of the EBP, is able to manage the EBP efficiently, provide clinicians with ongoing feedback on progress in implementation, and serve as a resource to clinicians who have questions about implementation issues. Participants also noted that the role would be most effective if the EBP support person served as a champion who would keep the momentum and enthusiasm going, and thereby promote long-term sustainability.
DISCUSSION The current study shows several important parallels with the existing implementation literature. These include commonalities in implementation that are true regardless of the practice being implemented (Damschroder & Hagedorn, 2011). For example, the importance of clinician buy-in to the adoption of a new practice has been noted by many others (McGovern, McHugo, Drake, Bond, & Merrens, 2013). In addition, one barrier to implementing and sustaining MedMAP related to what implementation researchers refer to as “relative advantage” (Rogers, 2003). Although the participants endorsed the value of outcomebased medicine, the costs for adopting MedMAP proved to be too time-consuming and, therefore, too impractical for use in routine clinical practice. Other similarities include recognition of the importance of leadership at multiple levels of an organization and the value of fidelity assessments and feedback to document progress in implementation. Torrey et al. (2012) identified several factors positively associated with high fidelity to the EBP, including active leadership and ongoing measurement and feedback to reinforce implementation. Torrey and colleagues also found that careful attention to redesigning the flow of work to facilitate the compatibility of the EBP with usual work routines was a key factor in successful implementation. However, participants in this study observed that work flow redesign in the participating CMHCs for the purpose of integrating MedMAP into routine care was difficult to accomplish. It is possible that some of the MedMAP standards are simply unrealistic in routine practice, especially in rural, under-resourced, communities. The difficulties experienced in the project with routine data collection of symptom measures are not at all unique to this study (Cradock, Young, & Sullivan, 2001). The point participants made about not needing to measure outcomes so often when patients are stable is a good one. However, some consumers continue to receive unnecessary medications year after year; therefore, a balance may need to be found between adherence to EBP guideline recommendations and prescriber
524
P. EL-MALLAKH ET AL.
flexibility in clinical judgment related to frequency of monitoring symptoms severity. Findings indicated that the web-based application was the most significant barrier to initial adoption of MedMAP. This finding is consistent with previous research on barriers to implementation of health information technology in clinical practice settings; typical barriers include resistance from physicians, inadequate computer skills among staff, and the tendency of electronic medical records to increase the amount of time practitioners devote to documentation (Goldzweig, Towfigh, Maglione, & Shekelle, 2009). As participants noted, use of existing and routinely-used electronic medical records can facilitate the integration of technology into implementation efforts. Additional findings suggest that financial constraints were a significant barrier to MedMAP implementation, a finding well-documented among other NIEBP implementation studies (Mancini et al., 2009). It is clear that a practice cannot be widely sustained if it lacks adequate funding (Bond et al., 2014). Unfortunately, implementation of MedMAP, which was intended to improve medication prescribing practices, occurred simultaneously with state-led efforts to control Medicaid spending through restrictions on medication usage. Researchers and policymakers have called for regulatory and financial systems to facilitate rather than obstruct care (Isett et al., 2007; Sederer, 2009; Torrey, Finnerty, Evans, & Wyzik, 2003). Findings about facilitators of implementation are very encouraging. For example, some mental health consumers expressed enthusiasm about their involvement in MedMAP implementation efforts and became more involved in partnering with their providers in their illness management. This enhanced consumer involvement achieved the goal of shared decisionmaking, a hallmark of recovery-oriented, patient-centered services. This finding, more than any other, supports the recommendations of others who emphasize the need to find ways to improve implementation of EBPs when faced with less than optimal implementation efforts, and continue to devote time and effort towards developing strategies to support the successful long-term EBP implementation (Brunette et al., 2008; Drake & Essock, 2009; Sederer, 2009).
Implications for Nursing Practice, Education, and Research Bridging the gap between research and practice remains a significant challenge in the mental health service system. Although mental health practitioners are increasingly encouraged to use EBPs, success stories in the dissemination, adoption, and sustainability of EBPs in mental health treatment are few (Sederer, 2009). Factors attributed to failure are now well-known. In particular, organizational structures that are resistant to change; time and work flow constraints; and lack of money have been identified as key factors that contribute to less than optimal EBP implementation and sustainability. The response to these factors
must be innovative to offset Sederer’s observation that “more of the same has no future” (p. 714). The authors support the recommendations from participants in this study as a guide to shaping the future of successful EBP implementation. In addition, nursing researchers and policymakers need to continue with efforts to determine the most effective ways to incorporate EBPs into practice. In this study, prescribers could not rely on the help of other team members because of reimbursement issues that limited the scope of practice of team members. Changes in policy and funding, so that the use of nurse practitioners and licensed clinical social workers when implementing EBPs isn’t limited, are crucial. In addition, professional nursing organizations, practitioners, and educators can advocate for increased availability of medications through participation in the legislative processes that regulate the budgets of public-funded state mental health systems. Similarly, changes are needed in nursing education. As one participant noted, a paradigm shift is needed, which can start in the education of clinicians. In addition to including content on EBPs in graduate nursing education, academic settings must prioritize the training and development of leaders who are able to foster change, promote interdisciplinary approaches to treatment, and influence the policy arena in ways that promote, rather than prohibit, the use of EBPs. Curricula also need to teach and allow students to apply theories that address organizational culture and change within organizations and health care systems. Clearly, a new wave of implementation research is warranted utilizing the “lessons learned” from the NIEBP Project to guide the refinement of strategies for EBP dissemination and implementation. Specifically, studies are needed to determine how successful sites create practice environments that promote, rather than prohibit, adoption and sustainability of EBPs. Building on the lessons learned from our study and that of others (Bond et al., 2008; Mancini et al., 2009; Brunette et al., 2008; Marshall et al., 2008; Whitley et al., 2009) can perhaps lead to new or improved models of care that result in improved satisfaction among providers and better outcomes for consumers of mental health services. Declaration of Interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
REFERENCES Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41, 228–236. Bond, G. R., Drake, R. E., McHugo, G. J., Rapp, C. A., & Whitley, R. (2009). Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice, 19, 569–581. Bond, G. R., McHugo, G. J., Becker, D. R., Rapp, C. A., & Whitley, R. (2008). Fidelity of supported employment: Lessons learned from the
A MEDICATION MANAGEMENT EVIDENCE-BASED PRACTICE National Evidence-Based Practice Project. Psychiatric Rehabilitation Journal, 31, 300–305. Brunette, M. F., Asher, D., Whitley, R., Lutz, W. J., Wieder, B. L., Jones, A. M., & McHugo, G. J. (2008). Implementation of integrated dual disorders treatment: A qualitative analysis of facilitators and barriers. Psychiatric Services, 59, 989–995. Chiles, J. A., Miller, A. L., Crismon, M. L., Rush, A. J., Krasnoff, A. S., & Shon, S. S. (1999). The Texas medication algorithm project: Development and implementation of the schizophrenia algorithm. Psychiatric Services, 50, 69–74. Cradock, J., Young, A. S., & Sullivan, G. (2001). The accuracy of medical record documentation in schizophrenia. Journal of Behavioral Health Services & Research, 28, 456–465. Damschroder, L. J., & Hagedorn, H. J. (2011). A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors, 25, 194–205. Drake, R. E., & Essock, S. M. (2009). The science-to-service gap in realworld schizophrenia treatment: The 95% problem. Schizophrenia Bulletin, 35, 677–678. Gilbert, D. A., Altshuler, K. Z., Rago, W. V., Shon, S. P., Crismon, M. L., Toprac, M. G., & Rush, A. J. (1998). Texas medication algorithm project: Definitions, rationale, and methods to develop medication algorithms. Journal of Clinical Psychiatry, 59, 345–351. Goldzweig, C. L., Towfigh, A., Maglione, M., & Shekelle, P. G. (2009). Costs and benefits of health information technology: New trends from the literature. Health Affairs, 28, w282–w293. Howard, P. B., El-Mallakh, P., Miller, A. L., Rayens, M. K., Bond, G. R., Henderson, K., & Cooley, A. T. (2009). Prescriber fidelity to a medication management evidence-based practice in the treatment of schizophrenia. Psychiatric Services, 60, 929–935. Isett, K. R., Burnam, M. A., Coleman-Beattie, B., Hyde, P. S., Morrissey, J. P., . . . Goldman, H. H. (2007). The state policy context of implementation issues for evidence-based practices in mental health. Psychiatric Services, 58, 914–921. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage. Mancini, A. D., Moser, L. L., Whitley, R., McHugo, G. J., Bond, G. R., Finnerty, M. T., & Burns, B. J. (2009). Assertive community treatment: Facilitators and barriers to implementation in routine mental health settings. Psychiatric Services, 60, 189–195.
525
Marshall, T., Rapp, C. A., Becker, D. R., & Bond, G. R. (2008). Key factors for implementing supported employment. Psychiatric Services, 59, 886–892. Mazade, N. A., & Glover, R. W. (2007). State mental health policy: Critical priorities confronting state mental health agencies. Psychiatric Services, 58, 1148–1150. McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., . . . Finnerty, M. T. (2007). Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services, 58, 1279–1284. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage. Miller, A. L., Chiles, J. A., Chiles, J. K., Crismon, M. L., Rush, A. J., & Shon, S. P. (1999). The Texas medication algorithm project (TMAP) schizophrenia algorithms. Journal of Clinical Psychiatry, 60, 649–657. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press. Sederer, L. I. (2009). Science to practice: Making what we know what we actually do. Schizophrenia Bulletin, 35, 714–718. Substance Abuse and Mental Health Services Administration. (2011). MedTEAM (Medication Treatment, Evaluation, and Management) EvidenceBased Practices (EBP) Kit (Publication No. SMA10-4549). Washington, DC: Author. Taylor, A. C., Bond, G. R., Tsai, J., Howard, P. B., El-Mallakh, P., Finnerty, M., . . . Miller, A. L. (2009). Scales to evaluate quality of medication management: Development and psychometric properties. Administration and Policy in Mental Health and Mental Health Services Research, 36, 247–254. Torrey, W. C., Bond, G. R., McHugo, G. J., & Swain, K. (2012). Evidencebased practice implementation in community mental health settings: The relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research, 39, 353–364. Torrey, W. C., Finnerty, M., Evans, A., & Wyzik, P. (2003). Strategies for leading the implementation of evidence-based practices. Psychiatric Clinics of North America, 26, 883–897. Tsai, J., & Bond, G. R. (2008). A comparison of electronic medical records to paper records in mental health centers. International Journal for Quality in Health Care, 20, 136–143. Whitley, R., Gingerich, S., Lutz, W. J., & Mueser, K. T. (2009). Implementing the illness management and recovery program in community mental health settings: Facilitators and barriers. Psychiatric Services, 60, 202–209.
Copyright of Issues in Mental Health Nursing is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
