Psychological Services 2014, Vol. 11, No. 2, 153–161

© 2013 American Psychological Association 1541-1559/14/$12.00 DOI: 10.1037/a0034855

Web-Based PTSD Training for Primary Care Providers: A Pilot Study Kristin W. Samuelson

Christopher J. Koenig, Nicole McCamish, Gerard Choucroun, Gary Tarasovsky, and Daniel Bertenthal

San Francisco Veterans Affairs Medical Center and California School of Professional Psychology at Alliant International University

San Francisco Veterans Affairs Medical Center

Karen H. Seal This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

San Francisco Veterans Affairs Medical Center and University of California, San Francisco Veterans with posttraumatic stress disorder (PTSD) symptoms frequently present to primary care providers (PCPs) and are reluctant to seek out or accept referrals to specialty mental health care. Most PCPs have not been trained to assess for and manage symptoms of PTSD. Web-based programs are increasingly used for medical education, but there are no published evaluations of online PTSD trainings for PCPs. We developed a 70-min Web-based training that focused on military-related PTSD for PCPs practicing in Veterans Affairs (VA) hospitals, but was applicable to PCPs treating veterans and other trauma-exposed patients outside VA settings. The training consisted of four modules: (1) Detection and Assessment; (2) Comorbid Conditions and Related Problems; (3) Pharmacological Interventions; and (4) Psychotherapeutic Interventions. Clinical vignettes dramatized key training concepts. Seventy-three PCPs completed the training and assessments pre- and posttraining and 30 days later. Paired t tests compared change in PTSD-related knowledge and comfort with PTSD-related skills, and qualitative methods were used to summarize participant feedback. After the training, mean knowledge score improved from 46% to 75% items correct, with sustained improvement at 30 days. Thirty days posttraining, PCPs reported significantly greater comfort regarding PTSD-related skills assessed; 47% reported using training content in their clinical practice. Qualitatively, PCPs appreciated the flexibility of asynchronous, self-paced online modules, but suggested more interactive content. Given the numerous barriers to specialty mental health treatment, coupled with a preference among veterans with PTSD for accessing treatment through primary care, improving PTSD competency among PCPs may help better serve veterans’ mental health needs. Keywords: posttraumatic stress disorder, primary care, veterans, Web-based

2008). A major concern among veteran providers is this population’s underutilization of mental health care (APA Presidential Task Force on Military Deployment Services for Youth, Families, & Service Members, 2007; Hoge et al., 2004; Seal et al., 2010). There are multiple patient-level barriers to mental health care including lack of access, particularly among rural veterans, stigma, cultural attitudes, denial, and apathy (Fortney et al., 2011; Hoge et al., 2004; Ouimette et al., 2011; Pietrzak et al., 2009). In addition, behaviors honed in the warzone and perceived as adaptive and ego-syntonic by service members, such as avoidance and hypervigilance, come to represent mental health symptoms upon return home. Veterans are particularly attuned to perceptions of stigma associated with mental health care and voice concerns that they will be perceived as weak or that seeking mental health assistance could negatively impact their careers (Hoge et al., 2004; Pietrzak et al., 2009). The “warrior culture” of the U.S. military, with its values of strength and resilience, contributes to veterans’ beliefs that mental health problems should be combated without outside help (Tanielian & Jaycox, 2008). Despite underutilization of mental health services, veterans with mental health disorders disproportionately use VA primary care services compared with OEF/OIF Veterans without mental health problems (Cohen et al., 2010). Approximately 11–12% of primary

Posttraumatic stress disorder (PTSD) affects 6 –7% of the general U.S. population (Kessler et al., 2005). Estimates are higher for veterans: lifetime PTSD rates for Vietnam veterans are as high as 31% (Kulka et al., 1990) and between 14 and 16% for Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn (OEF/OIF/OND) veterans (Hoge et al., 2004; Tanielian & Jaycox,

This article was published Online First December 23, 2013. Kristin W. Samuelson, San Francisco Veterans Affairs Medical Center and Department of Mental Health and California School of Professional Psychology at Alliant International University; Christopher J. Koenig, Nicole McCamish, Gerard Choucroun, Gary Tarasovsky, and Daniel Bertenthal, Departments of Medicine and Mental Health, San Francisco Veterans Affairs Medical Center, San Francisco, California; Karen H. Seal, Department of Medicine, San Francisco Veterans Affairs Medical Center, and Departments of Medicine and Psychiatry, University of California, San Francisco. This research was supported by U.S. Department of Defense Grant W81XWH-09-1-170 through the Northern California Institute of Research and Education. Correspondence concerning this article should be addressed to Kristin W. Samuelson, Alliant International University, 1 Beach Street, Suite 100, San Francisco, CA 94133. E-mail: [email protected] 153

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

154

SAMUELSON ET AL.

care patients seen in Veterans Affairs (VA) settings have PTSD (Magruder et al., 2005). As such, primary care providers (PCPs) can play a vital role in the initial assessment and treatment of PTSD. Primary care has been coined the “de facto mental health system,” as most mental health care is delivered in primary care settings, yet the effectiveness of this care has been plagued by underdiagnosis and inappropriate treatment (Higgins, 1994; Regier et al., 1978). Recent gains have been made in training PCPs in the identification and treatment of depression (Dobscha, Gerrity, & Ward, 2001; Harman et al., 2001), but focus on the management of PTSD in primary care settings is a more recent development. Primary care providers have been shown to lack the necessary training to provide appropriate diagnosis and treatment for PTSD (Magruder et al., 2005). In addition, they report feeling unprepared and unsure in their work with trauma survivors (Green et al., 2011). A retrospective medical record review of patients seen in VA primary care clinics revealed that PCPs had correctly identified PTSD in only 46.5% of 746 patients later found to have PTSD by gold standard clinical interview, and only 47.7% of these patients had been referred for mental health services (Magruder et al., 2005). Similarly, in a study of 539 primary care patients with anxiety disorders, investigators found that nearly half had not been treated (Weisberg, Dyck, Culpepper, & Keller, 2007); the most common reason was failure of their PCPs to recommend treatment (Rodriguez et al., 2004). Thus, PTSD patients presenting in primary care often go unrecognized and untreated, which can lead to the chronic psychosocial, occupational, and functional impairments commonly associated with PTSD (Thomas et al., 2010). Additionally, PCPs are tasked with the challenge of treating patients with PTSD who present with a myriad of comorbid physical and psychiatric problems. Patients with PTSD experience poorer health status (Schnurr & Jankowski, 1999) and more nonspecific physical symptoms (Dobie et al., 2004) than patients without PTSD and experience high rates of comorbid psychiatric disorders, particularly depression and substance use disorders (Kessler, Sonnega, Bromet, Hughes, & Nelson, 1995; Shalev et al., 1998). Magruder et al. (2005) found that 87% of VA primary care patients with PTSD suffered from one or more comorbid psychiatric disorders, with more than half reporting suicidal ideation. These comorbid conditions complicate the diagnostic picture and make it more difficult for PCPs to detect PTSD. To address these issues, the VA has implemented several programs to improve the delivery of mental health treatment for OEF/OIF/OND veterans. In 2007, the VA initiated an expansion of mental health services, which included increasing mental health staff in rural VA clinics, implementing telemental health to increase access to specialty mental health care for veterans in rural areas, and introducing the Primary Care Mental Health Integration (PC-MHI) initiative to colocate mental health providers in primary care settings (Zeiss & Karlin, 2008). In a recently published study of 526 OEF/OIF veterans, those who attended an Integrated Care (IC) clinic that offered integrated, colocated primary care and mental health services were more likely to have received a mental health evaluation and one mental health follow-up visit in the year after mental health diagnosis than veterans who received usual primary care with referrals to mental health as needed (Seal et al., 2011). Nevertheless, this same study showed that retention in mental health treatment was poor; the median number of mental health visits in the first year after evaluation was 1 visit for

veterans in both groups. The Re-Engineering Systems of Primary Care Treatment in the Military (RESPECT-Mil) Program is a collaborative care model developed by the Department of Defense (DoD) to enhance recognition and management of depression and PTSD. The program’s model of integrating PCPs, care facilitators, and behavioral health specialists in treating PTSD was recently evaluated with a randomized control trial comparing patients receiving collaborative care versus usual care (Schnurr et al., 2013). Patients receiving collaborative care were more likely to have a mental health visit and receive antidepressants, although, similar to findings of previous studies (Seal et al., 2010; 2011), less than 10% of the full sample received enough psychotherapy visits to qualify as evidence-based treatment. Moreover, there were no differences between groups in PTSD symptom improvement. The PCPs in that study received a 1-hr training in diagnosis and treatment of PTSD, but their knowledge was not evaluated. In addition, the PCPs were not required to identify or diagnose PTSD, and patient engagement was primarily the role of the case manager rather than the physician. The authors surmised that the collaborative care model may have been more effective if there had been more PCP involvement in assessment and diagnosis. PC-MHI is an important innovation, but many veterans and trauma victims seek care outside the VA and DoD systems (e.g., Borowsky & Cowper, 1999), without access to colocated integrated primary care-mental health providers. In addition, the presence of mental health providers within primary care does not preclude the need for PCPs to be knowledgeable in PTSD. Finally, findings of these early studies examining the effectiveness of integrated care models suggest that despite the presence of mental health providers in primary care clinics, veterans are still not receiving adequate mental health treatment or demonstrating significant symptom improvement (Schnurr et al., 2013; Seal et al., 2011). Thus, there remains a need to build competency in assessment and initial management of PTSD symptoms in primary care, both within and outside the VA. In particular, PCPs will be relied upon to explain mental health interventions to patients and encourage adherence. Also, short of referring patients to mental health, PCPs often are responsible for first-line medication management and for treatment of related symptoms (e.g., sleep disorders, pain) in patients with PTSD. Given time and cost constraints, geographic barriers, and scheduling challenges, use of the Internet for primary care training may represent an ideal educational tool. Recent Web-based training programs for PCPs on other topics, such as managing intimate partner violence (Short, Surprenant, & Harris, 2006), at-risk drinking (Gannon, Qaseem, Snow, & Turner, 2011), and chronic pain (Harris et al., 2008) in primary care settings have shown that medical education provided over the Web can demonstrably improve clinical competency. A meta-analysis of 201 Web-based continuing medical education (CME) trainings showed large effect sizes for improved participant knowledge, skills, and clinical practice behaviors compared with no educational offerings and showed comparable effects to those observed in in-person trainings (Cook et al., 2008). The DoD and VA have made efforts to provide training for PCPs in postdeployment mental health problems such as PTSD (Department of Veterans Affairs, 2002; Engel et al., 2008). These trainings are lengthy and provide a step-by-step approach to screening for PTSD and initiating medications. However, existing

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

WEB-BASED PTSD TRAINING

trainings do not provide education about differential diagnosis, comorbid conditions, and psychotherapeutic techniques for PTSD. To our knowledge, there are no published reports evaluating the efficacy of Web-based PTSD education programs for PCPs. To address these gaps, we developed and evaluated a brief, self-paced, online, and narrated training entitled “PTSD Training for Primary Care Providers.” It was designed for a broad audience of PCPs who treat patients with PTSD both within VA and outside of the VA health care system. The training focused on militaryrelated PTSD, but generalized to other sources of trauma (e.g., intimate partner violence, child maltreatment, motor vehicle accidents). The main goals of the training were to educate PCPs about detection and assessment of PTSD symptoms, evidence-based pharmacological and psychotherapeutic treatments for PTSD, and comorbid conditions and differential diagnoses. Our objective was not only to train PCPs to assess for and provide initial management of PTSD symptoms, but to be able to educate and motivate patients to seek out specialty mental health care when needed. We utilized clinical vignettes to illustrate concepts and to model a PCP working with apprehensive veterans with PTSD symptoms. We hypothesized that after the training PCPs would demonstrate significant improvements in PTSD-related knowledge and increased comfort with PTSD-related skills. In addition, we hypothesized that these gains would be maintained for at least 30 days after the training and that PCPs would be able to apply course material in their daily practice. We also collected detailed feedback about the content, delivery, and technical aspects of the online training to inform future implementation and dissemination efforts.

Method Participants The online “PTSD Training for Primary Care Providers” course was accredited by the University of California, San Francisco (UCSF) Continuing Medical Education (CME) Program. PCPs both within and outside of VA settings were recruited for the pilot training and evaluation study in two main ways: (1) E-mails to primary care clinic leads in the VA, DoD, and other health care systems, such as health maintenance organizations and community health centers, and (2) E-mails to roughly 5,000 clinicians who subscribed to the UCSF CME listserve. Potential participants were directed to click on an online hyperlink where they read an information sheet describing the study. Those consenting to participate then completed an eligibility screen. Eligible participants were English-speaking PCPs, including licensed physicians (internists, family practitioners, and pediatricians), nurse practitioners, physician assistants, as well as trainees and students in these fields. The study was approved by the Committee on Human Research, University of California, San Francisco, the Human Research Protection Program at the San Francisco VA Medical Center, and the funder, the DoD Human Research Protection Office.

Study Procedures After eligibility was confirmed, the study coordinator assigned the participant a confidential and anonymous username and password and a link to the data management platform (DatStat 4.7.1) containing the online training and study assessment instruments.

155

Before the training, participants completed an online baseline assessment to collect sociodemographics and information about participants’ professions, level of training, practice or training environment, and whether participants’ caseloads included military service personnel or veterans. The baseline assessment (T0) was followed by an introductory and 4 online narrated video training modules (see Figure 1). The online training was designed to be self-paced; thus, participants could stop and start the training as needed. Immediately after finishing the training, participants completed the online posttraining assessment (T1) and one month later they completed the 30-day posttraining assessment (T2). Total participation time, including completion of all 3 study assessments, was approximately 2 hours. After study completion, participants were given the option of receiving 2 units of CME credit through UCSF or $25.

Intervention A clinical psychologist (K.W.S.) and a primary care internist (K.H.S.) collaborated to develop the training. The training was scripted and narrated and included multimedia didactic content, case presentations with Q & A, and videotaped clinical vignettes to demonstrate the use of training concepts and techniques in clinical practice. A beta version of the training and measures were piloted in a focus group of PCPs and mental health clinicians to solicit feedback before finalizing content. The main aim of the training is to educate PCPs in the detection, assessment, initial management, and referral of patients with PTSD. The evidence-based pharmacological and psychotherapeutic interventions described were drawn from the International Society for Traumatic Stress Studies Treatment Guidelines, which follow recommendations from Foa (2009). The Introductory Module provides a rationale for providing education about PTSD to PCPs, focusing on barriers to mental health care among veterans and the physical health problems associated with PTSD that are frequently seen in primary care. These issues are dramatized in a video of an Iraq veteran describing his symptoms to his girlfriend and expressing his reluctance to seek mental health treatment, but willingness to explore help for his sleep problems. His girlfriend encourages him to go to the VA to see his PCP, thus setting the stage for the PCP PTSD training. The introduction presents an overview of the four training modules that follow, described in Figure 1.

Outcomes Assessment Primary outcomes. The primary outcomes were change in knowledge from pre- to posttest following the training (T1) and retention of knowledge from pretest to 30-day follow-up (T2). We also evaluated change in level of comfort in practicing PTSDrelated skills addressed in the training T1 and T2 time points compared to baseline (T0). PTSD-related knowledge. We initially constructed a total of 10 multiple-choice knowledge questions that tested knowledge of the content in each of the training modules. Clinical problemsolving questions used patient scenarios. Questions were then reviewed by an outside expert in PTSD and piloted on the first six PCP participants who completed the training. Consistent with test construction theory (Cohen & Swerdlik, 2002), questions were

SAMUELSON ET AL.

156 Module Introduction

Content • Introduction to PTSD in primary care • Health effects of PTSD • Under-detection of PTSD in primary care • Barriers to PTSD treatment in primary care • •

Module 1: Detection and Assessment

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

• Module 2: Comorbid Conditions and Related Problems

• •

Module 3: Pharmacological Interventions in the Management of PTSD in Primary Care

•

Module 4: Psychotherapeutic Interventions for PTSD

•

• •

• • •

Clinical Vignettes

• • •

Figure 1.

PTSD diagnostic criteria Validated screening tools: PC-PTSD Screen & PTSD Checklist Sample assessment questions Introduction to common comorbid conditions (e.g., depression, TBI, adjustment disorder, etc.) Case studies with differential diagnoses to distinguish PTSD from related disorders with overlapping symptoms Reviews evidence-based medications for PTSD and related problems Includes dosage, side effects, and contraindications Cautions against use of benzodiazepines Introduces evidence-based psychotherapies for PTSD to assist PCPs in explaining them to patients when referring Introduces Motivational Interviewing to help with mental health treatment engagement Provides guidance for when to refer patients to specialty mental health Introduction: Iraq veteran describes PTSD symptoms, but will only go to see his PCP for his “sleep problems” Module 1: PCP assesses for PTSD symptoms and provides feedback to veteran. Module 3: PCP recommends prazosin for the treatment of nightmares to a veteran who is resistant to medications Module 4: PCP models (a) describing Prolonged Exposure Therapy and (b) Motivational Interviewing with patient

Description of training modules.

deleted if a high rate (⬎60%) of respondents answered them incorrectly after completing the course. The final test used in study analyses consisted of eight items. Item-total correlations of knowledge items assessed ranged from .42 to .57, indicating a convergence among test items (Clark & Watson, 1995). The means of the four module test scores ranged from 34.2% to 52.7% correct pretraining, which differed from a chance mean of 25% and left room for improvement, which is a necessary condition for a measure sensitive to knowledge gain. Sensitivity to change was assessed through paired t tests assessing change in knowledge mean scores from baseline to posttraining. The knowledge mea-

sure was successful in showing significant increase in knowledge scores. Construct validity was established through a correlation between knowledge mean score and comfort mean score, described below (r ⫽ .25, p ⫽ .035). Participants who rated themselves as more comfortable applying PTSD-related skills scored higher on the knowledge test. Comfort. Twelve questions assessed PCPs’ self-reported comfort with PTSD-related skills targeted in the training. Items were scored using a Likert scale from 1 to 4 with higher scores indicating greater comfort. Skills related to each module were included. Examples of questions included an assessment of com-

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

WEB-BASED PTSD TRAINING

fort level in using a screening measure for PTSD in primary care (Module 1), comfort in differentiating PTSD from acute stress disorder (Module 2), comfort in prescribing medications for PTSD and nightmares specifically (Module 3), and comfort in describing psychotherapeutic options for PTSD (Module 4). Secondary outcomes: Feedback on the training. Immediately after the training and 30 days later, participants rated various aspects of the training on a 4-point Likert scale. Participants provided open-ended qualitative feedback including overall strengths and weaknesses, as well as specific comments about the utility and relevance of training content to their own clinical practice. In addition, we solicited detailed feedback about technical aspects of the online training that may have affected their ability to learn.

Data Analysis Change in knowledge and comfort were evaluated statistically by comparing changes in mean test scores over time using paired t tests based on null hypotheses of zero change from baseline. For both the Knowledge and Comfort items, for each time point, we calculated both the overall mean scores and subscores for each of the four modules. Qualitative data were extracted from the online data management system by collating each participant’s response to open-ended questions. An expert in qualitative data analysis (C.J.K.) used standard constant comparison techniques (Bryant & Charmaz, 2010) to construct general categories that accounted for both similarity and difference across responses. Similar responses were grouped into superordinate categories that are reported as general themes.

Results Participants From October 20, 2010 to December 6, 2012, 224 eligible PCPs were enrolled in the study; 117 PCPs completed the pretest, but may or may not have competed the online training; 81 completed the pretest, the online training, and the posttest; and 73 completed the online training and all three study assessments including the 30-day follow-up assessment. Background characteristics of the participants are included in Table 1. Of the 73 PCPs in the analytic sample, 69% were female and were primarily physicians (60%) and nurse practitioners (18%). Dominant practice settings were VA facilities (21%), community health centers (23%), and university-based practices (22%). Most reported caring for veterans or active duty military personnel (63%). We conducted analyses to determine whether the 73 participants who completed all three study assessments differed from the 44 participants who completed the pretest but either did not complete the training, or did not complete one of the follow-up assessments. The two groups were not significantly different on any background characteristics (all ps ⬎ .15) or on baseline knowledge of PTSD, t(116) ⫽ .11, p ⫽ .91.

Knowledge of PTSD-Related Detection and Initial Management of Symptoms Overall baseline PTSD-related knowledge was poor, with mean pretest scores of 45.9%. Completion of the PTSD training led to

157

Table 1 Characteristics of Participants Participant characteristic Gender Female Profession Physician - Internal medicine Physician - Family practice Physician - Pediatrics Physician’s assistant Nurse practitioner Student Post-graduate Years of clinical practice, mean (SD) Practice type Community health center University practice Veterans Affairs Hospital-based practice Department of Defense Health maintenance organization Locum tenens Federal or state clinics Other Main work focus Providing clinical services Administration Teaching Research Student Caseload includes military service personnel/veterans Yes

n

(%)

50

68.5%

21 20 3 2 13 8 6 13.5

28.8% 27.4% 4.1% 2.7% 17.8% 11.0% 8.2% (13.0)

17 16 15 10 1 4 3 1 6

23.3% 21.9% 20.5% 13.7% 1.4% 5.5% 4.1% 1.4% 8.2%

54 5 3 2 9

74.0% 6.8% 4.1% 2.7% 12.3%

42

62.7%

significant improvements in PTSD-related knowledge from pretest (T0) to posttest (T1) (mean change score ⫽ 28.8%, SD ⫾ 21.5%), representing a significant increase in knowledge, t(72) ⫽ 11.4, p ⬍ .001 (see Table 2). Overall PTSD-related knowledge was retained over a 30-day period with follow-up mean change scores (T2) maintaining a gain of 17.5% (SD⫹/⫺ 22.1%) compared with baseline (T0), t(72) ⫽ 6.7, p ⬍ .001. Compared with baseline (T0) knowledge, participants demonstrated significant improvements at posttest (T1) in mean knowledge scores for each of the four modules (all ps ⱕ .001); mean change scores ranged from 20% to 47% for knowledge of comorbid conditions, psychotherapeutic interventions, detection and assessment of PTSD, and pharmacological interventions in ascending order. At 30-day follow-up (T2), participants demonstrated significant retention for all modular content (all ps ⬍ .02), with the exception of psychotherapeutic interventions for PTSD.

Comfort in Providing PTSD-Related Care in Primary Care At posttraining, and at the 30-day follow-up assessment, participants reported increased comfort in providing PTSD-related care for their primary care patients (see Table 2). At both time points, participants demonstrated significantly improved comfort with skills pertaining to all of the modules (all ps at both time points ⬍ .001). At the item level, the biggest posttraining gains in comfort level sustained at 30 days were as follows: (1) assessing and prescribing medication for a patient with PTSD-related nightmares, (2) using the Primary Care PTSD (PC-PTSD) screen to

SAMUELSON ET AL.

158

Table 2 Pre-Training, Immediate Post-Training, and Follow-Up Knowledge Scores and Self-Reported Comfort in Assessing and Treating PTSD Post-training (T1)

Pre-training (T0)

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Module Knowledge (percent answering correctly) of each module assessed Module 1: Detection and assessment of PTSD Module 2: Comorbid conditions and differential diagnoses Module 3: Pharmocological interventions for PTSD Module 4: Psychotherapeutic interventions for PTSD Overall knowledge score Comfort (Likert scale of 1–4, with higher scores indicating greater comfort) Module 1: Detection and assessment of PTSD Module 2: Comorbid conditions and differential diagnoses Module 3: Pharmocological interventions for PTSD Module 4: Psychotherapeutic interventions for PTSD Overall comfort score

Follow-up (T2)

Mean score

Mean score

Mean change (SD)

p value

Mean score

Mean change (SD)a

p value

52.7% 51.6% 36.3% 34.2% 45.9%

79.5% 71.2% 83.6% 57.5% 74.7%

26.7% (36.4%) 19.6% (30.8%) 47.3% (40.7%) 23.3% (59.0%) 28.8% (21.5%)

⬍.001 ⬍.001 ⬍.001 .001 ⬍.001

64.4% 64.4% 70.5% 43.8% 63.4%

11.6% (40.4%) 12.8% (34.5%) 34.2% (35.2%) 9.6% (58.1%) 17.5% (22.1%)

.016 .002 ⬍.001 .163 ⬍.001

2.3 2.2 2.6 2.0 2.4

3.3 3.1 3.2 3.0 3.2

1.0 (0.7) 0.9 (0.7) 0.6 (0.6) 1.0 (0.8) 0.8

⬍.001 ⬍.001 ⬍.001 ⬍.001 ⬍.001

3.1 3.0 3.1 2.8 3.2

0.8 (0.7) 0.8 (0.6) 0.5 (0.5) 0.8 (0.7) 0.8

⬍.001 ⬍.001 ⬍.001 ⬍.001 ⬍.001

Note. n ⫽ 73. Change at follow-up represents change from pre-training scores.

a

quickly assess patients for PTSD, (3) differentiating acute stress disorder from PTSD, (4) explaining psychotherapeutic options for PTSD treatment to patients, and (5) prescribing medication to treat PTSD.

Participant Feedback The majority of participants (86.3%) endorsed the training as moderately to very enjoyable, educational (97.3%), not burdensome (82.2%), and effectively delivered as online content (95.9%). Of note, nearly 30% found the training to be too long. After

completing the training, 90% anticipated being able to apply the training to their clinical practices and at the 30-day follow-up evaluation, 47% reported having done so. Also, at 30-days, 86% reported that they would recommend this training to their PCP colleagues. As part of the training, participants had the opportunity to write responses to open-ended questions; common responses are summarized in Table 3. Before completing the training, all participants were asked why they participated in the training. The majority of providers expressed interest in developing clinical skills in treating patients exposed to trauma in general and PTSD

Table 3 Participants’ Pre- and Post-Training Qualitative Feedback Topical area

Quotations

Reasons for participation

• I want to better identify and treat my patients with PTSD. • Hoping this will make me a better doctor for veterans. • To gain better strategies for patients with complex PTSD related to non-combat trauma (a significant population in my practice) and to learn about assessment and management of combat trauma (a new area of my practice).

Relevance to clinical work

• I appreciated the brief introduction to pharmacologic and psychotherapies currently demonstrated to be effective and the supporting brief discussion as to WHY! • The need to explore issues of intrusive thoughts, nightmares, etc. Will also use this in evaluating patients with insomnia in case this is due to unappreciated PTSD. • Motivational interviewing [is one] application to help them [patients] make the choice to get treatment.

Training strengths

• The mixture of learning modalities kept me interested and learning. • You can do the module anywhere, anytime . . . Covers a topic that PCPs may not read about in their spare time (given all the other medical topics that they need to stay current on). • Watching the videos of interviewing, and veterans describing their symptoms made it more real than just reading or being told the symptoms or the interviewing techniques. • I liked the convenience of doing as many modules as I liked at a time that was convenient for me. In fact, the idea of a live webcast doesn’t appeal to me as I would have to commit to a specific time.

Training weaknesses

• Online training is challenging because it can be passive. I don’t know that this version has enough active steps by the learner to keep them engaged if they are alone watching the training in their own office. I suspect many people would multi-task away from the sessions. • It would have been nice to have the training more interactive — i.e., ask a question, then have to click a box, then the answer would appear.

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

WEB-BASED PTSD TRAINING

in particular. After the training, participants gave substantive feedback about strengths and weaknesses of the training, as well as the feasibility and technical aspects of online delivery. Providers felt that the training was relevant and applicable for their practice, particularly content about evidence-based treatment choices for PTSD, recognizing possible PTSD symptoms, and the skills associated with Motivational Interviewing to motivate patients for seeking mental health treatment. The most commonly reported strength was the multimedia video-taped clinical vignettes that illustrated main points. Participants also appreciated the asynchronous nature of online training and the comprehensive and logical organization. Weaknesses included a lack of interactive components and technical difficulties.

Discussion Multiple studies of veterans and civilians suffering from mental health problems reveal that a substantial proportion underutilize mental health services (Hoge et al., 2004; Tanielian & Jaycox, 2008; Wang et al., 2005) and are more likely to present to primary care (Department of Veterans Affairs, 2002). Following the influx of OEF/OIF/OND veterans presenting to VA primary care clinics with symptoms of PTSD, researchers observed that PCPs lacked the necessary training and knowledge base to provide appropriate diagnosis and treatment for PTSD (Magruder et al., 2005). To address these issues, the VA implemented policies in 2007 to colocate mental health providers in primary care settings. However, early studies examining the effectiveness of these initiatives show that, despite improvements in initial access to mental health care, veterans are not receiving an adequate dose of mental health treatment or experiencing significant symptom improvement (Schnurr et al., 2013; Seal et al., 2011). Among the many possible reasons for this gap in services may be a lack of PTSD-related competency and engagement among PCPs. Our Web-based training educates PCPs about initial PTSD assessment and symptom management, and works to build competency among PCPs to explain and promote evidence-based PTSD treatments to primary care patients as well as to motivate and encourage their retention in mental health treatment. This pilot study demonstrated preliminary effectiveness of a 70-min Web-based training for educating PCPs in the assessment and treatment of PTSD in primary care settings. PCPs who had little previous knowledge of PTSD demonstrated significant shortterm gains in PTSD-related knowledge and comfort after completing the training. The largest gains were found in knowledge of pharmacological interventions. Results from this study add to a growing literature demonstrating the effectiveness of Web-based curricula in the continuing education of physicians (Cook et al., 2008). Although there were initial large increases in PTSD knowledge, gains had diminished somewhat after 30 days, although still significantly improved compared with baseline. Gains in self-reported comfort level in practicing learned skills did not diminish after 30 days, however, indicating that the PCPs may feel more prepared and confident to treat patients with PTSD in practice. In one content domain, psychotherapies for PTSD, knowledge was not retained but comfort level in practicing the skill was retained, suggesting that PCPs were not aware of their diminished knowledge and might require further reinforcement, booster training,

159

clinical practice, and/or mentoring. Of note, the VA National Center for PTSD offers PTSD mentoring hotline and consultation programs that provide clinical support for VA clinicians, but to date, these resources have not been promoted to PCPs. Moreover, PCPs outside of the VA system may not be aware of these resources, although increasingly their caseloads include veterans of Iraq and Afghanistan with symptoms of PTSD. In this sample targeting both VA and non-VA PCPs, 63% reported seeing veterans in their practices. Thus, more clinical resources targeted at supporting and mentoring PCPs, both within and outside the VA, could greatly expand access to PTSD treatment for patients who might not otherwise receive it. Our results also showed that although PCPs appreciated learning about evidence-based treatments for PTSD, including hypothesized underlying mechanisms, their knowledge and understanding of these psychotherapies were not maintained after 30 days. Having a trusted PCP explain in plain language the rationale for evidence-based trauma-focused psychotherapies to a patient suffering from PTSD can be critical in encouraging patients to initiate and continue with treatment, particularly as these therapies can be emotionally trying for patients and large numbers of patients drop out prematurely (Maguen, Madden, Cohen, Bertenthal, & Seal, 2012; Seal et al., 2010). Participants reported finding the clinical vignettes particularly useful and enjoyable. The use of vignettes of a PCP modeling assessment and intervention techniques in training curricula can be particularly helpful for PCPs who report a lack of self-efficacy in working with trauma survivors (Green et al., 2011). For example, one vignette featured a PCP demonstrating Motivational Interviewing (Miller & Rollnick, 2002) with a patient reluctant to attempt therapy or medication for his PTSD symptoms, and several participants reported that this was one of the most useful components of the training. The need for educating PCPs in techniques to effectively motivate patients to consider pharmacological treatments is underscored by research showing that the most commonly reported barrier to seeking treatment among military personnel is concerns about medication side effects (Tanielian et al., 2008). Some limitations of this pilot study should be noted. First, as a project focused primarily on curriculum development, we approached the evaluation of the program’s effectiveness as a secondary goal and developed measures of knowledge that specifically targeted skills addressed in the curriculum. Future research examining effectiveness of this training program might include more robust, validated measures of knowledge and attitude change. It would have also been helpful to ask providers specifically whether they work in clinics with embedded mental health providers. Second, this was not a randomized, controlled trial that included a control group of PCPs who did not receive the training. In addition, although we assessed 30-day retention in knowledge and comfort with PTSD-related skills, we did not assess for longterm impact of the training, nor did we audit objective evidence of the training’s impact on clinical practice through medical chart review. Third, there were certainly biases introduced by a convenience sample of participants who opted to participate in the training; however, there were no significant differences in background characteristics or knowledge of PTSD between PCPs who completed all aspects of the training and assessment and those who did not. Fourth, the sample size was relatively small, although it was adequately powered to test a circumscribed set of study

SAMUELSON ET AL.

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

160

hypotheses. In addition, despite heavy recruitment, response and retention rates of PCP participants were low, which raises questions about the feasibility of larger-scale implementation. Only 33% of enrolled participants completed all aspects of the study, suggesting that the training and/or assessments were timeconsuming or burdensome. Future directions to address this problem might include shortening the training and adapting the training for mobile technologies. Finally, our curriculum focused primarily on the management of PTSD in veterans, specifically veterans of Iraq and Afghanistan. Future modifications of this curriculum would further broaden content to more fully address noncombat trauma. In sum, PCPs who completed an online 70-min self-paced training in the assessment and initial management of PTSD showed significant short-term gains in PTSD-related knowledge, and reported increased comfort in PTSD-related clinical skills assessed. Future implementation efforts will incorporate participant feedback that suggested more interactive content and resolution of Web-related technical problems. Despite the recent VA and DoD clinical practice innovations represented by the Primary Care Mental Health Integration initiative and RESPECT-mil, there are still numerous barriers to specialty mental health treatment and a demonstrated preference among PTSD patients for accessing treatment through primary care. Increasing PTSD competency among PCPs may help increase the reach of PTSD detection and symptom management in many patients who might not otherwise receive care.

References American Psychological Association (APA) Presidential Task Force on Military Deployment Services for Youth, Families, and Service Members. (2007). The psychological needs of U.S. military service members and their families: A preliminary report. Washington, DC: Author. Borowsky, S. J., & Cowper, D. C. (1999). Dual use of VA and non-VA primary care. Journal of General Internal Medicine, 14, 274 –280. doi:10.1046/j.1525-1497.1999.00335.x Bryant, A., & Charmaz, K. (2010). The Sage handbook of grounded theory. Newbury Park, CA: Sage. Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309 –319. doi:10.1037/1040-3590.7.3.309 Cohen, R. J., & Swerdlik, M. E. (2002). Psychological testing and assessment: An introduction to tests and measurement (5th ed.). New York, NY: McGraw-Hill Companies, Inc. Cohen, B. E., Gima, K., Bertenthal, D., Kim, S., Marmar, C. R., & Seal, K. H. (2010). Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan veterans. Journal of General Internal Medicine, 25, 18 –24. doi:10.1007/s11606009-1117-3 Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. Journal of the American Medical Association, 300, 1181–1196. doi:10.1001/jama.300.10.1181 Department of Veterans Affairs. (2002). Post-traumatic stress disorder: Implications for primary care. Employee Education System Veterans Health Initiative Independent Study Course. Washington, DC: Author. Dobie, D. J., Kivlahan, D. R., Maynard, C., Bush, K. R., Davis, T. M., & Bradley, K. A. (2004). Posttraumatic stress disorder in female veterans: Association with self-reported health problems and functional impairment. Archives of Internal Medicine, 164, 394 – 400. doi:10.1001/ archinte.164.4.394

Dobscha, S. K., Gerrity, M. S., & Ward, M. F. (2001). Effectiveness of an intervention to improve primary care provider recognition of depression. Effective Clinical Practice, 4, 163–171. Engel, C. C., Oxman, T., Yamamoto, C., Gould, D., Barry, S., Stewart, P., . . . Dietrich, A. J. (2008). RESPECT-Mil: Feasibility of a systems-level collaborative care approach to depression and post-traumatic stress disorder in military primary care. Military Medicine, 173, 935–940. Foa, E. (2009). Effective treatments for PTSD (2nd ed.). New York, NY: Guilford Press. Fortney, J. C., Burgess, J. F. Jr., Bosworth, H. B., Booth, B. M., & Kaboli, P. J. (2011). A re-conceptualization of access for 21st century healthcare. Journal of General Internal Medicine, 26, 639 – 647. doi:10.1007/ s11606-011-1806-6 Gannon, M. A., Qaseem, A., Snow, V., & Turner, B. (2011). Raising achievement: Educating physicians to address effects of at-risk drinking on common diseases. Quality in Primary Care, 19, 43– 47. Green, B. L., Kaltman, S., Frank, L., Glennie, M., Subramian, A., FrittsWilson, M., . . . Chung, J. (2011). Primary care providers’ experience with trauma patients: A qualitative study. Psychological Trauma: Theory, Research, Practice, and Policy, 3, 37– 41. doi:10.1037/a0020097 Harman, J. S., Mulsant, B. H., Kelleher, K. J., Schulberg, H. C., Kupfer, D. J., & Reynolds, C. F., 3rd. (2001). Narrowing the gap in treatment of depression. International Journal of Psychiatry in Medicine, 31, 239 – 253. doi:10.2190/Q3VY-T8V9-30MA-VC5C Harris, J. M., Elliott, T. E., Davis, B. E., Chabal, C., Fulginiti, J. V., & Fine, P. G. (2008). Educating generalist physicians about chronic pain: Live experts and online education can provide durable benefits. Pain Medicine, 9, 555–563. doi:10.1111/j.1526-4637.2007.00399.x Higgins, E. S. (1994). A review of unrecognized mental illness in primary care: Prevalence, natural history, and efforts to change the course. Archives of Family Medicine, 3, 908 –917. doi:10.1001/archfami.3.10 .908 Hoge, C., Castro, C., Messer, S., McGurk, D., Cotting, D., & Koffman, R. (2004). Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. The New England Journal of Medicine, 351, 13–22. doi:10.1056/NEJMoa040603 Kessler, R. C., Berglund, P., Delmer, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM–IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 593– 602. doi:10.1001/archpsyc.62 .6.593 Kessler, R. C., Sonnega, A., Bromet, E., Hughes, M., & Nelson, C. B. (1995). Posttraumatic stress disorder in the National Comorbidity Survey. Archives of General Psychiatry, 52, 1048 –1060. doi:10.1001/ archpsyc.1995.03950240066012 Kulka, R. A., Schlenger, W. E., Fairbank, J. A., Hough, R. L., Jordan, B. K., Marmar, C. R., . . . Grady, D. A. (1990). Trauma and the Vietnam War generation: Report of findings from the National Vietnam Veterans Readjustment Study. New York, NY: Brunner/Mazel. Magruder, K. M., Frueh, B. C., Knapp, R. G., Davis, L., Hamner, M. B., Martin, R. H., . . . Arana, G. W. (2005). Prevalence of posttraumatic stress disorder in Veterans Affairs primary care clinics. General Hospital Psychiatry, 27, 167–179. doi:10.1016/j.genhosppsych.2004.11.001 Maguen, S., Madden, E., Cohen, B. E., Bertenthal, D., & Seal, K. H. (2012). Time to treatment among veterans of conflicts in Iraq and Afghanistan with psychiatric diagnoses. Psychiatric Services, 63, 1206 – 1212. doi:10.1176/appi.ps.201200051 Miller, W., & Rollnick, S. (2002). Motivational Interviewing: Preparing people for change. (2nd ed.). New York, NY: Guilford Press Publications. Ouimette, P., Vogt, D., Wade, M., Tirone, V., Greenbaum, M. A., Kimerling, R., . . . Rosen, C. S. (2011). Perceived barriers to care among Veterans Health Administration patients with posttrauamtic stress disorder. Psychological Services, 8, 212–223. doi:10.1037/a0024360

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

WEB-BASED PTSD TRAINING Pietrzak, R. H., Johnson, D. C., Goldstein, M. B., Malley, J. C., & Southwick, S. M. (2009). Perceived stigma and barriers to mental health care utilization among OEF-OIF veterans. Psychiatric Services, 60, 1118 –1122. doi:10.1176/appi.ps.60.8.1118 Regier, D. A., Narrow, W. E., Rae, D. S., Manderscheid, R. W., Locke, B. Z., & Goodwin, F. K. (1978). The de facto US mental and addictive disorder service system: Epidemiologic catchment area prospective 1-year prevalence rates of disorders and services. Archives of General Psychiatry, 35, 685– 693. doi:10.1001/archpsyc.1978.01770300027002 Rodriguez, B. F., Weisberg, R. B., Pagano, M. E., Machan, J. T., Culpepper, L., & Keller, M. B. (2004). Frequency and patterns of psychiatric comorbidity in a sample of primary care patients with anxiety disorders. Comprehensive Psychiatry, 45, 129 –137. doi:10.1016/j.comppsych .2003.09.005 Schnurr, P. P., Friedman, M. J., Oxman, T. E., Dietrich, A. J., Smith, M. W., Shiner, B., . . . Thurston, V. (2013). RESPECT-PTSD: Reengineering systems for the primary care treatment of PTSD, a randomized controlled trial. Journal of General Internal Medicine, 28, 32– 40. doi:10.1007/s11606-012-2166-6 Schnurr, P. P., & Jankowski, M. K. (1999). Physical health and posttraumatic stress disorder: Review and synthesis. Seminars in Clinical Neuropsychiatry, 4, 295–304. Seal, K. H., Cohen, G., Bertenthal, D., Cohen, B. E., Maguen, S., & Daley, A. (2011). Reducing barriers to mental health and social services for Iraq and Afghanistan veterans: Outcomes of an Integrated Primary Care Clinic. Journal of General Internal Medicine, 26, 1160 –1167. doi: 10.1007/s11606-011-1746-1 Seal, K. H., Maguen, S., Cohen, B., Gima, K. S., Metzler, T. J., Ren, L., . . . Marmar, C. R. (2010). VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. Journal of Traumatic Stress, 23, 5–16. Shalev, A. Y., Freedman, S., Peri, T., Brandes, D., Sahar, T., Orr, S. P., & Pitman, R. K. (1998). Prospective study of posttraumatic stress disorder

161

and depression following trauma. The American Journal of Psychiatry, 155, 630 – 637. Short, L. M., Surprenant, Z. J., & Harris, J. M. (2006). A community-based trial of an online intimate partner violence CME program. American Journal of Preventive Medicine, 30, 181–185. doi:10.1016/j.amepre .2005.10.012 Tanielian, T., & Jaycox, L. H. (Eds.). (2008). Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery. Santa Monica, CA: RAND Corporation. doi:10.1037/ e527612010-001 Thomas, J. L., Wilk, J. E., Riviere, L. A., McGurk, D., Castro, C. A., & Hoge, C. W. (2010). Prevalence of mental health problems and functional impairment among active component and National Guard soldiers 3 and 12 months following combat in Iraq. Archives of General Psychiatry, 67, 614 – 623. doi:10.1001/archgenpsychiatry.2010.54 Wang, P., Berglund, P., Olfson, M., Pincus, H., Wells, K., & Kessler, R. (2005). Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 603– 613. doi:10.1001/archpsyc.62.6 .603 Weisberg, R. B., Dyck, I., Culpepper, L., & Keller, M. B. (2007). Psychiatric treatment in primary care patients with anxiety disorders: A comparison of care received from primary care providers and psychiatrists. The American Journal of Psychiatry, 164, 276 –282. doi:10.1176/appi .ajp.164.2.276 Zeiss, A., & Karlin, B. (2008). Integration of mental health and primary care services in the Department of Veterans Affairs Health Care System. Journal of Clinical Psychology in Medical Settings, 15, 73–78. doi: 10.1007/s10880-008-9100-4

Received May 1, 2013 Revision received August 2, 2013 Accepted August 7, 2013 䡲