The Journal of Emergency Medicine, Vol. -, No. -, pp. 1–11, 2014 Copyright Ó 2014 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter
http://dx.doi.org/10.1016/j.jemermed.2014.09.059
Education EMERGENCY MEDICINE RESIDENTS’ KNOWLEDGE OF MECHANICAL VENTILATION Susan R. Wilcox, MD,*† Todd A. Seigel, MD,‡§ Tania D. Strout, PHD, RN,k Jeffrey I. Schneider, MD,{ Patricia M. Mitchell, RN,{ Evie G. Marcolini, MD,** Michael N. Cocchi, MD,††‡‡ Howard A. Smithline, MD,§§ Lucienne Lutfy-Clayton, MD,§§ Marie Mullen, MD,kk Jonathan S. Ilgen, MD,{{ and Jeremy B. Richards, MD*** *Department of Emergency Medicine, †Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, ‡Department of Emergency Medicine, Rhode Island Hospital, Providence, Rhode Island, §University of California San Francisco, San Francisco, California, kDepartment of Emergency Medicine, Maine Medical Center, Portland, Maine, {Department of Emergency Medicine, Boston Medical Center, Boston, Massachusetts, **Department of Emergency Medicine, Yale–New Haven Hospital, New Haven, Connecticut, ††Department of Emergency Medicine, ‡‡Department of Anesthesia Critical Care, Division of Critical Care, Beth Israel Deaconess Medical Center, Boston, Massachusetts, §§Department of Emergency Medicine, Baystate Medical Center, Springfield, Massachusetts, kkDepartment of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, {{Division of Emergency Medicine, University of Washington School of Medicine, Seattle, Washington, and ***Division of Pulmonary, Critical Care and Sleep Medicine, Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts Reprint Address: Susan R. Wilcox, MD, Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114
, Abstract—Background: Although Emergency physicians frequently intubate patients, management of mechanical ventilation has not been emphasized in emergency medicine (EM) residency curricula. Objectives: The objective of this study was to quantify EM residents’ education, experience, and knowledge regarding mechanical ventilation. Methods: We developed a survey of residents’ educational experiences with ventilators and an assessment tool with nine clinical questions. Correlation and regression analyses were performed to evaluate the relationship between residents’ scores on the assessment instrument and their training, education, and comfort with ventilation. Results: Of 312 EM residents, 218 responded (69.9%). The overall correct response rate for the assessment tool was 73.3%, standard deviation (SD) ± 22.3. Seventy-seven percent (n = 167) of respondents reported # 3 h of mechanical ventilation education in their residency curricula over the past year. Residents reported frequently caring for ventilated patients in the ED, as 64% (n = 139) recalled caring for $ 4 ventilated patients per
month. Fifty-three percent (n = 116) of residents endorsed feeling comfortable caring for mechanically ventilated ED patients. In multiregression analysis, the only significant predictor of total test score was residents’ comfort with caring for mechanically ventilated patients (F = 10.963, p = 0.001). Conclusions: EM residents report caring for mechanically ventilated patients frequently, but receive little education on mechanical ventilation. Furthermore, as residents’ performance on the assessment tool is only correlated with their self-reported comfort with caring for ventilated patients, these results demonstrate an opportunity for increased educational focus on mechanical ventilation management in EM residency training. Ó 2014 Elsevier Inc. , Keywords—mechanical ventilation; critical care; education; residents
INTRODUCTION Although emergency physicians frequently intubate critically ill patients in the emergency department (ED), management of mechanical ventilation traditionally has
Institutional Review Board approval was obtained at each participating institution.
RECEIVED: 14 April 2014; FINAL SUBMISSION RECEIVED: 15 August 2014; ACCEPTED: 30 September 2014 1
2
S. R. Wilcox et al.
not been emphasized in emergency medicine (EM) practice and residency training curricula (1–4). Nonetheless, management of positive-pressure ventilation can influence outcomes of critically ill patients for several conditions commonly encountered in EM practice (5– 10). For example, patients with asthma, once intubated, are at high risk of complications and deterioration (7). Low-tidal-volume ventilation improves mortality in patients with acute respiratory distress syndrome (ARDS) (11). Careful management of oxygenation and ventilation by emergency care providers has been shown to improve outcomes in intubated patients with traumatic brain injury (12,13). Furthermore, due to hospital crowding, emergency physicians may be primarily responsible for prolonged management of mechanically ventilated patients (14–16). Even for patients who are in the ED only briefly, evidence suggests that ventilator-induced lung injury can occur in as little as 20 min (17). We designed this study to quantify EM residents’ experience and knowledge regarding mechanical ventilation. We surveyed EM residents to assess how frequently they receive education on mechanical ventilation, frequency with which they care for mechanically ventilated patients in the ED, and their subjective comfort with managing patients on mechanical ventilation. In addition, we created an assessment tool to characterize residents’ application of knowledge regarding mechanical ventilation involving common emergency scenarios. We hypothesized that the residents with the most experience in managing mechanical ventilators in the ED would perform superiorly on the knowledge assessment tool. MATERIALS AND METHODS Survey Instrument Development To quantify EM residents’ training experiences, we developed a 5-point Likert scale survey tool to assess residents’ level of training, hours of education on mechanical ventilation, and exposure to the topic at local and national conferences (Appendix). The survey also queried residents regarding the frequency with which they care for mechanically ventilated patients and their comfort with managing ventilators. Survey responses were dichotomized as affirmative or negative: the responses ‘‘often’’ and ‘‘frequently’’ were defined as affirmative responses, whereas ‘‘never,’’ ‘‘rarely,’’ or ‘‘don’t know’’ were defined as negative. Any responses left blank were scored as ‘‘don’t know.’’ Assessment Instrument Development A literature review did not identify preexisting assessment tools for assessing EM residents’ knowledge regarding
clinical issues involving mechanical ventilation. We identified one validated test with a focus on management of mechanical ventilation in the intensive care unit (ICU) designed for Internal Medicine residents, and this test served as a foundation for development of our assessment tool. A project team with backgrounds in EM and critical care, and experience in educational survey development, generated an assessment instrument with questions specific to EM (18–20). We created a series of questions involving key principles consistent with outlined objectives for resident education in mechanical ventilation, and similar in style and content to the validated test for internal medicine residents (18,21). These principles included respiratory physiology, modes of mechanical ventilation, and complications of mechanical ventilation (18,21). The content was modified to be relevant to management of mechanically ventilated patients in the ED. Specific clinical scenarios emphasized emergency management of ventilated patients with asthma, ARDS, and traumatic brain injury, as evidence supports the importance of conscientious ventilator management in these clinical scenarios (5,7,10–12,22–28). Our assessment tool was formatted using multiplechoice questions, an accepted means of assessing clinical competence, following guidelines recommended by the National Board of Medical Examiners (29–32). To enhance validity, candidate questions were reviewed and edited by subject experts in an iterative fashion to optimize content, length, and relevance to the assessment tool’s goals. Eleven faculty members from multiple institutions, with backgrounds ranging from community EM, academic EM, pulmonology/critical care, trauma surgery, anesthesiology, and critical care medicine, critically reviewed the survey and assessment tool. The faculty provided further comment, review, and editing of the questions. To assess validity regarding the response process, the survey and assessment tool were then piloted with faculty and senior EM residents to assess question clarity, to determine survey length, and identify potentially redundant questions (33,34). After piloting, the project was reviewed for final approval by EM faculty with critical care fellowship training who were not originally involved in the first two iterations of the survey development. Study Protocol The finalized versions of the survey and assessment tool were administered anonymously using Research Electronic Data Capture (REDCap, Nashville, TN) tools hosted at Massachusetts General Hospital (35). REDCap is a secure, Web-based application designed to support
EM Residents’ Knowledge of Mechanical Ventilation
data capture for research studies, providing an interface for validated data entry. The survey and assessment tool were distributed by email to all of the EM residents enrolled in eight EM residency training programs in the northeastern United States, for a total of 312 EM residents. These eight institutions include both 3-year and 4-year training programs. The survey was sent to all EM residents in each training program by a local investigator at each site. Residents at these institutions received an e-mail invitation to participate once a week for 3 weeks, beginning in early January 2013. The study protocol was approved by the institutional review boards of all participating institutions. Consent for participation in the survey was obtained from each resident at the time of participation, as the survey introduction stated that partaking of the survey indicated consent. Data Analysis Study data were exported into a Microsoft Excel (Microsoft Corporation, Redmond, WA) spreadsheet program and were then transferred into SPSS (v. 11.0, SPSS, Inc., Chicago, IL) for analysis. For all variables, missing data were excluded on a case-by-case basis. For the purposes of this study, we assumed the correct response rate for the assessment tool (test score) to be a surrogate for knowledge of mechanical ventilation. We examined the continuous outcome variable test score for normality in two ways. First, the outcome was examined visually using histograms and normal quantilequantile plots. After visual examination, Pearson’s second skewness coefficient was computed, revealing mild skew to the left, Sk2 = 0.61. Survey data regarding study participants and their training programs, mechanical ventilation educational experiences, and ventilator management experience were summarized using descriptive statistics. One-way analysis of variance was used to assess for differences in total test score across participating institutions. As our hypothesis was that the residents with the most exposure to managing mechanical ventilators in the ED would perform better on the knowledge assessment tool, we examined the relationship between these variables in several ways. Ordinary least-squares regression analyses were performed, with total test score serving as the outcome variable. The frequency with which residents managed mechanically ventilated patients was the predictor variable. To examine the relationship between these variables after controlling for other variables significantly correlated to test score in simple correlation analysis (Spearman’s r), hierarchical multiple regression models using the additional predictors of postgraduate year (PGY) of training, number of EM residency mechan-
3
ical ventilation lecture hours completed, number of hours of education outside EM residency curricula, and subjective comfort with managing mechanically ventilated ED patients were employed. Exploratory regression analyses were then conducted to determine which variables, alone and in combination, were the strongest predictors of total test score. In addition to assessing normality as described above, additional linear regression assumptions were evaluated using residual analyses and assessment of influence diagnostics. Multicollinearity was evaluated using variance inflation factors, which were all well below the recommended cut points. Multivariate logistic regression analyses were performed to evaluate the extent to which PGY of training, hours of residency curriculum-based formal ventilator management education, hours of extracurricular ventilator-related training, and the frequency with which participants reporting caring for ventilated patients in the ED influenced residents’ self-reported comfort with managing mechanically ventilated patients. Coefficient estimates, adjusted odds ratios (aORs), and 95% confidence intervals are reported for each variable. We accepted an alpha of < 0.05 as statistically significant. RESULTS Characteristics of the Study Subjects and Their Training Programs Study surveys were distributed to a total of 312 EM residents, with 219 residents responding (response rate = 70.2%). One resident completed only the first survey question and was subsequently dropped from the study, leaving data from 218 residents’ complete surveys for analysis (69.9%). Nine residents (4%) did not fully complete the knowledge assessment tool. Of the residents who completed the surveys, there was relatively equal distribution by PGY class (26.6% PGY-1, 28.0% PGY-2, 27.5% PGY-3, and 17.9% PGY-4.) The eight institutions included in this study represented both 3-year (37.7%) and 4-year EM programs (63.3%), and the response rate from the institutions ranged from 43.6% to 81.4%. Educational Experiences and Experience Managing Ventilated Patients Overall, study participants reported few residency-based educational opportunities regarding mechanical ventilation. Seventy-seven percent of residents (n = 167) reported receiving 3 or fewer hours of ventilation-related education in their residency curricula over the past year, and 34% (n = 73) reported receiving between 0 and 1 h of education. Similarly, 73% (n = 159) of residents described receiving 3 or fewer hours of mechanical
4
S. R. Wilcox et al.
ventilation education from nonresidency sources, including journal clubs, local or national conference attendance, and reading articles (Table 1). Responses regarding educational experiences were consistent among residents from the individual institutions, particularly when grouped by PGY of training. Conversely, residents reported frequently caring for mechanically ventilated patients in the ED. Sixty-four percent (n = 139) of residents reported that they care for 4 or more ventilated patients per month in the ED, and 22% (n = 48) reported caring for 10 or more ventilated patients per month. Fifty-three percent (n = 116)
Table 1. Residents’ Self-reported Education and Experience Regarding Mechanical Ventilation Respondents (%) How many hours of lecture on mechanical ventilation have you received through your EM residency in the last year? 0–1 2–3 4–5 More than 5 Don’t know How many hours of instruction have you received on mechanical ventilation from other EM sources (EM articles, discussion in EM journal clubs, EM lectures/ conferences, etc.) in the last year? 0–1 2–3 4–5 More than 5 Don’t know How often do you care for mechanically ventilated patients in the Emergency Department? Never Rarely (1–3 patients/month) Often (4–9 patients/month) Frequently (>10 patients/month) Don’t know How often do you feel comfortable managing mechanical ventilation and troubleshooting issues with ventilated patients in the ED? Never Rarely Often Frequently Don’t know Who primarily manages and makes changes to the mechanical ventilator for intubated patients in your ED? Respiratory therapist Nurse EM resident EM attending Other doctor (ICU physician, pulmonologist, etc.) Don’t know
73 (33.5) 94 (43.1) 18 (8.3) 12 (5.5) 21 (9.2)
68 (31.2) 91 (41.8) 24 (11.0) 18 (8.3) 17 (4.6)
18 (8.3) 58 (26.6) 91 (41.7) 48 (22.0) 3 (1.4)
15 (6.9) 82 (37.6) 94 (43.1) 22 (10.1) 5 (2.3)
170 (78.0) 0 (0) 27 (12.4) 8 (3.7) 0 (0) 13 (6.0)
EM = emergency medicine; ED = emergency department; ICU = intensive care unit.
of participants described feeling comfortable caring for mechanically ventilated ED patients ‘‘often’’ or ‘‘frequently’’; whereas 45% of respondents described ‘‘never’’ or ‘‘ rarely’’ feeling comfortable managing these patients. Only 16% (n = 35) described management of the ventilator as the responsibility of an emergency physician (resident or attending) at their institution. Seventy-eight percent (n = 170) identified the respiratory therapist as being primarily in charge of ventilator management (Table 1). Ventilator Management Knowledge The overall correct response rate for the nine-question assessment tool was 73.3%, SD 6 22.3%. Of the 218 residents who completed the assessment tool, 134 (61.5%) achieved a score of at least 70%. Statistically significant differences in total test scores were not noted between institutions (F = 1.035, p = 0.408). Increasing PGY level was associated with increased scores on the knowledge tool (Figure 1). Correlation analysis revealed statistically significant relationships between total test score and the frequency of managing ventilated ED patients, PGY of training, self-reported residency-related hours of ventilator management education, extracurricular ventilator education, and level of comfort with managing ventilated patients. The relationships between total test score and residency design or who bears primary responsibility for ventilator management were not significant (Table 2). Multivariate Results We evaluated the relationship between total test score and exposure to mechanical ventilation management with multivariate linear regression modeling. After adjusting for the effects of PGY of training, residency-related hours of education, extracurricular hours of education, and comfort with managing mechanically ventilated patients, we determined that self-reported exposure to management of patients on mechanical ventilation was not a significant predictor of total test score (t = 0.569, p = 0.570). Exploratory regression analyses revealed that the strongest and only significant predictor of total test score was residents’ self-reported confidence in caring for mechanically ventilated patients (F = 10.963, p = 0.001). On average, test scores increased by approximately 10 points (95% confidence interval 4.0–15.6 points, p = 0.001) when residents reported feeling comfortable managing ventilated patients ‘‘often’’ or ‘‘frequently.’’ The addition of any other predictor variables, alone or in combination, did not produce a more parsimonious model. Three variables were statistically significantly associated with resident comfort managing mechanically ventilated patients. The respondents’ year in residency was
EM Residents’ Knowledge of Mechanical Ventilation
5
Figure 1. The mean percent correct on the knowledge assessment tool stratified by postgraduate year (PGY) level, with error bars indicating standard deviations. *p < 0.05.
most strongly associated with conveying confidence in caring for ventilated patients, with an aOR of 10.049 (p < 0.001) for PGY-4s as compared to PGY-1s. Next, hours of residency education were associated with comfort, as residents reporting 4–5 h of curriculum-based education were more likely to report confidence than those reporting 1 or fewer hours (aOR 8.9, p = 0.018). Finally, caring for mechanically ventilated patients ‘‘frequently’’ or ‘‘often,’’ compared to ‘‘rarely’’ or ‘‘never’’ was associated with comfort (aOR 3.426, p = 0.003) (Table 3). DISCUSSION Emergency physicians frequently care for critically ill, mechanically ventilated patients in the ED and may be called upon to care for these patients elsewhere. With overcrowding and aging of the population, the incidence of mechanically ventilated patients in the ED is growing,
Table 2. Correlations between Training Program Characteristics and Total Score on Assessment Tool Characteristics of Training Program and Experience Residency design Residency training site Postgraduate year of training Hours of EM curricular education Hours of extracurricular education Frequency of managing ventilated patients Level of comfort with managing mechanically ventilated patients EM management of mechanical ventilator EM = emergency medicine.
Correlation with Total Test Score (r)
p Value
0.028 0.005 0.177 0.153 0.153 0.135
0.684 0.946 0.009 0.024 0.024 0.046
0.223
3 h in such settings over the last year. Although the residents reported few hours of education, 64% of residents responded that they often or frequently care for intubated patients in the ED. Although each PGY of training was associated with a modest improvement in scores on the knowledge assessment tool, the impact of the years in residency on the score on the assessment tool was less than may be expected, accounting for only 3% in the variation in scores. The prior study of internal medicine residents demonstrated a stronger correlation between PGY and test score,
6
S. R. Wilcox et al.
Table 3. Relationships between Candidate Predictor Variables and Resident’s Level of Comfort with Managing Mechanically Ventilated Patients
Variable Postgraduate year of training Hours of EM curricular education Hours of extracurricular education Frequency of managing ventilated patients EM management of mechanical ventilator
Correlation with Resident Comfort (r)
p Value
0.409 0.222 0.149 0.441
70%, the average score was only 73%. The factors associated with an improved score were the residents’ hours of education, postgraduate year, and reported comfort in caring for ventilated patients in the univariate model, and only comfort correlated with score in a multivariate model. 4. How is patient care impacted? With emergency department overcrowding, projections show that emergency physicians will continue to frequently care for mechanically ventilated patients. Given the evidence demonstrating the importance of good ventilator management, mechanical ventilation is an increasingly important topic in EM. This study demonstrates that there are opportunities to increase hours of education regarding mechanical ventilation in EM training, and that doing so can increase residents’ confidence and knowledge of mechanical ventilation.
11
http://dx.doi.org/10.1016/j.jemermed.2014.09.059
Education EMERGENCY MEDICINE RESIDENTS’ KNOWLEDGE OF MECHANICAL VENTILATION Susan R. Wilcox, MD,*† Todd A. Seigel, MD,‡§ Tania D. Strout, PHD, RN,k Jeffrey I. Schneider, MD,{ Patricia M. Mitchell, RN,{ Evie G. Marcolini, MD,** Michael N. Cocchi, MD,††‡‡ Howard A. Smithline, MD,§§ Lucienne Lutfy-Clayton, MD,§§ Marie Mullen, MD,kk Jonathan S. Ilgen, MD,{{ and Jeremy B. Richards, MD*** *Department of Emergency Medicine, †Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, ‡Department of Emergency Medicine, Rhode Island Hospital, Providence, Rhode Island, §University of California San Francisco, San Francisco, California, kDepartment of Emergency Medicine, Maine Medical Center, Portland, Maine, {Department of Emergency Medicine, Boston Medical Center, Boston, Massachusetts, **Department of Emergency Medicine, Yale–New Haven Hospital, New Haven, Connecticut, ††Department of Emergency Medicine, ‡‡Department of Anesthesia Critical Care, Division of Critical Care, Beth Israel Deaconess Medical Center, Boston, Massachusetts, §§Department of Emergency Medicine, Baystate Medical Center, Springfield, Massachusetts, kkDepartment of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, {{Division of Emergency Medicine, University of Washington School of Medicine, Seattle, Washington, and ***Division of Pulmonary, Critical Care and Sleep Medicine, Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts Reprint Address: Susan R. Wilcox, MD, Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA 02114
, Abstract—Background: Although Emergency physicians frequently intubate patients, management of mechanical ventilation has not been emphasized in emergency medicine (EM) residency curricula. Objectives: The objective of this study was to quantify EM residents’ education, experience, and knowledge regarding mechanical ventilation. Methods: We developed a survey of residents’ educational experiences with ventilators and an assessment tool with nine clinical questions. Correlation and regression analyses were performed to evaluate the relationship between residents’ scores on the assessment instrument and their training, education, and comfort with ventilation. Results: Of 312 EM residents, 218 responded (69.9%). The overall correct response rate for the assessment tool was 73.3%, standard deviation (SD) ± 22.3. Seventy-seven percent (n = 167) of respondents reported # 3 h of mechanical ventilation education in their residency curricula over the past year. Residents reported frequently caring for ventilated patients in the ED, as 64% (n = 139) recalled caring for $ 4 ventilated patients per
month. Fifty-three percent (n = 116) of residents endorsed feeling comfortable caring for mechanically ventilated ED patients. In multiregression analysis, the only significant predictor of total test score was residents’ comfort with caring for mechanically ventilated patients (F = 10.963, p = 0.001). Conclusions: EM residents report caring for mechanically ventilated patients frequently, but receive little education on mechanical ventilation. Furthermore, as residents’ performance on the assessment tool is only correlated with their self-reported comfort with caring for ventilated patients, these results demonstrate an opportunity for increased educational focus on mechanical ventilation management in EM residency training. Ó 2014 Elsevier Inc. , Keywords—mechanical ventilation; critical care; education; residents
INTRODUCTION Although emergency physicians frequently intubate critically ill patients in the emergency department (ED), management of mechanical ventilation traditionally has
Institutional Review Board approval was obtained at each participating institution.
RECEIVED: 14 April 2014; FINAL SUBMISSION RECEIVED: 15 August 2014; ACCEPTED: 30 September 2014 1
2
S. R. Wilcox et al.
not been emphasized in emergency medicine (EM) practice and residency training curricula (1–4). Nonetheless, management of positive-pressure ventilation can influence outcomes of critically ill patients for several conditions commonly encountered in EM practice (5– 10). For example, patients with asthma, once intubated, are at high risk of complications and deterioration (7). Low-tidal-volume ventilation improves mortality in patients with acute respiratory distress syndrome (ARDS) (11). Careful management of oxygenation and ventilation by emergency care providers has been shown to improve outcomes in intubated patients with traumatic brain injury (12,13). Furthermore, due to hospital crowding, emergency physicians may be primarily responsible for prolonged management of mechanically ventilated patients (14–16). Even for patients who are in the ED only briefly, evidence suggests that ventilator-induced lung injury can occur in as little as 20 min (17). We designed this study to quantify EM residents’ experience and knowledge regarding mechanical ventilation. We surveyed EM residents to assess how frequently they receive education on mechanical ventilation, frequency with which they care for mechanically ventilated patients in the ED, and their subjective comfort with managing patients on mechanical ventilation. In addition, we created an assessment tool to characterize residents’ application of knowledge regarding mechanical ventilation involving common emergency scenarios. We hypothesized that the residents with the most experience in managing mechanical ventilators in the ED would perform superiorly on the knowledge assessment tool. MATERIALS AND METHODS Survey Instrument Development To quantify EM residents’ training experiences, we developed a 5-point Likert scale survey tool to assess residents’ level of training, hours of education on mechanical ventilation, and exposure to the topic at local and national conferences (Appendix). The survey also queried residents regarding the frequency with which they care for mechanically ventilated patients and their comfort with managing ventilators. Survey responses were dichotomized as affirmative or negative: the responses ‘‘often’’ and ‘‘frequently’’ were defined as affirmative responses, whereas ‘‘never,’’ ‘‘rarely,’’ or ‘‘don’t know’’ were defined as negative. Any responses left blank were scored as ‘‘don’t know.’’ Assessment Instrument Development A literature review did not identify preexisting assessment tools for assessing EM residents’ knowledge regarding
clinical issues involving mechanical ventilation. We identified one validated test with a focus on management of mechanical ventilation in the intensive care unit (ICU) designed for Internal Medicine residents, and this test served as a foundation for development of our assessment tool. A project team with backgrounds in EM and critical care, and experience in educational survey development, generated an assessment instrument with questions specific to EM (18–20). We created a series of questions involving key principles consistent with outlined objectives for resident education in mechanical ventilation, and similar in style and content to the validated test for internal medicine residents (18,21). These principles included respiratory physiology, modes of mechanical ventilation, and complications of mechanical ventilation (18,21). The content was modified to be relevant to management of mechanically ventilated patients in the ED. Specific clinical scenarios emphasized emergency management of ventilated patients with asthma, ARDS, and traumatic brain injury, as evidence supports the importance of conscientious ventilator management in these clinical scenarios (5,7,10–12,22–28). Our assessment tool was formatted using multiplechoice questions, an accepted means of assessing clinical competence, following guidelines recommended by the National Board of Medical Examiners (29–32). To enhance validity, candidate questions were reviewed and edited by subject experts in an iterative fashion to optimize content, length, and relevance to the assessment tool’s goals. Eleven faculty members from multiple institutions, with backgrounds ranging from community EM, academic EM, pulmonology/critical care, trauma surgery, anesthesiology, and critical care medicine, critically reviewed the survey and assessment tool. The faculty provided further comment, review, and editing of the questions. To assess validity regarding the response process, the survey and assessment tool were then piloted with faculty and senior EM residents to assess question clarity, to determine survey length, and identify potentially redundant questions (33,34). After piloting, the project was reviewed for final approval by EM faculty with critical care fellowship training who were not originally involved in the first two iterations of the survey development. Study Protocol The finalized versions of the survey and assessment tool were administered anonymously using Research Electronic Data Capture (REDCap, Nashville, TN) tools hosted at Massachusetts General Hospital (35). REDCap is a secure, Web-based application designed to support
EM Residents’ Knowledge of Mechanical Ventilation
data capture for research studies, providing an interface for validated data entry. The survey and assessment tool were distributed by email to all of the EM residents enrolled in eight EM residency training programs in the northeastern United States, for a total of 312 EM residents. These eight institutions include both 3-year and 4-year training programs. The survey was sent to all EM residents in each training program by a local investigator at each site. Residents at these institutions received an e-mail invitation to participate once a week for 3 weeks, beginning in early January 2013. The study protocol was approved by the institutional review boards of all participating institutions. Consent for participation in the survey was obtained from each resident at the time of participation, as the survey introduction stated that partaking of the survey indicated consent. Data Analysis Study data were exported into a Microsoft Excel (Microsoft Corporation, Redmond, WA) spreadsheet program and were then transferred into SPSS (v. 11.0, SPSS, Inc., Chicago, IL) for analysis. For all variables, missing data were excluded on a case-by-case basis. For the purposes of this study, we assumed the correct response rate for the assessment tool (test score) to be a surrogate for knowledge of mechanical ventilation. We examined the continuous outcome variable test score for normality in two ways. First, the outcome was examined visually using histograms and normal quantilequantile plots. After visual examination, Pearson’s second skewness coefficient was computed, revealing mild skew to the left, Sk2 = 0.61. Survey data regarding study participants and their training programs, mechanical ventilation educational experiences, and ventilator management experience were summarized using descriptive statistics. One-way analysis of variance was used to assess for differences in total test score across participating institutions. As our hypothesis was that the residents with the most exposure to managing mechanical ventilators in the ED would perform better on the knowledge assessment tool, we examined the relationship between these variables in several ways. Ordinary least-squares regression analyses were performed, with total test score serving as the outcome variable. The frequency with which residents managed mechanically ventilated patients was the predictor variable. To examine the relationship between these variables after controlling for other variables significantly correlated to test score in simple correlation analysis (Spearman’s r), hierarchical multiple regression models using the additional predictors of postgraduate year (PGY) of training, number of EM residency mechan-
3
ical ventilation lecture hours completed, number of hours of education outside EM residency curricula, and subjective comfort with managing mechanically ventilated ED patients were employed. Exploratory regression analyses were then conducted to determine which variables, alone and in combination, were the strongest predictors of total test score. In addition to assessing normality as described above, additional linear regression assumptions were evaluated using residual analyses and assessment of influence diagnostics. Multicollinearity was evaluated using variance inflation factors, which were all well below the recommended cut points. Multivariate logistic regression analyses were performed to evaluate the extent to which PGY of training, hours of residency curriculum-based formal ventilator management education, hours of extracurricular ventilator-related training, and the frequency with which participants reporting caring for ventilated patients in the ED influenced residents’ self-reported comfort with managing mechanically ventilated patients. Coefficient estimates, adjusted odds ratios (aORs), and 95% confidence intervals are reported for each variable. We accepted an alpha of < 0.05 as statistically significant. RESULTS Characteristics of the Study Subjects and Their Training Programs Study surveys were distributed to a total of 312 EM residents, with 219 residents responding (response rate = 70.2%). One resident completed only the first survey question and was subsequently dropped from the study, leaving data from 218 residents’ complete surveys for analysis (69.9%). Nine residents (4%) did not fully complete the knowledge assessment tool. Of the residents who completed the surveys, there was relatively equal distribution by PGY class (26.6% PGY-1, 28.0% PGY-2, 27.5% PGY-3, and 17.9% PGY-4.) The eight institutions included in this study represented both 3-year (37.7%) and 4-year EM programs (63.3%), and the response rate from the institutions ranged from 43.6% to 81.4%. Educational Experiences and Experience Managing Ventilated Patients Overall, study participants reported few residency-based educational opportunities regarding mechanical ventilation. Seventy-seven percent of residents (n = 167) reported receiving 3 or fewer hours of ventilation-related education in their residency curricula over the past year, and 34% (n = 73) reported receiving between 0 and 1 h of education. Similarly, 73% (n = 159) of residents described receiving 3 or fewer hours of mechanical
4
S. R. Wilcox et al.
ventilation education from nonresidency sources, including journal clubs, local or national conference attendance, and reading articles (Table 1). Responses regarding educational experiences were consistent among residents from the individual institutions, particularly when grouped by PGY of training. Conversely, residents reported frequently caring for mechanically ventilated patients in the ED. Sixty-four percent (n = 139) of residents reported that they care for 4 or more ventilated patients per month in the ED, and 22% (n = 48) reported caring for 10 or more ventilated patients per month. Fifty-three percent (n = 116)
Table 1. Residents’ Self-reported Education and Experience Regarding Mechanical Ventilation Respondents (%) How many hours of lecture on mechanical ventilation have you received through your EM residency in the last year? 0–1 2–3 4–5 More than 5 Don’t know How many hours of instruction have you received on mechanical ventilation from other EM sources (EM articles, discussion in EM journal clubs, EM lectures/ conferences, etc.) in the last year? 0–1 2–3 4–5 More than 5 Don’t know How often do you care for mechanically ventilated patients in the Emergency Department? Never Rarely (1–3 patients/month) Often (4–9 patients/month) Frequently (>10 patients/month) Don’t know How often do you feel comfortable managing mechanical ventilation and troubleshooting issues with ventilated patients in the ED? Never Rarely Often Frequently Don’t know Who primarily manages and makes changes to the mechanical ventilator for intubated patients in your ED? Respiratory therapist Nurse EM resident EM attending Other doctor (ICU physician, pulmonologist, etc.) Don’t know
73 (33.5) 94 (43.1) 18 (8.3) 12 (5.5) 21 (9.2)
68 (31.2) 91 (41.8) 24 (11.0) 18 (8.3) 17 (4.6)
18 (8.3) 58 (26.6) 91 (41.7) 48 (22.0) 3 (1.4)
15 (6.9) 82 (37.6) 94 (43.1) 22 (10.1) 5 (2.3)
170 (78.0) 0 (0) 27 (12.4) 8 (3.7) 0 (0) 13 (6.0)
EM = emergency medicine; ED = emergency department; ICU = intensive care unit.
of participants described feeling comfortable caring for mechanically ventilated ED patients ‘‘often’’ or ‘‘frequently’’; whereas 45% of respondents described ‘‘never’’ or ‘‘ rarely’’ feeling comfortable managing these patients. Only 16% (n = 35) described management of the ventilator as the responsibility of an emergency physician (resident or attending) at their institution. Seventy-eight percent (n = 170) identified the respiratory therapist as being primarily in charge of ventilator management (Table 1). Ventilator Management Knowledge The overall correct response rate for the nine-question assessment tool was 73.3%, SD 6 22.3%. Of the 218 residents who completed the assessment tool, 134 (61.5%) achieved a score of at least 70%. Statistically significant differences in total test scores were not noted between institutions (F = 1.035, p = 0.408). Increasing PGY level was associated with increased scores on the knowledge tool (Figure 1). Correlation analysis revealed statistically significant relationships between total test score and the frequency of managing ventilated ED patients, PGY of training, self-reported residency-related hours of ventilator management education, extracurricular ventilator education, and level of comfort with managing ventilated patients. The relationships between total test score and residency design or who bears primary responsibility for ventilator management were not significant (Table 2). Multivariate Results We evaluated the relationship between total test score and exposure to mechanical ventilation management with multivariate linear regression modeling. After adjusting for the effects of PGY of training, residency-related hours of education, extracurricular hours of education, and comfort with managing mechanically ventilated patients, we determined that self-reported exposure to management of patients on mechanical ventilation was not a significant predictor of total test score (t = 0.569, p = 0.570). Exploratory regression analyses revealed that the strongest and only significant predictor of total test score was residents’ self-reported confidence in caring for mechanically ventilated patients (F = 10.963, p = 0.001). On average, test scores increased by approximately 10 points (95% confidence interval 4.0–15.6 points, p = 0.001) when residents reported feeling comfortable managing ventilated patients ‘‘often’’ or ‘‘frequently.’’ The addition of any other predictor variables, alone or in combination, did not produce a more parsimonious model. Three variables were statistically significantly associated with resident comfort managing mechanically ventilated patients. The respondents’ year in residency was
EM Residents’ Knowledge of Mechanical Ventilation
5
Figure 1. The mean percent correct on the knowledge assessment tool stratified by postgraduate year (PGY) level, with error bars indicating standard deviations. *p < 0.05.
most strongly associated with conveying confidence in caring for ventilated patients, with an aOR of 10.049 (p < 0.001) for PGY-4s as compared to PGY-1s. Next, hours of residency education were associated with comfort, as residents reporting 4–5 h of curriculum-based education were more likely to report confidence than those reporting 1 or fewer hours (aOR 8.9, p = 0.018). Finally, caring for mechanically ventilated patients ‘‘frequently’’ or ‘‘often,’’ compared to ‘‘rarely’’ or ‘‘never’’ was associated with comfort (aOR 3.426, p = 0.003) (Table 3). DISCUSSION Emergency physicians frequently care for critically ill, mechanically ventilated patients in the ED and may be called upon to care for these patients elsewhere. With overcrowding and aging of the population, the incidence of mechanically ventilated patients in the ED is growing,
Table 2. Correlations between Training Program Characteristics and Total Score on Assessment Tool Characteristics of Training Program and Experience Residency design Residency training site Postgraduate year of training Hours of EM curricular education Hours of extracurricular education Frequency of managing ventilated patients Level of comfort with managing mechanically ventilated patients EM management of mechanical ventilator EM = emergency medicine.
Correlation with Total Test Score (r)
p Value
0.028 0.005 0.177 0.153 0.153 0.135
0.684 0.946 0.009 0.024 0.024 0.046
0.223
3 h in such settings over the last year. Although the residents reported few hours of education, 64% of residents responded that they often or frequently care for intubated patients in the ED. Although each PGY of training was associated with a modest improvement in scores on the knowledge assessment tool, the impact of the years in residency on the score on the assessment tool was less than may be expected, accounting for only 3% in the variation in scores. The prior study of internal medicine residents demonstrated a stronger correlation between PGY and test score,
6
S. R. Wilcox et al.
Table 3. Relationships between Candidate Predictor Variables and Resident’s Level of Comfort with Managing Mechanically Ventilated Patients
Variable Postgraduate year of training Hours of EM curricular education Hours of extracurricular education Frequency of managing ventilated patients EM management of mechanical ventilator
Correlation with Resident Comfort (r)
p Value
0.409 0.222 0.149 0.441
70%, the average score was only 73%. The factors associated with an improved score were the residents’ hours of education, postgraduate year, and reported comfort in caring for ventilated patients in the univariate model, and only comfort correlated with score in a multivariate model. 4. How is patient care impacted? With emergency department overcrowding, projections show that emergency physicians will continue to frequently care for mechanically ventilated patients. Given the evidence demonstrating the importance of good ventilator management, mechanical ventilation is an increasingly important topic in EM. This study demonstrates that there are opportunities to increase hours of education regarding mechanical ventilation in EM training, and that doing so can increase residents’ confidence and knowledge of mechanical ventilation.
11
