Simulation Training for Pediatric Residents on Central Venous Catheter Placement: A Pilot Study* Scott M. Thomas, MD1; Wesley Burch, AAH2; Sarah E. Kuehnle, BA3; Robert G. Flood, MD4; Anthony J. Scalzo, MD2,4; James M. Gerard, MD4

Objective: To assess the effect of simulation training on pediatric residents’ acquisition and retention of central venous catheter insertion skills. A secondary objective was to assess the effect of simulation training on self-confidence to perform the procedure. Design: Prospective observational pilot study. Setting: Single university clinical simulation center. Subjects: Pediatric residents, postgraduate years 1–3. Interventions: Residents participated in a 60- to 90-minute ultrasound-guided central venous catheter simulation training session. Video recordings of residents performing simulated femoral central venous catheter insertions were made before (baseline), after, and at 3-month following training. Three blinded expert raters independently scored the performances using a 24-item checklist and 100-mm global rating scale. At each time point, residents rated their confidence to perform the procedure on a 100-mm scale. Measurements and Main Results: Twenty-six residents completed the study. Compared with baseline, immediately following training, median checklist score (54.2% [interquartile range, 40.8– 68.8%] vs 83.3% [interquartile range, 70.0–91.7%]), global rating score (8.0 mm [interquartile range, 0.0–64.3 mm] vs 79.5 mm [interquartile range, 16.3–91.7 mm]), success rate (38.5% vs 80.8%), and self-confidence (8.0 mm [interquartile range, 3.8–19.0 mm] vs 52.0 mm [interquartile range, 43.5–66.5 mm]) *See also p. 908. 1 Department of Pediatrics, Division of Pediatric Emergency Medicine, Washington University School of Medicine in St. Louis, St. Louis, MO. 2 Saint Louis University Clinical Simulation Center, St. Louis, MO. 3 Saint Louis University School of Medicine, St. Louis, MO. 4 Department of Pediatrics, Division of Pediatric Emergency Medicine, Saint Louis University School of Medicine, St. Louis, MO. This study was conducted at the Saint Louis University Clinical Simulation Center, St. Louis, MO. Supported, in part, by an intramural Fleur-de-lis Grant. This study used a central venous catheter task trainer that was purchased for the Saint Louis University Clinical Simulation Center by the not-for-profit Glennon Guild. The authors have disclosed that they do not have any potential conflicts of interest. For information regarding this article, E-mail: [email protected] Copyright © 2013 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies DOI: 10.1097/PCC.0b013e31829f5eda

e416

www.pccmjournal.org

all improved (p < 0.05 for all variables). Compared with baseline, median checklist score (54.2% [interquartile range, 40.8–68.8%] vs 54.2% [interquartile range, 45.8–80.4%], p = 0.47), global rating score (8.0 mm [interquartile range, 0.0–64.3 mm] vs 35.5 mm [interquartile range, 5.3–77.0], p = 0.62), and success rate (38.5% vs 65.4%, p = 0.35) were similar at 3-month follow-up. Self-confidence, however, remained above baseline at 3-month follow-up (8.0 mm [interquartile range, 3.8–19.0 mm] vs 61.0 mm [interquartile range, 31.5–71.8 mm], p < 0.01). Conclusions: Simulation training improved pediatric residents’ central venous catheter insertion procedural skills. Decay in skills was found at 3-month follow-up. This suggests that simulation training for this procedure should occur in close temporal proximity to times when these skills would most likely be used clinically and that frequent refresher training might be beneficial to prevent skills decay. (Pediatr Crit Care Med 2013; 14:e416–e423) Key Words: central venous catheter placement; patient simulation; procedural skills training; resident education; selfconfidence; ultrasound

I

n recent years, the use of simulation to augment clinical training in medicine has become widespread. Simulation training has been shown to improve residents’ acquisition and retention of advanced life support and other critical care skills (1–7). Several adult studies have demonstrated benefits to actual patients related to simulation training on ultrasound (US)-guided central venous catheter (CVC) insertion (8–10). These studies have found that following simulation training, internal medicine residents performed real CVC insertions with fewer needle passes, arterial punctures, catheter adjustments, and higher success rates (8) and that it reduced catheter-related blood stream infections (9, 10). The Accreditation Council for Graduate Medical Education (ACGME) does not mandate that pediatric residents perform or achieve competency in the CVC procedure (11). For pediatric residents entering into fields where this procedure will be performed, however, residency can provide an initial opportunity to learn and practice this skill. The use of simulation to augment pediatric clinical skills training has been November 2013 • Volume 14 • Number 9

Online Clinical Investigations

well described (12) and is congruous with the ACGME’s Next Accreditation System which will emphasize the responsibility of sponsoring institutions to ensure the quality and safety of the environment for learning and patient care (13). Similar to training for other procedural skills, simulation can be used to offer a safe environment for pediatric residents to initially learn and hone skills needed for the CVC procedure. In light of the encouraging results found in the adult literature with regard to CVC simulation training (8–10), its role in pediatrics should be further explored. Given that the CVC procedure is not an ACGME-mandated skill and that pediatric residents are likely to have less clinical exposure to this procedure, the educational efficacy and residents’ perceptions regarding CVC simulation training found in adult studies may differ in pediatrics. This hypothesis-generating pilot study was conducted to collect initial data that can be used to further develop and validate optimal strategies for incorporating CVC simulation training into existing clinical-based training within the field of pediatrics. The primary purpose of this study was to assess the effect of a brief simulation training session on the initial acquisition and retention of CVC insertion skills in a group of novice pediatric residents. Secondary objectives were to assess the effect of simulation training on residents’ selfconfidence to perform the procedure and their beliefs regarding the educational benefit of this training.

MATERIALS AND METHODS Study Design and Setting This prospective observational pilot study was approved by the Institutional Review Board at Saint Louis University. Subjects provided informed written consent prior to participation. Between September 2011 and January 2012, residents participated in a single, 60- to 90-minute US-guided CVC training session conducted at the Saint Louis University Clinical Simulation Center. Training was conducted in a small-group format with one to three residents per session. At each session, residents were shown a 10-minute instructional video on USguided CVC insertion from the Videos in Clinical Medicine series from the New England Journal of Medicine (14). One of the investigators (S.M.T.) then led a hands-on training session using a SonoSite 180 Plus portable US with L38/10–5 megahertz linear transducer (SonoSite, Bothell, WA) and a VascularAccessChild task trainer (Simulab, Seattle, WA). Residents were allowed time for individual supervised practice on the US-guided CVC insertion procedure. During the practice sessions, the investigator directly observed and provided immediate feedback to each resident while they practiced the CVC insertions. Residents performed multiple insertions until they successfully placed a CVC and expressed feeling comfortable performing the procedure. We collected procedural skills and self-confidence data immediately prior to and after each training session. To assess retention of skills and self-confidence over time, residents returned to the simulation center 3 months after their training session. No training was conducted at the follow-up sessions. Pediatric Critical Care Medicine

Subjects All categorical pediatric residents in our training program were eligible for inclusion in the study. Residents in other programs, such as internal medicine pediatrics, were excluded from participation. The pediatric residency program at our institution typically comprised 14 categorical residents in each postgraduate year (PGY). Residents perform 2 to 3 months in the PICU, 4 months in the neonatal ICU, and 5 months in the emergency department over the course of their training. Residents do not receive formal US training and are not required to perform CVC insertions. Residents interested in learning CVC insertion are allowed to perform the procedure under direct supervision by an attending in the PICU and emergency department. Approximately 10–15% of CVC insertions at our hospital are performed by residents. The use of US guidance when performing CVC insertions is left to the discretion of the supervising attending. Previous Experience and Self-Confidence Data Before the start of each training session and at 3-month followup, residents completed written questionnaires to provide data on previous US and CVC insertion experience. In addition, they rated their self-confidence to perform the US-guided CVC procedure on a 100-mm visual analog scale with two anchors ranging from 0 (not at all confident) to 100 (very confident). For this study, we defined self-confidence as the “belief in your ability to competently perform US-guided central venous insertion on a real patient with a reasonably high chance of success.” We readministered the self-confidence questionnaire to subjects immediately after completion of the training sessions and after the simulated CVC procedures at the end of the 3-month follow-up sessions. Skills Performance Data Data were collected on six variables related to skills performance including checklist score, global rating score, number of needle passes, needle cannulation success rate, CVC insertion success rate, and time to completion. Successful needle cannulation was defined as introduction of the needle into the task trainer vein with free flow of fluid. Successful CVC insertion was defined as advancing the CVC into the task trainer vein. To assess skills performance, video recordings of subjects individually performing unsupervised simulated US-guided femoral CVC insertions were made before (baseline) and after each training session and at 3-month follow-up. Subjects were aware that their video-recorded insertion performances would be assessed at a later time. The video recording sessions took place at the Saint Louis University Clinical Simulation Center. The room setup was identical for each session and included the US machine, CVC task trainer, central catheter kits, and sterile supplies. Subjects were recorded from two camera angles using remotely controlled pan/tilt/zoom video cameras. One of the investigators (S.M.T.) was present in the room serving as a nonsterile assistant. No feedback, guidance, or prompting was provided to subjects during the video-recorded procedures. Subjects were stopped if they had not successfully achieved www.pccmjournal.org

e417

Thomas et al

needle cannulation of the vein within 20 minutes of the procedure start time. After completion of all video recording sessions, three blinded expert raters independently scored the insertion performances using a 24-item trichotomous checklist and a single-item global rating scale (GRS) instrument (Fig. 1). The checklist, based on one developed and reported by Barsuk et al (15) for internal jugular and subclavian CVC placement, was modified to account for the femoral vein location assessed in our study. For subjects who failed to achieve needle cannulation of the vein within 20 minutes and were therefore stopped, raters were instructed to score subsequent checklist items as “not done.” The GRS instrument is a 100-mm visual analog scale with two anchors ranging from 0 (highly unlikely) to 100 (highly likely). For the GRS, raters scored subjects based on the following question, “Would the subject likely be successful at performing US-guided CVC insertion on a real patient without significant complications or issues with sterile technique?” In addition, raters recorded the number of needle passes and time to completion for each of the subjects. Time to completion included preparation time and ended when a sterile dressing was placed over the inserted CVC or the subject verbally indicated that they were done with the procedure. Two of the raters, board certified in pediatric critical care, were recruited from outside institutions. Neither was involved in any aspect of designing or conducting the study, and both were blinded to the subjects’ PGY class, training status (e.g., before, after, or 3-month follow-up), and the hypothesis being tested. The third rater is board certified in pediatric emergency medicine and was one of the study investigators (J.M.G.) but did not participate in any of the training sessions and had no knowledge of the training status of subjects on the videotapes at the time of scoring. Each video-recorded performance was scored by a single rater. To ensure that each rater scored an equal mixture of subjects from each time point, a random numbers generator was used to randomly number each videotape. The videotapes were then numerically divided into three groups and distributed among the raters. In a separate analysis, 10 randomly selected videotapes were scored by all three of the raters for assessment of checklist and GRS interrater agreement. Posttraining Survey At the end of each training session, to elicit feedback on the training, subjects completed a structured survey that addressed various aspects of the training, including realism, educational benefit, and enjoyment with the training. Subjects rated their responses on a 100-mm visual analog scale with two anchors ranging from 0 (not at all) to 100 (very much). Statistical Analyses Performance and self-confidence data were nonnormally distributed. Nonparametric tests for repeated measures were therefore used to analyze the data. The Friedman test with post hoc Bonferroni-corrected Wilcoxon signed-rank tests was e418

www.pccmjournal.org

used to analyze checklist and GRS scores, number of needle passes, time to completion, and self-confidence scores. These data are presented as medians, interquartile ranges (IQRs), and ranges. For ease of analysis and reporting, checklist items were converted to dichotomous variables. Items that were “done incorrectly” or “not done” were collapsed into a single value of “incorrect.” Time to completion was assessed for the subset of subjects who successfully completed the procedure both before and after training. Cochran Q test with post hoc Bonferronicorrected McNemar tests was used to analyze needle cannulation and CVC insertion success rates. These data are presented as counts with corresponding percentages. Cronbach’s α was calculated to assess internal consistency of the checklist. Intraclass correlation coefficients (ICCs) were calculated to assess interrater agreement for the checklist and GRS scoring instruments. A two-way random effects model (ICC) (1, 2) for absolute agreement was used. Post hoc tests were two-sided. An α of less than 0.05 was considered statistically significant. Statistical analyses were performed using SPSS version 17.0 software (SPSS, Chicago, IL).

RESULTS During the study period, 27 subjects (nine PGY-1, eight PGY-2, and 10 PGY-3) were enrolled in the study. One resident (PGY1) did not return for 3-month follow-up and was excluded from data analysis. Among the 26 residents who completed the study, prior to training, 13 residents (50.0%) indicated that they had previously performed at least one clinical bedside US. Four residents (15.4%) had previous experience performing CVC insertions, and two residents (7.7%) had previously used US for CVC placement. During the period between training sessions and 3-month follow-up, two residents (7.7%) indicated that they had each placed one central catheter; one of these was performed using US. Performance data are shown in Table 1. Despite multiple attempts to cannulate the vein, seven subjects prior to training, one subject post training, and two subjects at 3-month follow-up failed to achieve needle cannulation within 20 minutes. Compared with baseline, immediately following training, checklist score (54.2% [IQR, 40.8–68.8%] vs 83.3% [IQR, 70.0–91.7%], p < 0.01), global rating score (8.0 mm [IQR, 0.0–64.3 mm] vs 79.5 mm [IQR, 16.3–91.7 mm], p = 0.02), and successful CVC insertion rate (38.5% vs 80.8%, p = 0.02) all improved. Compared with baseline, immediately following training, no differences were seen for the number of needle passes (1.5 vs 1.0, p = 0.10) or for those who achieved successful needle cannulation of the vein (73.1% vs 96.2%, p = 0.21). When compared with baseline, no differences were seen for any of the performance measures at 3-month follow-up. Performance scores for individual checklist items are shown in Table 2. Eight subjects successfully completed the CVC procedure before and immediately after simulation training. For these subjects, following training, the median time to completion of the November 2013 • Volume 14 • Number 9

Online Clinical Investigations

Skill Key: A = Done correctly, B = Done incorrectly, C = Not Done 1. Performs time out

ABC

2. Flushes the ports on the catheter with sterile saline

ABC

3. Clamps each port (OK to keep distal port open)

ABC

4. Removes cap to the distal port from end of catheter to accommodate wire

ABC

5. Degerms hands using alcohol based cleaner

ABC

6. Cleanses area with chlorhexidine

ABC

7. In a sterile fashion, subject gets into cap, mask, gown and gloves

ABC

8. Drapes area in the usual sterile fashion

ABC

9. Requests the assistance of the non-sterile assistant to set up the ultrasound probe

ABC

10. Localizes the vein using anatomic landmarks with the ultrasound machine

ABC

11. Anesthetizes the skin and deeper structures using 1% lidocaine

ABC

12. Using the large needle (or catheter-syringe complex), cannulates the vein while aspirating

ABC

(must be done with US) 13. Removes the syringe from the needle or advances the catheter-syringe complex into the

ABC

vein removing both the syringe and needle 14. Advances the guide wire into the vein to an appropriate length

ABC

15. Knicks the skin with the scalpel to advance the dilator

ABC

16. Advances the dilator over the wire and dilates the vein

ABC

17. Advances the catheter over the guide wire

ABC

18. Never releases the guide wire

ABC

19. Once the catheter is inserted, removes the guide wire in its entirety

ABC

20. Advances the catheter to the hub

ABC

21. Ensures there is blood flow/flushes each port

ABC

22. Secures the catheter in place (suture or staple)

ABC

23. Places dressing over the catheter

ABC

24. Maintains sterile technique

ABC

Number of Needle Passes: _________ Procedure Time: ________ Please rate the subject’s overall performance by drawing a single vertical line through the horizontal line below. Would the subject likely be successful at performing US-guided CVC insertion on a real patient without significant complications or issues with sterile technique? Highly Unlikely

Highly Likely

Figure 1. Ultrasound (US)-guided central venous catheter (CVC) procedural skills checklist and global rating scale instruments. Skill key: A = done correctly, B = done incorrectly, C = not done.

Pediatric Critical Care Medicine

www.pccmjournal.org

e419

Thomas et al

Table 1. Performance and Self-Confidence Assessments for Ultrasound-Guided Central Venous Catheter Insertion Before, After, and at 3 Months Following Simulation Training (n = 26)

Before

Aftera

3 Mo After

Change From Baseline at 3 Mo

13.0 (54.2)

20.0 (83.3)

13.0 (54.2)

0 (0.0)

0.47

9.8 (40.8)–16.5 (68.8)

16.8 (70.0)–22.0 (91.7)

11.0 (45.8)–19.3 (80.4)

4 (16.7)–22 (91.7)

9 (37.5)–24 (100)

6 (25.0)–22 (91.7)

8.0

79.5

35.5

27.5

0.62

0.0–64.3

16.3–92.0

5.3–77.0

0–93

0–100

0–100

1.5

1.0

2.5

1.0

1.0

1.0–4.0

1.0–2.0

2.0–4.0

1–11

1–10

1–6

Successful needle cannulation, n (%)

19 (73.1)

25 (96.2)

24 (92.3)

5 (19.2)

0.54

Successful central venous catheter insertion, n (%)

10 (38.5)

21 (80.8)

17 (65.4)

7 (26.9)

0.35

8.0

52.0

61.0

53.0

< 0.01

3.8–19.0

43.5–66.5

31.5–71.8

0–89

14–98

5–79

Variable

pb

Checklist score, n (% correct)  Median  IQR  Range Global rating scale score, mm  Median  IQR  Range Needle passes, n  Median  IQR  Range

Self-confidence, mm  Median  IQR  Range

IQR = interquartile range. a Checklist score, global rating scale score, successful central venous catheter insertion rate, and self-confidence improved immediately following training (p < 0.05 for all variables). b Corrected p values for the differences between baseline and 3-mo follow-up.

procedure decreased from 24.4 minutes (IQR, 18.5–29.7 min) to 16.2 minutes (IQR, 14.5–19.5 min) (p = 0.04). Four subjects successfully completed the CVC procedure at all three of the time points. For these subjects, compared with baseline, median time to completion at 3-month follow-up was shorter, although this difference did not reach statistical significance (22.6 min [IQR, 18.0–28.3 min] vs 16.5 min [IQR, 11.6–18.6 min], p = 0.20). Self-confidence data are shown in Table 1. Compared with baseline, immediately following training, self-confidence improved (8.0 mm [IQR, 3.8–19.0 mm] vs 52.0 mm [IQR, 43.5–66.5 mm], p < 0.01). Self-confidence remained above baseline at 3-month follow-up (8.0 mm [IQR, 3.8–19.0 mm] vs 61.0 mm [IQR, 31.5–71.8 mm], p < 0.01). Cronbach α coefficient for the checklist was 0.91 indicating good internal consistency (16). The ICC for the checklist instrument was 0.77 indicating substantial agreement (17). The ICC for the GRS instrument was 0.98 indicating near perfect agreement (17). e420

www.pccmjournal.org

Posttraining survey results are shown in Table 3. Residents rated the sessions as moderately realistic. They reported higher ratings for the perceived educational benefits and enjoyment with the simulation training sessions.

DISCUSSION Although the benefits of simulation training for the CVC insertion procedure have been shown in other fields (7–10), studies regarding its use in pediatrics are lacking. Given that the CVC procedure is not an ACGME-mandated skill for pediatric training and that many pediatric residents are likely to never perform the CVC procedure on a real patient, broadly applied CVC simulation training for all pediatric residents might be considered an inefficient use of educational time and is therefore hard to justify. In light of the encouraging results found in the adult literature, however, for the subset of pediatric residents who will perform clinical CVC insertions, the role of simulation training should be further explored. As a November 2013 • Volume 14 • Number 9

Online Clinical Investigations

Table 2. Checklist Item Scores for Ultrasound-Guided Central Venous Catheter Insertion Before, After, and at 3 Months Following Simulation Training (n = 26) Before Itema

After

A

B

C

A

C

A

B

C

1 (3.8)

1 (3.8)

24 (92.3)

16 (61.5)

2 (7.7)

8 (30.8)

6 (23.1)

1 (3.8)

19 (23.5)

Flushes ports

23 (88.5)

0 (0.0)

3 (11.5)

24 (92.3)

1 (3.8)

1 (3.8)

14 (53.8)

5 (19.2)

7 (26.9)

Clamps ports

16 (61.5)

0 (0.0)

10 (38.5)

21 (80.8)

1 (3.8)

4 (15.4)

12 (46.2)

4 (15.4)

10 (38.5)

Removes cap

16 (61.5)

4 (15.4)

6 (23.1)

24 (92.3)

1 (3.8)

1 (3.8)

15 (57.7)

9 (34.6)

2 (7.7)

Degerms hands

18 (69.2)

1 (3.8)

7 (26.9)

21 (80.8)

1 (3.8)

4 (15.4)

17 (65.4)

0 (0.0)

9 (11.1)

Cleanses skin

13 (50.0)

13 (50.0)

0 (0.0)

16 (61.5)

9 (34.6)

1 (3.8)

18 (69.2)

8 (30.8)

0 (0.0)

Dons sterile attire

20 (76.9)

4 (15.4)

2 (7.7)

22 (84.6)

4 (15.4)

0 (0.0)

16 (61.5)

8 (30.8)

2 (7.7)

Drapes area

15 (57.7)

10 (38.5)

1 (3.8)

22 (84.6)

4 (15.4)

0 (0.0)

22 (84.6)

4 (15.4)

0 (0.0)

Requests ultrasound probe

24 (92.3)

0 (0.0)

2 (7.7)

24 (92.3)

2 (7.7)

0 (0.0)

24 (92.3)

1 (3.8)

1 (3.8)

Localizes vein

22 (84.6)

3 (11.5)

1 (3.8)

20 (76.9)

5 (19.2)

1 (3.8)

22 (84.6)

4 (15.4)

0 (0.0)

Anesthetizes skin

23 (88.5)

0 (0.0)

3 (11.5)

20 (76.9)

0 (0.0)

6 (23.1)

12 (46.2)

1 (3.8)

13 (50.0)

Cannulates vein

15 (57.7)

9 (34.6)

2 (7.7)

21 (80.8)

5 (19.2)

0 (0.0)

21 (80.8)

5 (19.2)

0 (0.0)

Removes syringe

19 (73.1)

2 (7.7)

5 (19.2)

24 (92.3)

1 (3.8)

1 (3.8)

25 (96.2)

1 (3.8)

0 (0.0)

9 (34.6)

11 (42.3)

6 (23.1)

22 (84.6)

3 (11.5)

1 (3.8)

18 (69.2)

6 (23.1)

2 (7.7)

Knicks skin

18 (69.2)

2 (7.7)

6 (23.1)

19 (73.1)

6 (23.1)

1 (3.8)

16 (61.5)

4 (15.4)

6 (23.1)

Advances dilator

14 (53.8)

6 (23.1)

6 (23.1)

22 (84.6)

3 (11.5)

1 (3.8)

17 (65.4)

5 (19.2)

4 (15.4)

Advances catheter

12 (46.2)

5 (19.2)

9 (34.6)

19 (73.1)

5 (19.2)

2 (7.7)

18 (69.2)

6 (23.1)

2 (7.7)

Never releases wire

5 (19.2)

10 (38.5)

11 (42.3)

14 (53.8)

8 (30.8)

4 (15.4)

5 (19.2)

12 (46.2)

9 (34.6)

12 (46.2)

1 (3.8)

13 (50.0)

21 (80.8)

2 (7.7)

3 (11.5)

16 (61.5)

2 (7.7)

8 (30.8)

Advances catheter

8 (30.8)

2 (7.7)

16 (61.5)

19 (73.1)

4 (15.4)

3 (11.5)

14 (53.8)

3 (11.5)

9 (34.6)

Flushes ports

6 (23.1)

7 (26.9)

13 (50.0)

16 (61.5)

4 (15.4)

6 (23.1)

7 (26.9)

10 (38.5)

9 (34.6)

Secures catheter

10 (38.5)

0 (0.0)

16 (61.5)

21 (80.8)

0 (0.0)

5 (19.2)

13 (50.0)

0 (0.0)

13 (50.0)

Places dressing

6 (23.1)

1 (3.8)

19 (73.1)

20 (76.9)

2 (7.7)

4 (15.4)

12 (46.2)

1 (3.8)

13 (50.0)

8 (30.8)

20 (76.9)

2 (7.7)

4 (15.4)

12 (46.2)

6 (23.1)

8 (30.8)

Performs time out

Advances wire

Removes wire

Maintains sterility

10 (38.5)

8 (30.8)

B

3 Mo After

A = done correctly, B = done incorrectly, C = not done. Data are presented as number (%). a See Figure 1 for description of items.

first step, we conducted this pilot study to measure the effect of CVC simulation training on educational outcomes among novice pediatric residents in a controlled simulation laboratory setting (T1 level study) (18). The majority of residents in our study had no clinical experience performing the CVC procedure. Not surprisingly, we found that their skills performances were poor at baseline. We found that their skills improved significantly immediately following a brief training session. These skills, however, decayed considerably within 3 months following training. This is consistent with previous reports on the immediate effect of simulation training (4, 6, 19) and the decay of psychomotor skills over time (20–22). This performance profile suggests that CVC Pediatric Critical Care Medicine

simulation training sessions would be the most effective if they occur in close temporal proximity to times when these skills would likely be used clinically, for example, just prior to or at the beginning of a critical care rotation. It also suggests that frequent CVC refresher training might be beneficial to prevent skills decay. With regard to specific psychomotor skills, following simulation training, we found that the most striking improvement involved the residents’ handling and insertion of the guidewire, a critical step for successful CVC insertion. Prior to training, among the 19 residents who successfully achieved needle cannulation, nine residents (47.4%) failed to successfully insert the CVC. In reviewing these performances, we found that almost www.pccmjournal.org

e421

Thomas et al

uniformly these subjects inadvertently displaced the needle from the vein during the guidewire insertion step. As a result of the simulation training, residents clearly became more adept at this technical skill which largely contributed to the increased CVC insertion success rate immediately following the training sessions. In addition, residents performed better at not releasing the guidewire and at maintaining guidewire sterility during this step of the procedure. Adult US-guided CVC simulation studies which have shown improved patient outcomes used longer training interventions and required that trainees demonstrated a certain degree of proficiency on simulated insertions prior to performing procedures clinically (8–10). Barsuk et al (7) found substantial retention of CVC insertion skills up to 1 year following rigorous simulation-based mastery learning. Believing that some residents in our study might question the relevance of this training to their future practice and therefore be less receptive to a longer training session, we intentionally chose a shorter educational intervention. Furthermore, we did not require that residents achieve a mastery standard before concluding the training sessions. The decay in skills found in our study can be attributed to this shorter less rigorous training intervention. Prior to training, we found that residents had low selfconfidence to perform the CVC procedure. Consistent with previous reports on the effect of simulation, residents’ selfconfidence improved following the training sessions (6, 15, 23, 24). Improved self-confidence is generally regarded as a benefit of simulation training. Several studies, however, have demonstrated that following simulation training, self-confidence correlates poorly with actual skills abilities (6, 25) and may remain disparately high as psychomotor skills decay over time (6). In this study, as a group, we found that skills performance had deteriorated at 3-month follow-up while self-confidence remained at the immediate posttraining level. One explanation for this finding might be that following the training, residents retained a basic cognitive understanding of the procedure and therefore remained confident that they could perform the procedure. Without interval practice to reinforce the underlying psychomotor skills, however, these skills deteriorated. Regardless of the reason, similar to previous reports, this finding further supports that self-confidence, alone, should not be used to gauge a trainee’s readiness to independently perform a procedure (6, 25, 26). This study has several important limitations. The small number of subjects coupled with a larger than expected variance in performance scores at each time point resulted in our study having low power to detect differences between the groups for all of the performance measures. Although most of the performance measures appeared to be better than baseline at 3-month follow-up, as a result of the low power of our study, we were unable to conclude that these differences were statistically significant. If the distributions of scores and differences in scores found in our study bore out in a larger study, a post hoc power analysis of our data suggests that 100 subjects would be needed to conclude that the checklist and GRS e422

www.pccmjournal.org

scores were statistically better than baseline at 3-month followup. Furthermore, with respect to performance measures, by stopping residents who failed to achieve needle cannulation within 20 minutes, we were unable to assess their knowledge of subsequent steps in the procedure. This approach may have resulted in lower checklist scores particularly in the pretraining group where seven residents were stopped. Also, due to the limitations of the video cameras and recording angles, we were unable to assess inadvertent arterial puncture, an important potential complication of the CVC procedure. In addition, we are limited by the lack of scoring instruments for the US-guided CVC procedure validated in a pediatric setting. Although the validity and reliability of the checklist used in our study have been previously demonstrated, these studies were conducted on internal medicine residents performing internal jugular and subclavian CVC insertions (8, 9, 15). We modified this checklist for the femoral CVC insertion site and used it to assess a different group of residents. In the adult literature, several Likert-type scale GRS instruments have been reported for the CVC procedure (24, 27). Similar to Huang et al (27), we found near perfect agreement among raters for our GRS instrument. Although we provide some initial evidence for the reliability of the checklist and GRS instruments, validity and other reliability metrics have not been evaluated in this setting. Although it served its purpose for this pilot study, a single-item GRS is somewhat of a crude assessment tool for evaluating a procedure that requires several unique skill sets including US skills and CVC procedural skills. Finally, this study is a convenience sample of pediatric residents at a single institution. This approach allowed us to assess the effect of CVC simulation training on a homogenous group of novice performers. As this was a convenience sample, it is unknown if the characteristics of residents enrolled in the study differed from those who chose to not participate. Although most of the residents rated the training sessions as enjoyable and educationally beneficial, we provided this training to a number of residents who will likely never perform the CVC procedure clinically. In the present era, where educational time has become increasingly limited, a more efficient and potentially more effective strategy would be to provide rigorous simulation-based mastery learning only to those pediatric residents who desire to perform the procedure clinically. Future studies are needed to determine the efficacy of such training in this subgroup of pediatric residents. In addition, future studies with sufficient numbers of subjects are needed to vigorously evaluate the impact of this training on pediatric residents’ clinical performances (T2 level studies) and patient outcomes (T3 level studies). Concomitant with these efforts, additional work is needed to further develop and validate scoring tools to evaluate performances for this procedure in both the simulated and clinical pediatric settings.

CONCLUSIONS This study provides initial evidence that brief US-guided CVC simulation training could be beneficial for training pediatric residents to perform this procedure. The decay in November 2013 • Volume 14 • Number 9

Online Clinical Investigations

skills within 3 months suggests that this simulation training would be most effective if it occurs in close temporal proximity to times when these skills would likely be used clinically and that frequent refresher training should be considered to prevent skills decay.

ACKNOWLEDGMENTS We thank the Glennon Guild for its generous support of the Saint Louis University Clinical Simulation Center which made this study possible. We thank the videotape reviewers for their considerable time and effort. Finally, we thank the pediatric residents at our institution for their enthusiastic participation in the training sessions and their ongoing dedication to learning.

REFERENCES

1. Eppich WJ, Adler MD, McGaghie WC: Emergency and critical care pediatrics: Use of medical simulation for training in acute pediatric emergencies. Curr Opin Pediatr 2006; 18:266–271 2. Donoghue AJ, Durbin DR, Nadel FM, et al: Effect of high-fidelity simulation on Pediatric Advanced Life Support training in pediatric house staff: A randomized trial. Pediatr Emerg Care 2009; 25: 139–144 3. Wayne DB, Didwania A, Feinglass J, et al: Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: A case-control study. Chest 2008; 133:56–61 4. Wayne DB, Butter J, Siddall VJ, et al: Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med 2006; 21:251–256 5. Wayne DB, Siddall VJ, Butter J, et al: A longitudinal study of internal medicine residents’ retention of advanced cardiac life support skills. Acad Med 2006; 81:S9–S12 6. Gerard JM, Thomas SM, Germino KW, et al: The effect of simulation training on PALS skills among family medicine residents. Fam Med 2011; 43:392–399 7. Barsuk JH, Cohen ER, McGaghie WC, et al: Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Acad Med 2010; 85:S9–12 8. Barsuk JH, McGaghie WC, Cohen ER, et al: Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009; 37:2697–2701 9. Barsuk JH, Cohen ER, Feinglass J, et al: Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009; 169:1420–1423

Pediatric Critical Care Medicine

10. Burden AR, Torjman MC, Dy GE, et al: Prevention of central venous catheter-related bloodstream infections: Is it time to add simulation training to the prevention bundle? J Clin Anesth 2012; 24:555–560 11. Accreditation Council on Graduate Medical Education. Program requirements for graduate medical education in pediatrics. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/Program Requirements/320_pediatrics_07012007.pdf. Accessed February 15, 2011 12. Weinberg ER, Auerbach MA, Shah NB: The use of simulation for pediatric training and assessment. Curr Opin Pediatr 2009; 21:282–287 13. Nasca TJ, Philibert I, Brigham T, et al: The next GME accreditation system–rationale and benefits. N Engl J Med 2012; 366:1051–1056 14. Tsui JY, Collins AB, White DW, et al: Videos in clinical medicine. Placement of a femoral venous catheter. N Engl J Med 2008; 358:e30 15. Barsuk JH, McGaghie WC, Cohen ER, et al: Use of simulationbased mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med 2009; 4:397–403 16. Nunnally JC, Bernstein IH: Psychometric Theory. Third Edition. New York, McGraw-Hill, 1994 17. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 1977; 33:159–174 18. McGaghie WC, Draycott TJ, Dunn WF, et al: Evaluating the impact of simulation on translational patient outcomes. Simul Healthc 2011; 6(Suppl):S42–S47 19. Nishisaki A, Hales R, Biagas K, et al: A multi-institutional high-fidelity simulation “boot camp” orientation and training program for first year pediatric critical care fellows. Pediatr Crit Care Med 2009; 10:157–162 20. Grant EC, Marczinski CA, Menon K: Using pediatric advanced life support in pediatric residency training: Does the curriculum need resuscitation? Pediatr Crit Care Med 2007; 8:433–439 21. Skidmore MB, Urquhart H: Retention of skills in neonatal resuscitation. Paediatr Child Health 2001; 6:31–35 22. Smith KK, Gilcreast D, Pierce K: Evaluation of staff’s retention of ACLS and BLS skills. Resuscitation 2008; 78:59–65 23. Yager PH, Lok J, Klig JE: Advances in simulation for pediatric critical care and emergency medicine. Curr Opin Pediatr 2011; 23:293–297 24. Millington SJ, Wong RY, Kassen BO, et al: Improving internal medicine residents’ performance, knowledge, and confidence in central venous catheterization using simulators. J Hosp Med 2009; 4:410–416 25. Brydges R, Nair P, Ma I, et al: Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 2012; 46:648–656 26. Wayne DB, Butter J, Siddall VJ, et al: Graduating internal medicine residents’ self-assessment and performance of advanced cardiac life support skills. Med Teach 2006; 28:365–369 27. Huang GC, Newman LR, Schwartzstein RM, et al: Procedural competence in internal medicine residents: Validity of a central venous catheter insertion assessment instrument. Acad Med 2009; 84:1127–1134

www.pccmjournal.org

e423

Simulation training for pediatric residents on central venous catheter placement: a pilot study.

To assess the effect of simulation training on pediatric residents' acquisition and retention of central venous catheter insertion skills. A secondary...
464KB Sizes 0 Downloads 0 Views