Nurse Education Today 35 (2015) e6–e15

Contents lists available at ScienceDirect

Nurse Education Today journal homepage: www.elsevier.com/nedt

Review

The effects of simulation-based learning using standardized patients in nursing students: A meta-analysis☆,☆☆ Pok-Ja Oh a, Kyeong Deok Jeon b, Myung Suk Koh a,⁎ a b

Department of Nursing at Sahmyook University in Seoul, South Korea Department of Nursing, Graduate School, Sahmyook University, Seoul, South Korea

a r t i c l e

i n f o

Article history: Accepted 20 January 2015 Keywords: Students Nursing Patient simulation Meta-analysis

s u m m a r y Purpose: The aim of this study was to evaluate the effect of simulation-based learning using standardized patients (SPs) on cognitive, affective, and psychomotor domain outcomes of learning in nursing students. Methods: MEDLINE via PubMed, Cochrane Library CENTRAL, EMBASE, CINAHL, and several Korean electronic databases (to June 2014) were searched. The RevMan 5.3 program of the Cochrane library was used for data analysis. Results: A meta-analysis was conducted of 18 controlled trials (4 randomized and 14 non-randomized designs), with a total of 1326 nursing students. Overall, simulation-based learning using SPs appeared to have beneficial effects on the cognitive, affective, and psychomotor domains of learning. In subgroup analysis, use of SPs showed significant effects on knowledge acquisition (d = 0.38, p = .05, I2 = 42%), communication skill (d = 1.86, p b .001, I2 = 15%), self-efficacy (d = 0.61, p b .001, I2 = 6%), learning motivation (d = 0.77, p b .001, I2 = 0%) and clinical competence (d = 0.72, p b .001, I2 = 0%). Treatment effects on critical thinking (p = .75) and learning satisfaction (p = .43) were not significant. Conclusion: The findings of the current study suggest that simulation-based learning using SPs might have a positive impact on self efficacy and learning motivation that affects knowledge and clinical skill acquisition. Therefore, these findings demonstrate that, if integrated appropriately, an SP educational approach can be used in academic settings as an active learning methodology. © 2015 Elsevier Ltd. All rights reserved.

Introduction In today's dynamic and complicated health care environment, new nursing graduates are expected to be capable of mastering nursing skills in a timely manner and to become critical thinkers with the ability to solve complex problems efficiently (Norman, 2012). Therefore, in practice-based healthcare professions, teaching and learning methods are focused on assimilation of clinical knowledge and skills. However, contemporary teaching and learning approaches do not always facilitate the development of a requisite level of these clinical skills (Yuan et al., 2012). In order to develop these skills, which include communication skills, the ability to collaborate productively in groups or teams, critical reasoning, and self-evaluation, many nursing education institutions have adopted simulation-based learning (SBL) in their curricula (Engel, 1997). Simulation-based learning could be a key component in adequate preparation of nursing students for the transition into the ever-changing health care environment (Norman, 2012). ☆ Grant support: This paper was supported by the Sahmyook University Research Fund. ☆☆ Previous presentation: None. ⁎ Corresponding author at: Department of Nursing, Sahmyook University, Kongnungdong Hwarangro-815 Nowon-gu, Seoul 139-742, South Korea. Tel.: +82 2 3399 1586; fax: +82 2 3399 1594. E-mail address: [email protected] (M.S. Koh).

http://dx.doi.org/10.1016/j.nedt.2015.01.019 0260-6917/© 2015 Elsevier Ltd. All rights reserved.

In this context, the use of simulation as an educational tool is becoming increasingly prevalent in medical and nursing education (Yuan et al., 2012). Simulation is composed of different modalities, including virtual reality, high-fidelity human simulators, and standardized patients (Luctkar-Flude et al., 2012). Use of standardized patients (SPs), one of the simulation-based learning (SBL) approaches, has recently been introduced to support the development of health assessment competence and therapeutic communication among nurse learners (RobinsonSmith et al., 2009). Developed by Barrows in the 1960s, a standardized patient is defined as an individual who has been carefully trained to present an illness or scenario in a systematic, unvarying manner (Barrows, 1993; Becker et al., 2006). Standardized patients present an actual patient problem in a clinically relevant and realistic way. Nursing educators considering application of a SP curriculum are confronted with problems created by the disparity of information found in previous studies. It is important for educators to find the appropriate teaching method that enhances the knowledge and clinical performance skills of nurses. Although there is a growing body of literature on educational use of SPs in teaching and leaning, only two systematic reviews have been conducted to investigate the use of SPs in nursing and medical education. These reviews suggested that the use of SPs in medical and nursing students improves knowledge acquisition (Norman, 2012), psychomotor skill (May et al., 2009; Norman, 2012) and communication skills

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

(May et al., 2009). As mentioned above, two reviews for the use of SPs have suggested its effectiveness as an educational strategy in nursing education; however, there is a lack of systematic review and/or metaanalysis regarding the effectiveness of the SP educational approach in nursing education. Thus, this review was conducted in order to identify the best available evidence regarding the effects of the SP educational use on knowledge acquisition and skill improvement in nursing. In meta-analysis data from controlled trials are combined in order to evaluate the validity of the evidence for determination of the effectiveness of intervention. Thus, there are challenges in evaluating studies in this area. To better understand the impact of using the SP educational approach, we conducted a meta-analysis of clinical trials of the SP educational approach. Because of the small number of RCT studies on this topic, we included a non-RCT design. The primary aim of this study was to evaluate a robust estimate of the effect of the SP educational approach on cognitive, affective, and psychomotor domain outcomes of learning and to identify intervention moderators.

e7

Study Selection Relevant studies were identified by electronic search, and were then downloaded to a reference management database, followed by elimination of duplicates. Two levels were applied for study selection: titles and abstracts were used in primary screening; then, the full text was used if needed. Independent screening of each study was performed by two of the authors, according to the inclusion criteria. Studies were included if they: (1) include undergraduate and graduate nursing students, (2) measure simulation learning using standardized patients, (3) measure learning outcomes (i.e., knowledge acquisition, clinical skill performance, or critical thinking), (4) are randomized control trials (RCTs) and non-RCTs and (5) include sufficient data for calculation of effect sizes between the treatment and control groups. Data Extraction

Our review followed the guidelines proposed by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) (Liberati et al., 2009).

The following information from the study was extracted based on a predesigned coding manual form: (1) authors and years of publication, (2) country and study design, (3) sample size and characteristics of participants, (4) intervention details (number of sessions, duration, and total time) and (5) measurement and outcomes. Pilot testing was performed on three studies before data extraction by two independent reviewers.

Eligibility Criteria

Risk of Bias Assessment

Eligibility criteria are detailed in accordance with the PICOS (participants, interventions, controls, outcomes, and studies) framework.

A pilot test was conducted on two studies, followed by independent review of each study by two authors for determination of methodological quality. Disagreements were resolved during meetings between the authors. The seven-item scale of RoB (Risk of Bias), developed by the Cochrane Bias Method Group, was used for evaluation of RCT studies. The Risk of Bias Assessment tool for Non-randomized Studies (RoBANS), developed by Kim et al. (2013b), was used for non-RCTs. As in the Cochrane Risk-of-Bias tool, the bias types in RoBANS are selection, performance, detection, attrition, and reporting biases. However, the domains of selection and performance biases were modified to include the selection of participants, confounding variables, and the measurement of exposure (Kim et al., 2013a).

Methods

Participants Participants were undergraduate and graduate nursing students. Interventions Simulation learning was defined as an educational strategy that replaces or amplifies experiences that replicate aspects of the real world in an interactive fashion (Gaba, 2004) and using standardized patients (SPs) who carefully trained in order to accurately and consistently play the role of a patient with a health concern and provide helpful verbal and written feedback to the learner (Nestel et al., 2010). Interventions were excluded if simulation learning interventions were used along with computer simulation. Controls Both no treatment (usual learning) and active (attention, placebo) control conditions were considered. Outcome The primary outcomes were knowledge acquisition and clinical skill performance. Secondary outcomes were learning outcomes (i.e. problem solving ability, critical thinking, and self-reported levels of confidence). Studies Both randomized controlled trials (RCTs) and non-RCTs including comparative study were considered for inclusion. Data Sources Studies were identified through the Cochrane Library CENTRAL, EMBASE, MEDLINE, CINAHL, and several Korean databases (KMBASE, KOREAMED, RISS, KISS, and NANET). The main search strategy combined terms indicating simulation learning, nursing students, and study design (see Appendices 1–3 for complete list). Searches were limited to articles in Korean or English, and any study published from the earliest publication date within each database to June 2014 was considered. In addition, we conducted a search of the Google Scholar database and a manual review of reference lists.

Statistical Analysis For studies with data of sufficient quality and similar in simulation learning and outcome measures, we combined data in a meta-analysis in order to provide a pooled effect estimate. All data were entered into RevMan 5.2.11 (http://tech.cochrane.org/revman/download), where standardized deviations and 95% confidence intervals (CIs) were calculated and pooled. Mean and standard deviations of outcomes were used for computation of standardized mean differences (SMD). For each analysis, a heterogeneity test was performed using I2 statistics, which measures the extent of inconsistency among results and is interpreted approximately as the proportion of total variation across studies attributable to heterogeneity and not to chance. I2 = 25% was considered low, 50% moderate, and 75% high (Higgins and Green, 2008). I2 values higher than 50% were considered as having substantial heterogeneity, and the random-effects model was therefore applied for analysis of the data (Higgins and Green, 2008). In addition, we performed subgroup analysis according to prespecified variables, including study design and intervention characteristics (i.e., course name, student level). If there was no statistical heterogeneity, we would have used a fixed-effect model. Subsequently, we performed subgroup analyses according to categories of learning outcomes and potential moderating variables such as study design (pre–post vs post only). Design was chosen as a potential moderator because different designs were included in the meta-analysis and we considered it important to analyze by subgroup. To test for publication bias, a funnel plot, which graphs the effect size of each study according to its respective SE, was used. We assumed

e8

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

the existence of publication bias if there were no small studies with effect sizes favoring control groups (Higgins and Green, 2008). A twotailed p value of less than .05 was considered significant. Results Study Selection From a search of databases and other references, 7341 studies were gathered. Following removal of duplicates, 1111 publications were screened. Screening of titles and abstracts resulted in identification of 82 studies having potential relevance. A review of these studies found that 64 studies did not meet the inclusion criteria; the reasons were as follows: 54 studies were non-relevant interventions (i.e. high fidelity simulation), five studies had insufficient data, two studies were descriptive, two were non-nursing student studies, and one study had a one-group design. A total of 18 studies were included in the current meta-analysis (Fig. 1). Overall Description of the Studies The characteristics of the 18 studies included in the review are summarized in Table 1. Fifteen studies (83.3%) from Korea and one study each from the U.S., Canada, and Iran were included. Seven studies (38.9%) used a pre–post design and 11 studies compared only post-test. Four studies (22.2%) used an RCT design and 14 studies used a non-RCT design. The sample size across the 18 included studies varied between 30 and 180, with a total of 1329 participants. The grades of nursing students were reported

as follows: sophomore (eight studies), junior (six studies), and senior (four studies). The subject courses of simulation learning were diverse, with medical surgical nursing (k = 4), fundamental nursing (k = 4), psychiatric nursing (k = 2), health assessment (k = 2), emergency care (k = 2), maternity care (k = 3), and comprehensive class (k = 1). Among 18 studies, the number of sessions varied from 1 to 6, with a mean of 1.7 sessions. Total hours of simulation learning were between 12 min and 6.5 h, with a mean of 2.6 h. All studies included one control group. Usual teaching control groups were most common (88.9%); however, in two studies (11.1%), additional learning, such as a case study class was included. The outcome measurements found in this study were categorized according to three domains of learning: cognitive outcomes, affective outcomes, and psychomotor outcomes. As the cognitive domain outcomes, knowledge acquisition (k = 4), problem solving ability (k = 3), critical thinking (k = 3) and communication skill (k = 9) were evaluated. Major measures of these were the investigator-designed scale. As the affective domain outcomes, selfefficacy (k = 6), learning satisfaction (k = 7), and learning motivation (k = 3) were evaluated. Clinical competence (k = 15) was tested as the psychomotor domain outcome and major measure of clinical competence was the investigator-designed tool. Clinical performance was assessed by instructors or investigators. Quality of Studies Of the four RCT studies, two studies (50%) reported adequate details on randomization and two studies did not describe details on randomization sequence. In the case of allocation concealment, none of the studies (100%) provided adequate details. Thus, they were all judged

Fig. 1. Selection of included studies.

Table 1 Descriptive summary of included studies (N = 18). Author (year)

Country Design

Sample/grade

Outcome (results)

Therapeutic communication skills & knowledge (−)/self-evaluation of SP encounter Nursing competence (post-only) (+)/communication skill (post-only) (+)/learning satisfaction (post-only) (−) Clinical competence (post-only) (+)/selfdirected learning readiness (post-only) (+)/problem-solving ability (post-only) (+) Nursing competence (+)/communication skill (+)/knowledge (+)/learning satisfaction (−) Knowledge (−)/self-confidence (−)/satisfaction (+)/communication abilities (+)/nursing performance (+) Anxiety (+)/self-confidence (+)

2h

Traditional education

Investigator designed scale/Yoo's (2000) scale (communication scale & learning satisfaction)

2 days/2

2.4 h

Manikin practice

Investigator-designed tool/Guglielmino's (1977) scale/Heppner & Peterson's (1982) scale

1 week/1

20 min

Regular lectures

1 day/1

6h

Case study

9 practical stations

6.5 h

Traditional education

Investigator designed scale/Yoo's (2000) scale/investigator designed scale/Yoo's (2000) scale Investigator designed scale/10 cm NRS/Yoo's (2000) scale (education satisfaction & communication)/checklist Spielberger anxiety scale/investigator designed scale

Non-RCT Nursing/junior (post-only)

1 day/1

33 min

Unclear

Investigator designed scale/Youn's (2004) scale

Clinical competence (−)/critical thinking disposition (−)

Non-RCT (pre–post) Non-RCT (pre–post)

Unclear

30 min

Unclear

2 days/2

2.3 h

Case study

Unclear/6

2.1 h

2 weeks/3

6h

Traditional education Traditional lecture

Yoon's (2004) scale/Woo's (2000) scale/Yang & Park's (2004) scale Kim et al,'s (2010) medication nursing measurement scale/investigator designed scale (knowledge & problem-solving capacity)/10 cm NRS tool/Kim's (2012) scale Professor's assessment checklist/investigator designed scale Yoo's (2001) scale/investigator's scale/Yoo's (2001) scale

Critical thinking disposition (−)/problem solving process (+)/clinical competence (+) Skill performance (+)/Knowledge (+)/problem-solving capacity (+)/self-confidence (−)/educational satisfaction (−) Nursing competency (+)/satisfaction of simulation education (−) Decision making skill (−)/nursing skill performance (+)/communication skill (+)

1 day/1

3h

Investigator designed tool (followed nursing education algorithms) Investigator designed HAEME/respiratory assessment checklist Ayres' (2005) scale (motivation to transfer & self-efficacy in nursing) Investigator designed scale

Nursing performance ability (+)

2 days/2

General education Unclear Community volunteer SP 2h Traditional education 1.4 h Manikin

1 day/1

12 min

Manikin

Investigator designed scale/Yoo's (2000) scale/Sherer & Maddux's (1982) scale

4h

Traditional education

6h

Manikin

RCT (pre–post)

Nursing/senior

E: 58 C: 89 Psychiatric nursing (communication & depression care) Non-RCT Nursing/junior E: 22 C: 22 Health assessment (post-only) (acute meningitis patient) Non-RCT Nursing/sophomore E: 31 C: 31 Fundamental nursing (post-only) (subcutaneous insulin injection) RCT Nursing/junior E: 52 C: 56 DM diet education (post-only) (multimedia learning)

Korea

Hyun et al. (2009)

Korea

Jang (2013)

Korea

Non-RCT (pre–post)

Nursing/junior

Khadivzadeh & Erfanian (2012) Kim (2012)

Iran

RCT (pre–post)

Nursing/senior

Korea

Ko & Kim (2014) Kwak (2013)

Korea

Lee et al. (2010) Lee (2011)

Korea

Lee et al. (2013) Luctkar-Flude et al. (2012) Park & Kwon (2012) Seong (2008)

Korea

E: 25 C: 29 Oncology nursing (pain, side effects, and emergency situations) 60 Midwifery course (IUD)

E: 29 C: 25 Maternity nursing (postpartum hemorrhage) Nursing/junior E: 33 C: 32 Emergency and critical nursing Nursing/sophomore E: 20 C: 19 Fundamental nursing (medication error prevention)

Non-RCT Nursing/junior E: 68 C: 39 (post-only) Non-RCT Nursing/sophomore E: 36 C: 36 (post-only) Non-RCT (pre–post) Non-RCT (post-only) Non-RCT (pre–post) Non-RCT (post-only)

Nursing/sophomore E: 96 C: 84 Nursing/sophomore E: 14 C: 16 Nursing/senior

E: 23 C: 21

Nursing/sophomore E: 35 C: 36

E: 48 C: 54

Adult nursing (care for dyspnea patients) Maternity nursing (intrapartum and postpartum nursing care) Adult nursing (care of urinary patients) Health assessment course (respiratory) Psychiatric nursing (depression patient care) Adult nursing (subcutaneous insulin injection practice) Comprehensive practice

Unclear 1 day/1

Sok et al. (2009)

Korea

RCT Nursing/senior (post-only)

Yoo et al. (2002)

Korea

Yoo (2001)

Korea

1 days/1 Non-RCT Nursing/sophomore E: 36 C: 39 Fundamental nursing (post-only) course (Foley catheterization, communication) 7 days/2 Non-RCT Nursing/sophomore E: 36 C: 40 Fundamental nursing (post-only) course (basic nursing care + communication skill)

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

1 day/1

Eom et al. (2010)

Korea

Measurement

Communication knowledge test

Korea

Korea

Control

Traditional instruction

Choi et al. (2013)

Canada

Intervention Duration/no. Total of session time 1h

USA

Korea

Class (content)

7 weeks/1

Becker et al. (2006)

Korea

Sample size

Self-efficacy (−)/satisfaction (control: +)/respiratory assessment (+) Motivation to transfer (+)/self-efficacy in nursing (+) Clinical competence (+)/communication skill (+)/learning satisfaction scale (−)

Clinical competence (post-only) (+)/communication skill (post-only) (+)/self-efficacy scale (pre–post) (+) Foley catheterization skills Investigator designed scale (Foley (+)/communication skills (+)/learning catheterization tool & communication skill)/Keller's (1983) learning motivation scale motivation (+) Clinical competence (+)/communication Investigator-designed scale (clinical skill (+)/learning motivation (+) competence tool & communication skill)/Keller's (1983) learning motivation scale

Exp. = experimental group; Con. = control group; RCT = randomized controlled trials; Non-RCT = non-randomized controlled clinical trial; NRS: Numeric rating scale; HAEME: Health Assessment Educational Modality Evaluation; (+) = e9

statistically significant difference between two groups; (−) = no significant difference between two groups.

e10

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

to be “unclear”. For practical reasons, participants and providers of the intervention could not be masked according to group allocation in these studies. One study (25.5%) reported on blinding of participants and intervention providers and others did not report that. In terms of blinding outcome assessor, two studies (50%) reported on blinding of outcome assessment. In assessment of attrition bias, all studies were considered low risk. Because the pre-specified expected outcome of interest of the studies was clearly reported, risk of reporting bias was considered low. Regarding other types of bias, monitoring of the intervention procedure, along with use of an intervention manual, in psychological intervention studies was considered essential in assessing the risk of bias (Ranchor et al., 2012). All studies provided an intervention manual (or scenario), however, it was not clear whether or not they evaluated the intervention procedure. Assessment of 14 non-RCT studies for selection bias, performance bias, detection bias, attrition bias, and reporting bias was performed using RoBANS. Overall, except for detection bias, these studies were rated as low risk for these biases. Regarding detection bias, blinding outcome assessor was used in seven studies (50%) and seven studies (50%) were rated as unclear.

Effects of Simulation Based Learning Using Standardized Patients Effects of Simulation Based Learning Using SPs on Cognitive Domain of Learning A combined analysis across 19 studies (1346 subjects) that measured the cognitive domain of learning showed a large amount of heterogeneity (I2 = 85%) and significant treatment effects on the cognitive domain of learning (d = 0.85, 95% CI [0.55, 1.16], p b .001). Knowledge acquisition, problem solving capacity, critical thinking, and communication skill were measured as outcome variables on the cognitive domain of learning. Due to the heterogeneity, subgroup analyses were conducted based on study design and learning environment variables such as student level (grade). In subgroup analyses, three studies involving 222 nursing students tended to have significant effects on knowledge acquisition (d = 0.38, 95% CI [0.00, 0.76], p = .05), indicating some heterogeneity between study estimates (I2 = 42%) (Fig. 3). In addition, seven studies involving 512 nursing students showed significant effects on communication skill (p b .001) and statistical homogeneity was observed between study estimates (I2 = 0–15%) (Fig. 3). However, there were no significant effects in the subgroups of critical thinking studies (p = .75,

Fig. 2. Forest plot of effect size and 95% CI by simulation based learning using SPs on knowledge, problem solving capacity, and critical thinking and funnel plot of effect sizes by standard error.

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

I2 = 0%) and problem solving capacity (p = .14, I2 = 90%). In examination of publication bias, a review of the funnel plot of effect sizes by SEs showed a slightly uneven distribution of outcome (p = .028) (Fig. 2). Effects of Simulation Based Learning Using SPs on Affective Domain of Learning A combined analysis across 16 studies (967 subjects) that measured self-efficacy, learning satisfaction, and learning motivation showed significant treatment effects on the affective domain of learning (d = 0.42, 95% CI [0.19, 0.64], p b .001), indicating significant heterogeneity between study estimates (I2 = 65%). In subgroup analyses, significant effects on self-efficacy (d = 0.61, 95% CI [0.37, 0.86], p b .001) and learning motivation (d = 0.77, 95% CI [0.43, 1.10], p b .001) were observed, indicating statistical homogeneities between study estimates (I2 = 0–43%) (Fig. 3). However, there was no significant treatment effects on learning satisfaction (p = .43, I2 = 43%). In examination of publication bias, a review of the funnel plot of effect sizes by SEs showed a somewhat even distribution of outcome (Fig. 3). Effects of Simulation Based Learning Using SPs on Psychomotor Domain of Learning Results of a combined analysis across 15 studies (1103 subjects) that measured the psychomotor domain of learning (i.e., clinical competence) are shown in Fig. 4. Although significant heterogeneity was observed (I2 = 89%), the current meta-analysis showed a significant treatment effect on the psychomotor domain of learning (d = 1.06, 95% CI [0.67, 1.46], p b .001). In subgroup analysis according to study design, four studies of pre–post design involving 338 subjects showed a significant effect on clinical competence (d = 0.72, 95% CI [0.49, 0.94], p b .001), indicating a statistical homogeneity (I2 = 0%). A funnel plot of effect sizes by their SEs showed a fairly even distribution of studies (Fig. 4). Discussion Although simulation-based learning using SPs is becoming increasingly prevalent in nursing and medical education, few comprehensive systematic reviews or meta-analyses of its effectiveness have been reported. In response to this need, we conducted a meta-analysis of the effects of simulation-based learning using SPs on Bloom's three domains of learning; cognitive, affective, and psychomotor outcomes of learning.

e11

The results of our meta-analyses suggest that simulation-based learning using SPs appeared to have beneficial effects on the cognitive (d = 0.85, p b .001), affective (d = 0.42, p b .001), and psychomotor domains of learning (d = 1.06, p b .001). The current results suggest that the SP educational approach might work better in the psychomotor domain of learning, such as clinical competence, followed by cognitive and affective. In simulation-based learning, students can analyze their technical and affective skills critically, as well as evaluate decisions. Once they have analyzed their mistakes and received feedback from their instructors, the scenario can be repeated for augmentation of skills and to increase their ability to retain and apply knowledge. This type of simulation might be helpful to students in increasing their capacity to synthesize information from multiple sources and to safely utilize their clinical skills (Gaba, 2004). However, before a conclusion could be made with regard to the findings, a more thorough evaluation of the heterogeneity (I2 = 65–89%) was necessary. Studies gathered for conduct of a metaanalysis differ in their characteristics; such differences could impact the effects. Thus, we conducted subgroup analyses based on study design and learning environment variables such as student level (grade). The cognitive outcomes evaluated in our analysis consisted of knowledge, problem solving capacity, critical thinking and communication skill. These outcomes are in agreement with major categories of cognitive processes, which can be thought of as degrees of difficulty (Krathwohl, 2002). In our subgroup analyses according to categories of the cognitive domain, use of the SP educational approach for nursing students resulted in statistically significant enhancement of scores on standardized knowledge exams and communication skill. Our results showing significant effects on knowledge acquisition and communication skills are in accordance with those of other systematic review studies (May et al., 2009; Norman, 2012) in this simulation-based learning (SBL) field. These results support that the use of SPs is particularly helpful in psychosocial nursing where observation of the student– patient interaction is often considered burdensome (O'Connor et al., 1999). These results confirmed that use of SPs has a particularly important effect on student–patient interaction nursing. In the case of clinical thinking, SPs did not work on clinical thinking, perhaps because of low power attributable to a small number of studies. In fact, the number of sessions and operating hours of the SP educational approach was short, with a mean of 1.7 sessions and a mean of 2.6 h, respectively. Therefore, conduct of further well-designed studies with

Fig. 3. Forest plot of effect size and 95% CI by simulation based learning using SPs on self-efficacy, communication skill, satisfaction, and motivation and funnel plot of effect sizes by standard error.

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

0.2 0.4

0.3

Standard error

0.1

0.0

e12

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

Standardised mean difference

Fig. 4. Forest plot of effect size and 95% CI by simulation based learning on clinical competence and funnel plot of effect sizes by standard.

a large sample will be needed in order to draw a conclusion regarding the effectiveness of clinical thinking. When facing ambiguous situations, unique cases, or unresolved problems not covered in their textbooks, use of cognitive skills enables students to arrive at clinical judgments based on the available information (Oermann and Gaberson, 2009). Thus, development of high-level cognitive skills is essential. The affective domain includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasm, motivations, and attitudes (Krathwohl, 2002). In this study, outcome variables of the affective domain were measured: (1) learning motivation (2) self-efficacy and (3) learning satisfaction. In subgroup analyses, use of the SP educational approach for nursing students resulted in significantly improved self-efficacy and learning motivation. However, the SP educational approach did not show a statistically significant benefit on perceived learning satisfaction, perhaps because of low power attributable to within-group heterogeneity. Our results showing significant effects on perceived self-efficacy are not in accordance with those of other systematic review studies in this simulation-based learning (SBL) field (Kim et al., 2013a; Lapkin et al., 2010; Weaver, 2011). Higher levels of confidence can potentially have a beneficial effect on learning outcomes. On the other hand, previous systematic review studies of high-fidelity patient simulation (Kim et al., 2013a; Lapkin et al., 2010; Weaver, 2011) reported significant effects on learning satisfaction. Learner satisfaction is important as it may potentially enhance students' engagement, thereby facilitating learning (Lapkin et al., 2010). Conduct of additional RCT studies on learning satisfaction will be needed before we can draw a more accurate conclusion. In this study, clinical competence (k = 15) of the psychomotor domain was evaluated the most as the outcome variable of the SP educational approach. This result supports that standardized patients are

used most effectively to teach and evaluate clinical skills such as interpersonal communication, history taking and interviewing, physical and psychological assessment, and patient education (Barrows, 1993; Baumann and Rideout, 2001; Stroud et al., 1999; Thomas et al., 2001). In the psychomotor domain of learning, use of the SP educational approach had significant effects on clinical competence (d = 1.06). Even though clinical competence was the only outcome variable of the psychomotor domain, there was significant heterogeneity (I2 = 89%). The 15 current studies of clinical competence were conducted using a mixed group study design (pre–post vs post only design/RCT vs non RCT). According to subgroup analyses, four studies of nonequivalent control group pre–post design showed a significant moderate effect on clinical competence, indicating no heterogeneity (I2 = 0%). Our result is in accordance with those of other systematic review studies in this topic (May et al., 2009; Norman, 2012). In addition, the results of this study indicate that use of the SP educational approach can enable learning and practice of formative skills in a controlled, less threatening environment. This approach also focuses students' attention on the acquisition of clinical skills and allows them to develop comfort in managing sensitive patient issues. In addition, it allows for immediate feedback on students' performance and provides the opportunity for improvement (Becker et al., 2006). The National Board of Medical Examiners has incorporated SPs into the clinical skills portion of the medical licensing examination (De Champlain et al., 1998a,b, 1999). This fact supports that the SP method provides valid and reliable measurements of clinical skills. Limitation There are several limitations of this review. First, because several gray literature reports (i.e., dissertations) were retrieved from Korean

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

searches, and no unpublished research and non-English studies were retrieved as a result of international searches, some relevant studies might have been overlooked, resulting in inclusion of a small number of RCT studies. Second, based on the instruments used for evaluation of outcomes in the reviewed studies, evidence for use in determining the best instrument is inconsistent. Most research focused on validation of performance checklists; however, the inter-rater reliability of the marking was rarely described. The third limitation involves the methodological quality of selected studies; the reviewed RCTs lack methodological quality since allocation concealment and double blindness were not clarified. Most of the reviewed studies had a nonequivalent control group post only design, which was at risk for selection bias. The use of small sample sizes in many studies resulted in insufficient power to detect effects of the SP educational approach on the outcomes. Conduct of more high quality RCTs with larger sample sizes is necessary in order to determine the effect of the SP educational approach. Finally, funnel plot asymmetry, which may indicate publication bias, was observed for cognitive outcomes. Most of the reviewed studies did not provide sufficient evidence regarding the effect of the SP educational approach in application of knowledge and skills in novel situations. Evaluation of higher levels of the cognitive domain, such as application, analysis, and transfer is needed. Conclusion Despite some limitations in this study, a tentative conclusion can be reached, that simulation-based learning using SPs might have beneficial effects on knowledge acquisition, communication skills, self-efficacy, learning motivation, and clinical skill acquisition. The findings of the current study suggest that use of an SP educational approach might have a positive impact on self-efficacy and learning motivation that affects knowledge and clinical skill acquisition. These findings therefore indicate that an SP educational approach, if integrated appropriately, can be used in academic settings as an active learning methodology. Conduct of this meta-analysis resulted in a comprehensive synthesis of available empirical studies; however, convincing evidence for use in evaluation of the SP educational approach for critical thinking and problem solving capacity is still lacking. Conduct of further well-designed studies with a large sample size will promote better understanding with regard to whether the SP educational approach has a late effect on clinical thinking and problem solving capacity. Author Contributions Conception and design: All authors. Collection and assembly of data: All authors. Data analysis and interpretation: All authors. Manuscript writing: All authors. Final approval of manuscript: All authors. Acknowledgments This paper was supported by the Sahmyook University Research Fund (20155931). The authors would like to thank the following people who provided support at various stages of the study process. JuYoung Seo, Volunteer Research Assistant, Sahmyook University, Seoul, Korea. YoSub Ha, Volunteer Research Assistant, Korea University, Seoul, Korea. Appendix 1. Search Strategy — MEDLINE (PubMed) (June 2014) #1. ((“Patient Simulation”[Mesh]) OR “Education, Nursing”[Mesh]) OR “Models, Educational”[Mesh] 78,930

e13

#2. ((“Patient Simulation”[tiab] OR “Education, Nursing”[tiab] OR “Models, Educational”[tiab] OR “Competency-Based Education”[tiab] OR “simulation”[tiab] OR “simulator”[tiab] OR “human patient simulator”[tiab] OR “patient simulator”[tiab] OR “standardized patient”[tiab] OR “nursing education”[tiab] OR “educational intervention”[tiab] OR “critical thinking”[tiab] OR “learning theories”[tiab] OR “Simulation Laboratories”[tiab] OR “Simulation scenarios”[tiab] OR “decision making”[tiab] OR “Simulation experiences”[tiab] OR “Constructivism learning theory”[tiab] OR “Adult-learning theory”[tiab] OR “simulation program”[tiab])) 200,110 #3. 1 or 2 267,404 #4. (“Students, Nursing”[Mesh]) OR “Students, Medical”[Mesh] 38,362 #5. ((“Students, Nursing”[tiab] OR “Students, Medical”[tiab] OR “students”[tiab] OR “student”[tiab])) 164,973 #6. 4 OR 5 176,844 #7. 3 AND 6 24,482 #8. (RCT & CCT) (“controlled clinical trial”[ptyp]) OR (“randomized controlled trial”[ptyp]) 452,802 #9. #7 AND #8 883 Appendix 2. Search Strategy — EMBASE (June 2014) #1. (EMtree) ‘nursing education’/exp OR ‘educational model’/exp OR ‘simulator’/exp OR ‘simulation’/exp 249,512 #2. (Advanced) ‘Patient Simulation’:ab,ti OR ‘Education, Nursing’:ab,ti OR ‘Competency-Based Education’:ab,ti OR ‘simulation’:ab,ti OR ‘Patient simulation’:ab,ti OR ‘simulator’:ab,ti OR ‘human patient simulator’:ab,ti OR ‘patient simulator’:ab,ti OR ‘standardized patient’:ab,ti OR ‘nursing education’:ab,ti OR ‘Models, Educational’:ab,ti OR ‘educational intervention’:ab,ti OR ‘critical thinking’:ab,ti OR ‘learning theories’:ab,ti OR ‘Simulation Laboratories’:ab,ti OR ‘Simulation scenarios’:ab,ti OR ‘decision making’:ab,ti OR ‘Simulation experiences’:ab,ti OR ‘Constructivism learning theory’:ab,ti OR ‘Adult-learning theory’:ab,ti OR ‘simulation program’:ab,ti 221,797 #3. #1 or #2 395,649 #4. (EMtree) ‘nursing student’/exp OR ‘medical student’/exp 59,042 #5. (Advanced) ‘nursing student’:ab,ti OR ‘medical student’:ab,ti OR ‘students’:ab,ti OR ‘student’:ab,ti 218,594 #6. #4 OR #5 238,626 #7. #3 AND #6 27,157 #8. (RCT & CCT) (‘controlled clinical trial’/de OR ‘randomized controlled trial’/de) 464,875 #9. #7 AND #8 677 Appendix 3. Search Strategy — Cochrane Library CENTRAL (June 2014) #1. MeSH descriptor: [Patient Simulation] explode all trees 295 #2. MeSH descriptor: [Education, Nursing] explode all trees 651 #3. MeSH descriptor: [Models, Educational] explode all trees 222 #4. “Patient Simulation” OR “Education, Nursing” OR “Models, Educational” OR “Competency-Based Education” OR “simulation” OR “simulator” OR “human patient simulator” OR “patient simulator” OR “standardized patient” OR “nursing education” OR “educational intervention” OR “critical thinking” OR “learning theories” OR “Simulation Laboratories” OR “Simulation scenarios” OR “decision making” OR “Simulation experiences” OR “Constructivism learning theory” OR “Adult-learning theory” OR “simulation program” 13,388 #5. #1 or #2 or #3 or #4 13,388 #6. MeSH descriptor: [Students, Nursing] explode all trees 238 #7. MeSH descriptor: [Students, Medical] explode all trees 525 #8. “Students, Nursing” or “medical student” or “students” or “student” 14,121 #9. #6 OR #7 OR #8 14,121 #10. #5 AND #9 1528 → Trials 1202

e14

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15

References Ayres, H.W., 2005. Factors related to motivation to learn and motivation to transfer learning in a nursing population. Unpublished doctoral dissertation. North Carolina State University, Raleigh. Barrows, H.S., 1993. An overview of the uses of standardized patients for teaching and evaluating clinical skills. Acad. Med. 68, 443–453. Baumann, A., Rideout, E. (Eds.), 2001. Standardized patients as an educational resource. Transforming Nursing Education Through Problem-based Learning. Jones & Bartlett, Sudbury, MA, pp. 281–291. Becker, K.L., Rose, L.E., Berg, J.B., Park, H., Shatzer, J.H., 2006. The teaching effectiveness of standardized patients. J. Nurs. Educ. 45 (4), 103–111. Choi, S.J., Kwon, M.S., Kim, S.H., Jung, Y.S., Jo, G.Y., 2013. Effect of using standardized patients on nursing competence, communication skills, and learning satisfaction in health assessment. J. Korean Acad. Soc. Nurs. Edu. 19 (1), 97–105. http://dx.doi.org/ 10.5977/jkasne.2013.19.1.197. De Champlain, A.F., Clauser, B.E., Margolis, M.J., Klass, D.J., Nungester, R.J., 1998a. Assessing decision consistency with a sequentially administered large-scale standardized patient examination: a Monte Carlo investigation. Acad. Med. 73 (10 Suppl.), S75–S77. De Champlain, A.F., Macmillan, M.K., Margolis, M.J., King, A.M., Klas, D.J., 1998b. Do discrepancies in standardized patients' checklist recording affect case and examination mastery-level decisions? Acad. Med. 73 (10 Suppl.), S75–S77. De Champlain, A.F., MacMillan, M.K., Margolis, M.J., Klass, D.J., Nungester, R.J., Schimpfiauser, F., et al., 1999. Modeling the effects of security breaches on students' performances on a large-scale standardized patient examination. Acad. Med. 74 (10 Suppl.), S49–S51. Engel, C.E., 1997. Not just a method but a way of learning. In: Boud, D., Feletti, G. (Eds.), The Challenge of Problem Based Learning. Kogan Page, London. Eom, M.R., Kim, H.S., Kim, E.K., Seong, K.Y., 2010. Effects of teaching method using standardized patients on nursing competence in subcutaneous injection, self-directed learning readiness, and problem solving ability. J. Korean Acad. Nurs. 40 (2), 151–160. http://dx.doi.org/10.4040/jkan.2010.40.2.151. Gaba, D., 2004. The future vision of simulation in health care. Qual. Saf. Health Care 16 (Suppl. 1), i2–i10.

Guglielmino, L.M., 1997. Development of the self-directed learning readiness scale. Unpublised doctoral dissertation. University of Georgia, Athens. Heppner, P.P., Peterson, C.H., 1982. The development and implications of a personal problem-solvng inventory. J. Couns. Psychol. 29 (1), 66–75. Higgins, J.P., Green, S., 2008. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.0. The Cochrane Collaboration and John Wiley & Sons, England. Hyun, K.S., Kang, H.S., Kim, W.O., Park, S.H., Lee, J., Sok, S.H., 2009. Development of a multimedia learning DM diet education program using standardized patients and analysis of its effects on clinical competency and learning satisfaction for nursing students. J. Korean Acad. Nurs. 39 (2), 249–258. http://dx.doi.org/10.4040/jkan.2009.39.2.249. Jang, K.I., 2013. Development and effects of an oncolog nursing simulation program for nursing students. Unpublished doctoral dissertation. The catholic University, Seoul. Kelle, J.M., 1983. Motivaton design of instruction. Lawrence Erlbaum Associates, Montreal. Khadivzadeh, T., Erfanian, F., 2012. The effects of simulated patients and simulated gynecologic models on student anxiety in providing IUD services, Simul. Healthc. 7 (5), 282–287. http://dx.doi.org/10.1097/SIH.0b013e31826064b7. Kim, K.S., Kim, J.A., Kwon, S.H., Song, M.S., 2010. Development of the nursing process based performance measurement tool for medication management and blood transfusion. J. Korean Clin. Nurs. Res. 16 (1), 177–197. Kim, S.H., 2012. Effects of simulation-based practice using standardized patients for the care of women with postpartum hemorrhage on nursing student's clinical performance competence and critical thinking deposition. Korean Parent Child Health J. 15 (2), 71–79. Kim, J.H., Park, I.H., Shin, S.J., 2013a. Systematic review of Korean studies on simulation within nursing education. J. Korean Acad. Nurs. 19 (3), 307–319. http://dx.doi.org/ 10.5977/jkasne.2013.19.3.307. Kim, S.Y., Park, J.E., Lee, Y.H., Seo, H.J., Sheen, S.S., Hahn, S.H., et al., 2013b. Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising validity. J. Clin. Epidemiol. 66, 408–414. http://dx.doi.org/10.1016/j. jclinepi.2012.09.016. Ko, E., Kim, H.Y., 2014. Effects of multi-mode simulation learning on nursing students' critical thinking disposition, problem solving process, and clinical competence. Korean J. Adult. Nurs. 26 (1), 107–116. http://dx.doi.org/10.7475/kjan.2014.26.1.107. Krathwohl, D.R., 2002. A revision of Bloom's taxonomy: an overview. Theory Pract. 41 (4), 212–218. http://dx.doi.org/10.1207/s15430421tip4104_2.

P.-J. Oh et al. / Nurse Education Today 35 (2015) e6–e15 Kwak, K.H., 2013. Development and application of medication error preventive simulation education for nursing students. Unpublished doctoral dissertation, Seoul. Lapkin, S., Levett-Jones, T., Bellchambers, H., Fernandez, R., 2010. Effectiveness of patient simulation manikins in teaching clinical reasoning skills to undergraduate nursing students: a systematic review. Clin. Simul. Nurs. 6 (6), e207–e222. http://dx.doi. org/10.1016/j.ecns.2010.05.005. Lee, S.J., Roh, Y.S., Kim, J.O., Jang, K.I., Ryoo, E.N., Park, Y.M., 2010. Comparison of multimode simulation and SimMan simulation on evaluation of nursing care for patients with dyspnea. J. Korean Acad. Soc. Nurs. Edu. 16 (1), 51–60. Lee, S.E., 2011. Evaluation of the standardized patients (SP) managed instruction for a clinical maternity nursing course. J. Korean Acad. Soc. Nurs. Edu. 17 (1), 14–24. Lee, S.J., Park, Y.M., Noh, S.M., 2013. The effects of simulation training with hybrid model for nursing students on nursing performance ability and self confidence. Korean J. Adult. Nurs. 25 (1), 170–182. Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gotzsche, P.C., Ioannids, J.P.A., et al., 2009. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care intervention: explanation and elaboration. Ann. Intern. Med. 151 (4), W65–W94. http://dx.doi.org/10.1136/bmj.b2700. Luctkar-Flude, M., Wilson-Keates, B., Larocque, M., 2012. Evaluating high-fidelity human simulators and standardized patients in an undergraduate nursing health assessment course. Nurse Educ. Today 32, 448–452. http://dx.doi.org/10.1016/j.nedt.2011.04.011. May, W., Park, J.H., Lee, J.P., 2009. A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996–2005. Med. Teach. 31 (6), 487–492. http://dx.doi.org/10.1080/01421590802530898. Nestel, D., Clark, S., Tabak, D., Ashwell, V., Muir, E., Paraskevas, P., et al., 2010. Defining responsibilities of simulated patients in medical education. Simul. Healthc. 5 (3), 161–168. http://dx.doi.org/10.1097/SIH.0b013e3181de1cb6. Norman, J., 2012. Systematic review of the literature on simulation in nursing education. ABNF J. 23 (2), 24–28. O'Connor, F.W., Albert, M., Tomas, M.D., 1999. Incorporating standardized patients into a psychosocial nurse practitioner program. Arch. Psychiatr. Nurs. 13, 240–247. Oermann, M.H., Gaberson, K.B., 2009. Evaluation and Testing in Nursing Education. Spring Publishing Company, NY. Park, S.Y., Kwon, Y.R., 2012. The effect of using standardized patients in psychiatric nursing practical training for nursing college students. J. Korean Acad. Psychiatr. Ment. Health Nurs. 21 (1), 79–88. Ranchor, A.V., Fleer, J., Sanderman, R., Van der Ploeg, K.M., Coyne, J.C., Schroevers, M., 2012. Psychological interventions for cancer survivors and cancer patients in the

e15

palliative phase (Protocol). Cochrane Database Sys. Rev. 1, CD009511. http://dx.doi. org/10.1002/14651858.CD009511. RevMan 5.2.11http://tech.cochrane.org/revman/download. Robinson-Smith, G., Bradley, P.K., Meakim, C., 2009. Evaluating the use of standardized patients in undergraduate psychiatric nursing experience. Clin. Simul. Nurs. 5 (6), e203–e211. http://dx.doi.org/10.1016/j/ecns/2009.07.001. Seong, K.Y., 2008. Effect of practice nursing education: Using standardized patients on subcutaneous insulin injection. Unpublished master's. Eulji University, Seoul. Sok, S.H.R., Kang, H.Y., Kim, W.O., Hyn, K.S., Lee, J.A., Park, S.H., 2009. Effects and development of clinical competency evaluation using standardized patients among nursing students: based on abdominal surgical patients. Korean J. Adult Nurs. 21 (5), 468–476. Sherer, M., Maddux, J.E., 1982. The self-efficacy scale: construction and validation. Psychol. Rep. 5, 663–671. Stroud, S.D., Smith, C.A., Edlund, B.J., Erkel, E.A., 1999. Evaluating clinical decision-making skills of nurse practitioner students. Clin. Excell. Nurse Pract. 3, 230–237. Thomas, M.D., O'Connor, F.W., Albert, M.L., Boutain, D., Brandt, P.A., 2001. Case-based teaching and learning experiences. Issues Ment. Health Nurs. 22, 517–531. Weaver, A., 2011. High-fidelity patient simulation in nursing education: a integrative review. Nurs. Educ. Perspect. 32 (1), 37–40. http://dx.doi.org/10.5480/1536-502632.1.37. Woo, O.H., 2000. The effects of a PBL(Problem-based learning) on the problem solving process students by their meta-cognitive levels. Unpublished master's. Korea National University of Education, Cheongwon. Yuan, H.B., Williams, B.A., Fang, J.B., Ye, Q.H., 2012. A systematic review of selected evidence on improving knowledge and skills through high-fidelity simulation. Nurse Educ. Today 32 (3), 294–298. http://dx.doi.org/10.1016/j.nedt.2011.07.010. Yoo, M.S., 2000. Development of standardized patient managed instruction for a fundamentals of nursing course. Unpublised doctoral dissertation. Younsei University, Seoul. Yoo, M.S., 2001. The effectiveness of standardized patient managed instruction for a fundamental nursing course. J. Korean Acad. Soc. Nurs. Edu. 7 (1), 94–112. Yoo, M.S., Yoo, I.Y., Son, Y.J., 2002. The effectiveness of standardized patient managed instruction in teaching foley catheterization skills, communication skills and learning motivation. J. Korean Acad. Fundam. Nurs. 9 (1), 66–75. Yoon, J., 2004. Development of scale for critical thinking deposition: Focused on nursing. Unpublished doctoral. The Catholic University, Seoul.