Nurse Education Today 35 (2015) 706–711
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
Implementation and outcome evaluation of high-fidelity simulation scenarios to integrate cognitive and psychomotor skills for Korean nursing students Heejung Ahn a,1, Hyun-Young Kim b,⁎ a b
Medical Research Collaborating Center, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 110-744, Republic of Korea College of Nursing, Eulji University, 77 Gyoryong-ro, 771 Beon-gil, Jung-gu, Daejeon 301-746, Republic of Korea
a r t i c l e
i n f o
Article history: Accepted 23 January 2015 Keywords: Nursing students Nursing education Patient simulations Program evaluation
s u m m a r y Objectives: This study is involved in designing high-fidelity simulations reflecting the Korean nursing education environment. In addition, it evaluated the simulations by nursing students' learning outcomes and perceptions of the simulation design features. Design: A quantitative design was used in two separate phases. Settings and participants: For the first phase, five nursing experts participated in verifying the appropriateness of two simulation scenarios that reflected the intended learning objectives. For the second phase, 69 nursing students in the third year of a bachelor's degree at a nursing school participated in evaluating the simulations and were randomized according to their previous course grades. Methods: The first phase verified the two simulation scenarios using a questionnaire. The second phase evaluated students' perceptions of the simulation design, self-confidence, and critical thinking skills using a quasiexperimental post-test design. ANCOVA was used to compare the experimental and control groups, and correlation coefficient analysis was used to determine the correlation among them. Results: We created 2 simulation scenarios to integrate cognitive and psychomotor skills according to the learning objectives and clinical environment in Korea. The experimental group had significantly higher scores on selfconfidence in the first scenario. The positive correlations between perceptions of the simulation design features, self-confidence, and critical thinking skill scores were statistically significant. Conclusions: Students with a more positive perception of the design features of the simulations had better learning outcomes. Based on this result, simulations need to be designed and implemented with more differentiation in order to be perceived more appropriately by students. © 2015 Elsevier Ltd. All rights reserved.
Introduction Nursing education puts emphasis on knowledge, skills, and attitudes. Nursing educators are responsible for improving nursing education while addressing changes in the medical environment, which stresses patient safety and quality of care (Brady, 2011; Handwerker, 2012). Because competent nurses are skilled at properly dealing with unexpected risks to patients (Whyte et al., 2012), nursing students must develop the abilities to incorporate cognitive skills, skilled practical knowledge, and ethical awareness into clinical practice during their professional nursing apprenticeships. The clinical practicum at clinical sites is intended to provide care to real patients and is a critical element in nursing education methods, as
⁎ Corresponding author. Tel.: +82 42 259 1716; fax: +82 42 259 1709. E-mail addresses: [email protected] (H. Ahn), fl[email protected] (H.-Y. Kim). 1 Tel.: +82 2 740 8934; fax: +82 2 766 8172.
http://dx.doi.org/10.1016/j.nedt.2015.01.021 0260-6917/© 2015 Elsevier Ltd. All rights reserved.
it is the process by which knowledge, skills, and attitudes are integrated. However, the limited number of clinical sites is a significant issue to be resolved in nursing education (Landeen and Nielson, 2008), particularly in Korean nursing education. The reason is that while many new nursing colleges have been established recently in Korea, the number of hospitals offering clinical education has not changed, so the opportunity for each student to participate in clinical practice has steadily decreased. Simulation-based learning is learner-centered and is based on constructivism to stimulate lively discussions on clinical practice. Students maximize the effects of their learning by experiencing mistakes and self-reflection on their own actions in a non-threatening environment (Benner et al., 2010; Handwerker, 2012). Prior studies showed that simulation-based learning resulted in nursing students' improvement in clinical judgment, self-efficacy, clinical abilities, and self-confidence. Furthermore, studies demonstrate that simulation can potentially resolve the issue of the limited number of clinical sites for clinical practice (Akhu-Zaheya et al., 2013; Khalaila, 2014; Weatherspoon and Wyatt, 2012).
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Two factors can be considered in the evaluation of simulation-based learning: The first factor that can be examined is learning outcomes that show how effective the simulation was for students, such as critical thinking with regard to the ability to integrate cognitive and psychomotor skills and self-confidence. The other factor is the simulation activity itself (Adamson and Rojers, 2012; Wilson and Klein, 2012). In simulation learning, higher student satisfaction results in better learning outcomes, and the design features of a simulation influence its learning outcomes (Smith and Roehrs, 2009), thus, that students' evaluation results reflect their education is essential. This study is a follow-up to a study determining the learning topics and objectives for simulation-based learning aiming to combine nursing knowledge and clinical skills in Korean nursing education (Kim and Min, 2013). In this study, we designed a simulation incorporating simulation design characteristics (Jeffries and Rogers, 2012), and examined its learning effects. We also analyzed the correlation between students' perception of the high-fidelity simulation design and their learning outcomes, following suggestions about the need to analyze the effect of students' experience with simulation in their learning outcomes (Khalaila, 2014; Zulkosky, 2012). Background
Study Aim The ultimate aim of this study was to implement two high-fidelity simulations in order to help nursing students integrate their cognitive and psychomotor skills. We first verified the appropriateness of simulation scenarios reflecting intended learning objectives for the third year nursing students pursuing a bachelor's degree. In addition, we intended to evaluate the students' simulation experience using the SDS and learning outcomes (self-confidence and critical thinking skills) by applying the simulation scenarios developed in this study. We also planned to analyze the correlation between students' simulation experience and these learning outcomes. Methods Study Design This study used a quantitative design in two separate phases. The first phase was to examine the validity of two scenarios developed in this study for designing a high-fidelity simulation. The second phase had a quasi-experimental, randomized comparison group using a post-test design in order to evaluate students' perceptions of simulation design and learning outcomes (Fig. 1). Sample characteristics To examine the validity of the scenarios in the first phase, convenience sampling was used. Five nursing experts from different hospitals, each with over ten years of clinical expertise in the adult nursing field, took part in this study. Using prior studies as a reference (Kerr et al., 2013; Zulkosky, 2012), with an effect size of 0.70, power of .80, and significance level (α) of .05, we needed 34 subjects in each group for the second phase. Convenience Randomization by course grades of the previous semester
Experimental group
Control group
Pre-test measurement General characteristics Self-confidence in 20 nursing skills Critical thinking skills Antibiotic preparation for a patient with newly diagnosed pneumonia
Group 1 (n=35) Simulation activity and debriefing
Tracheostomy suction to prevent increased intracranial pressure
The basic principal of simulation-based learning is to be studentcentered and outcome-focused. Students construct meaning from what they do for their own learning, and educators plan learning activities according to learning outcomes. Previous research shows that educators need to share learning objectives with students to maximize the effect of simulation-based learning, and they should determine simulation scenarios suitable to the students' level and learning objectives (Akhu-Zaheya et al., 2013; Handwerker, 2012). Simulation design features includes objectives/information, support/ cues, complexity of problem-solving, guided reflection/debriefing, and fidelity to reality (Jeffries and Rogers, 2012). In addition, a scenario includes events that are triggered to solve a nursing problem according to the learning objectives of the level of previous knowledge and skill of the learner. The designer documents decisions and concerns related as to how to trigger problem-solving that would need to be answered within the case scenario (Waxman, 2010; Wilson and Klein, 2012). Therefore, examination of the content validity of the scenarios described is crucial. Expert examination on whether the scenarios are accurate and valid for learning objectives needs to be done, and revisions according to the feedback need to be made as necessary (Cioffi, 2001; Waxman, 2010). One of the core elements for consideration when designing a simulation is the level of student support by educators with consideration of students' levels. Student support is given by means of the cues provided during the simulation as well as facilitation of guided reflection on their simulation activity during debriefing (Dubose et al., 2010). During the simulation experience, students and educators can interact with each other in various ways including student-driven, partially instructor-driven, or instructor-driven types of simulations, depending on the students' experience with classroom learning, clinical practice, and simulation-based learning (Dubose et al., 2010). Prior studies have suggested the need for additional research on how different design types and experiences change learning outcomes, reflecting such features (Khalaila, 2014; Zulkosky, 2012). In prior research, evaluations of learning outcomes have been common, while evaluations of the students' perception are rare. Therefore, we need to evaluate the simulation activity itself in simulation-based education and analyze the correlation between the quality of education and learning outcomes. One of the tools for evaluating a simulation activity itself is the Simulation Design Scale (SDS) developed by the National League for Nursing (NLN) based on Jeffries' Simulation Framework. It is a useful tool for evaluating students' perception of simulation design (Adamson and Rojers, 2012; Wilson and Klein, 2012).
707
Group 2 (n=34) Simulation activity and debriefing
Group 2 (n=34) Lecture and case studies
Measurement Self-confidence in scenario 1 Critical thinking skills Perception of simulation design (SDS)
Group 1 (n=35) Lecture and case studies
Measurement Self-confidence in scenario 2 Critical thinking skills Perception of simulation design (SDS)
Fig. 1. Process of the second phase.
708
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
sampling was used and we recruited students who wished to join the simulation, and 72 students in the third year of a bachelor's degree at a nursing school volunteered. We then randomly assigned the subjects to the experimental group (36 subjects) and control group (36 subjects) based on the students' grades in the adult health nursing and fundamentals of nursing courses from the previous semester. During the study, a total of three subjects withdrew (one from the experimental group and two from the control group), and the analysis was done on the remaining 69 subjects. Variables and Measurements Validity of scenarios. We used a 5-point Likert scale for verifying the appropriateness of the scenarios constructed based on clinical cases. Responses were scored on a scale from 1 (strongly disagree) to 5 (strongly agree). General characteristics of students. We collected data from the study participants, including demographic data such as sex and age, whether they had experience with high-fidelity simulation, and their grades in the previous semester. We also measured their self-confidence in the 20 core fundamental nursing skills as defined by the Korean Accreditation Board of Nursing (Korean Accrediation Board of Nursing Education, 2012). Perceptions of the simulation design. The Simulation Design Scale (SDS) was used to evaluate the perceptions of simulation design. The SDS includes 20 questions for assessing the objectives, problem-solving, student support (cues), guided reflection/debriefing, and fidelity to reality using a 5-point Likert scale. The estimate of the reliability was Cronbach's α = .94 at the time of the development of the scale, and Cronbach's α = .87 in this study. In addition, we asked the subjects to rate how appropriate the learning objectives were for third-year students' level. A 5-point Likert scale was used for the variable, and responses ranged from 1 (very inappropriate) to 5 (very appropriate). Self-confidence. Items for the questionnaire were constructed for measuring students' self-confidence with the behaviors the students were expected to perform during the simulation activities. There were two kinds of self-confidence questionnaires according to the intended learning objectives in each scenario. The content validity of the selfconfidence questionnaires on scenarios 1 and 2 were 8.9 and 9.1, respectively. Self-confidence was measured after each of the two scenarios was implemented. A 5-point Likert scale was used, and the responses ranged from 1 (strongly disagree) to 5 (strongly agree). Critical thinking skills. To gauge critical thinking skills, we used a tool developed to measure Korean nursing students' critical thinking tendency (Kwon et al., 2006). It is composed of 35 items in total, each measured on a 5-point scale. Critical thinking skills were measured three times (pre-test, after each of the two scenario implementations). Higher scores mean stronger critical thinking skills. The estimate of the reliability was Cronbach's α = .80 at the time of developing the scale, and Cronbach's α = .81 in this study.
antibiotic solution for AST (after the skin test)’. The main objectives for scenario 2 were ‘Explain the signs and symptoms of increased intracranial pressure’, ‘Evaluate the risk of developing increased intracranial pressure’, and ‘Demonstrate a tracheal suction while preventing increased intracranial pressure’. The two scenarios' learning objectives are shown in Appendices A and B. Problem-solving. In the two scenarios, we presented events that would trigger students' nursing care actions as one or two problem-solving elements, which can be performed in the 15 min assigned for simulation activities. Students' support (cues). Forty minutes was assigned for pre-briefing to share learning outcomes and to establish the nursing care plans. During this phase, students were to find cues for problem-solving and establish their nursing plans based on the information in the scenarios. The simulation activities were designed as partially instructor-driven type simulations. In other words, the subjects entered the simulation room and performed their plan without any interruption first. If the students needed additional directions on care planning beyond that established during the pre-briefing, an instructor could give verbal cues in order to help the students achieve their learning outcomes (Dubose et al., 2010). Guided reflection/debriefing. After the simulation activities were completed, debriefing was performed for 35 min by the group. If the students were not able to accurately recall their activities, video recordings were used to help the students reflect on their actions. Fidelity. We developed scenarios containing physiological parameters and verbal responses that can be applied to high-fidelity simulators (METI 3G) according to the learning objectives we set. Control group. The control group received an education through lectures and case conferences for an hour and a half, using the same scenarios as in the high-fidelity simulation education. Data analyses Data analyses were performed with IBM SPSS Statistics for Windows version 21.0 (IBM Corp., Armonk, NY, USA). In the first phase, we used means and standard deviations to examine the validity of the scenarios developed in this study. In the second phase, we conducted a t-test and χ2 test for testing the homogeneity of the study participants, and ANCOVA to examine the differences in self-confidence and critical thinking skills between the experimental and control groups. Finally, we analyzed the correlation between the perception of simulation design, self-confidence and critical thinking skills using Pearson's correlation coefficient. All P-values are two-sided and P b .05 was considered statistically significant.
Design of High-fidelity Simulation
Ethical considerations
Objectives. Before developing the scenarios, we set detailed learning objectives for the simulation learning topics that had been chosen by reflecting on the curriculum of the first and second semesters in the third year (Kim and Min, 2013). On the basis of previous studies, we determined four to five primary learning objectives for each scenario (Waxman, 2010). The main objectives for scenario 1 were ‘Explain the signs and symptoms of CAP’, ‘Establish the prioritization of the nursing intervention in which the patient has a high fever’, and ‘Prepare the
The study procedure of this research project was approved by the Institutional Review Board in our university. All the information was handled anonymously and confidentially, and was never used for any other purpose. We explained to the study participants the goal of the study, the fact that they were free to withdraw from the study at any stage, and that they would not be at a disadvantage if they did not participate in the study. We proceeded with the study after receiving informed consent for study participation and video recording from the participants.
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711 Table 1 The validity of scenario 1: antibiotic preparation for a patient with newly diagnosed pneumonia (Appendix A), and scenario 2: tracheostomy suction to prevent increased intracranial pressure (Appendix B). Variables evaluated
Scenario 1
Scenario 2
Mean ± SD
Mean ± SD
Situations in the scenario occur frequently in clinical practice. The physiological characteristics such as the patient's vital signs properly reflect a real clinical situation. The treatment process such as the doctor's orders properly reflects a real clinical situation. The students' behaviors are appropriate for a clinical nursing plan. The students' actions indicate that the student is able to perform the required duties.
5.00 ± 0.00 4.60 ± 0.55 4.75 ± 0.50 4.17 ± 0.41 4.75 ± 0.50 4.17 ± 0.41
709
Table 3 Mean and standard deviation for students' perception of the simulation design and learning objectives (n = 69). Variables
Simulation design
Objectives Problem-solving Student support Guided reflection Fidelity Total Overall appropriateness of the learning objectives
Scenario 1
Scenario 2
Mean ± SD
Mean ± SD
3.59 ± 0.60 3.33 ± 0.50 3.54 ± 0.61 4.35 ± 0.54 3.88 ± 0.85 3.69 ± 0.44 4.24 ± 0.63
3.40 ± 0.76 3.29 ± 0.62 3.16 ± 0.66 4.06 ± 0.92 3.56 ± 0.97 3.47 ± 0.62 4.43 ± 0.55
4.50 ± 0.58 4.33 ± 0.52 5.00 ± 0.00 4.83 ± 0.41
Results
On the other hand, no statistically significant difference was found between the experimental and control groups in critical thinking skills in scenario 1. The experimental group (group 2) demonstrated significantly higher critical thinking skills after the scenario 2 simulation than the control group (group 1) did; however, the actual mean difference was found not to be significant (Table 5).
Validity of Scenarios The clinical validity of each scenario (Appendices A and B) was tested by evaluating the variables in Tables 1. In particular, the test of whether the situations in the scenario occurred frequently in clinical practice and whether the students' actions indicated that the student was able to perform the required duties had high average scores in both scenarios (Tables 1).
Correlation Between the Perceptions of the Simulation Design and the Learning Outcomes A significant weak positive correlation between the perceptions of the simulation design and self-confidence as well as critical thinking skills was found in the experimental group (Table 6). Discussion
Socio-demographic Characteristics of the Nursing Students No statistically significant difference was found between the two groups for the students' grades in the previous semester or selfconfidence in the 20 core fundamental skills. Moreover, all students had experience with a simulation lecture. Thus, the two groups can be considered homogeneous (Table 2). Students' Perception of the Simulation Design and Learning Objectives Using the SDS, five factors incorporated into the simulation design have been represented in Table 3. For the design of both scenarios, the students rated guided reflection and fidelity highly. In addition, most students agreed or strongly agreed that the learning objectives were appropriate for third-year students considering the required curriculum of the simulation class. Self-confidence and Critical Thinking Skills The experimental group (group 1) demonstrated significantly higher self-confidence than the control group (group 2) did in scenario 1. However, no significant difference was found in the level of selfconfidence after the scenario 2 simulation between the experimental group (group 2) and control group (group 1) (Table 4).
In this study, two high-fidelity simulations based on the learning objectives representative of a third-year nursing student's curriculum (from a 4-year nursing college) were evaluated. Importantly, learning objectives is one of the characteristics of simulation design (Jeffries and Rogers, 2012), and clear objectives should be integrated within the curriculum because the objectives guide all of the aspects of simulation design (Arthur et al., 2013). In this study, high-fidelity simulation design features constructed to fit specific learning objectives demonstrated significant positive correlations with certain learning outcomes. Therefore, improved performance in course material may be achievable with the addition of rigorous simulation design. Learning objectives need to be based on students' prior knowledge and skills, include intended outcomes and expected behaviors, and be presented specifically enough for students to effectively participate in simulation activities (Jeffries and Rogers, 2012). Simulation-based education on a human patient may limit the student's ability to achieve all of their learning objectives due to technical issues and potential complexities in working with a human simulator. Therefore, scenarios need to be clearly connected with the objectives of the curriculum (Bremner et al., 2006), and educators need to choose or develop scenarios suitable for the students' level with consideration of the fidelity and problem-solving elements of the simulation design (Akhu-Zaheya et al., 2013; Jeffries and Rogers, 2012). To enhance the integrity of a
Table 2 Demographic characteristics of participants in the second phase. Characteristics
Age (years) Previous semester scores from adult health nursing Previous semester scores from fundamentals of nursing Self-confidence in the 20 core fundamental nursing skills Critical thinking Gender Female Male Participated in the simulation lecture Yes No
Group 1 (n = 35)
Group 2 (n = 34)
N (%) or Mean ± SD
N (%) or Mean ± SD
20.13 ± 1.24 84.75 ± 8.27 87.03 ± 4.58 2.92 ± 0.64 3.22 ± 0.68 32 (91.4) 3 (8.6) 32 (100) 0 (0)
20.81 ± 2.65 84.28 ± 8.60 86.69 ± 5.18 3.18 ± 0.49 3.36 ± 0.37 32 (94.1) 2 (5.9) 32 (100) 0 (0)
χ2 or t
p-value
−1.332 −0.222 0.281 −1.834 −1.085 .186
.190 .825 .779 .071 .282 .327
710
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Table 4 Mean and difference of self-confidence (n = 69).
Scenario 1 Scenario 2
Group 1
Group 2
Mean ± SD
Mean ± SD
Experimental group 4.05 ± 0.48 Control group 3.37 ± 0.41
Control group 3.86 ± 0.53 Experimental group 3.56 ± 0.34
Fa
p-Value
Table 6 Correlations between the perceptions of the simulation design and the two learning outcomes (n = 69). Perceptions of simulation design Self confidence
4.674
.034
2.908
.093
Critical thinking skills
Pearson r p-Value Pearson r p-Value
.252 .038 .315 .021
a ANCOVA, covariate = self-confidence in the 20 core fundamental nursing skills as defined by the Korean Accreditation Board of Nursing at pre-test.
simulation, scenario developers must have relevant clinical expertise to describe the course of events and intervention responses according to learning objectives accurately (Waxman, 2010). The scenarios used in this study were developed to facilitate students' learning on clinical cases that frequently occur in Korean university hospitals, and, in fact, both scenarios properly replicated clinical situations according to the experts. On the other hand, the students rated problem-solving of simulation design from 3.29 to 3.33 points; thus, they were not as satisfied with their ability to solve problems in the simulation. Two previous studies reported similar findings when nursing students at a 4-year college performed poorly or were not able to achieve expected results in a problem-solving and decision-making process after undergoing basic assessments in cases about pulmonary embolism and hypovolemia (Cooper et al., 2010; Fero et al., 2010). In these studies, students were thought to already have the required knowledge but failed to apply it to the given situations thus, educators should support them by using obvious cues during simulation activity. Educators who are planning simulations should consider when and how they can support their students before the simulation (Jeffries and Rogers, 2012). In this study, a partial-instructor driven type simulation was used to provide some student support (Dubose et al., 2010). To guarantee that students actively participated in the simulation, problem-solving was initiated using cues provided in the proctor's scripts. For example, scenario 1 tested the student's ability to decide on the order of tasks in administering antibiotics and antipyretics and performing a blood culture in a patient with a newly developed high fever. In addition, the students had to provide the rationale for their decisions, which tested their problem-solving skills. Additional information was provided to the students by an instructor when decisions or judgments during the simulation activities were questionable. However, the students rated the level of support that was provided between 3.16 and 3.54, which may mean that the students had hoped for more support during the simulation. Our findings are similar to the results of a previous study with a similar study design. They found that fourth-year nursing students in Korea perceived a very low level of student support being provided while they conducted an emergency care management simulation (Hur et al., 2013). In addition, newly graduated nurses perceived a similar level of student support as in our study while completing a simulation (Wilson and Klein, 2012). The student felt supported by the instructor's assistance during the simulation activity, but when cues were delivered, they tended to be reported inconsistently and often lacked clarity (Jeffries, 2005). Considering that the results of previous studies and this study found that student support scored
Table 5 Mean and difference of critical thinking skills (n = 69).
Scenario 1 Scenario 2 a
Group 1
Group 2
Mean ± SD
Mean ± SD
Experimental group 3.31 ± 0.31 Control group 3.37 ± 0.35
Control group 3.34 ± 0.27 Experimental group 3.27 ± 0.33
ANCOVA, covariate = critical thinking skills at pre-test.
lower than other factors of the simulation design did, efficient support should be provided to third- and fourth-year nursing students during simulation activities. In simulation-based education, a guided reflection process after the simulation is important so students can thoroughly reconstruct their previous cognitive framework since reflection during simulation is typically impossible due to time constraints (Bland et al., 2011). In particular, video recording can be an effective means of reflection because subjects can easily remember their actions/decisions and evaluate them objectively (Chronister and Brown, 2012). Guided reflection may have been rated the highest in our study because students were fully able to understand their experiences using a video-assisted verbal debriefing. We randomized students to obtain unbiased results (Adler et al., 2007; Khalaila, 2014), in order to analyze whether a correlation existed between their perception of the simulation design and learning outcomes. The experimental group had significantly higher scores than the control group did for measures of self-confidence such as being able to establish a nursing care plan for a patient with high fever in scenario 1. For the simulation in scenario 2 to evaluate a client's risk of developing increased intracranial pressure, while there was no significant statistical difference in self-confidence between the experimental and control groups, the mean score of the experimental group was much higher than the control group. There has been controversy over whether simulation education can be expected to increase the self-confidence of nursing students. The results of other studies show that no significant changes in self-confidence were found after simulating on a hemodynamic assessment (Liaw et al., 2012). On the other hand, nursing students' confidence improved significantly in emergency care, which was considered a relatively difficult task (Hur and Park, 2012). Considering these results, researchers might have to consider the level of difficulty or significance of the tasks being tested in the simulation when the students realize their mistakes during high-fidelity simulation (McCaughey and Traynor, 2010). We have discussed the significance of how students' perceptions of a simulation design contribute to their learning experience and its correlation with self-confidence and critical thinking skills as learning outcomes. The most significant implication of this study is that there was a significant, positive correlation between the students' perceived appropriateness of the simulation design and the learning outcomes. This agrees with the result of a study that showed a significantly positive correlation between perceived appropriateness and post-education self-confidence in an emergency care management simulation (Hur et al., 2013) and verifies the need for more rigorous study designs based on the recommendations of future studies (Khalaila, 2014; Zulkosky, 2012). Study Limitations
Fa
p-Value
0.001
.971
4.093
.047
Two limitations of this study were the use of convenience sampling as the sampling method and the cumulative effect of simulation education that could not be evaluated. Students' self-confidence and nursing performance are expected to improve gradually with more frequent exposure to simulations, so more notable effects should be detected with the application of the method for a greater period of time (Khalaila, 2014).
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Conclusions and Implications In this study, we selected topics relevant to third-year nursing students in a 4-year nursing program while taking into account the clinical practicum curriculum of the nursing program. After the students completed their simulations, we examined the correlation between the components of the simulation design and the student's educational outcomes. A significant, positive correlation between the simulation design and learning outcomes was revealed. In particular, nursing students' self-confidence improved after practicing problem-solving tasks based on the learning objectives in a simulation scenario. More research is needed to decide the most appropriate type of simulation design suitable for a student's level toward the development of a more effective simulation program. Acknowledgment This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MEST) (Grant number: 2012-2012R1A1A1007988). Appendix A. Supplementary data Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.nedt.2015.01.021. References Adamson, K.A., Rojers, K.J., 2012. Evaluation: a critical step in simulation practice and research. In: Jeffries, P.R. (Ed.), Simulation in Nursing Education: From Conceptualization to Evaluation. National League for Nursing, New York, pp. 131–161. Adler, M.D., Trainor, J.L., Siddall, V.J., McGaghie, W.C., 2007. Development and evaluation of high-fidelity simulation case scenarios for pediatric resident education. Ambul. Pediatr. 7 (2), 182–186. Akhu-Zaheya, L.M., Gharaibeh, M.K., Alostaz, Z.M., 2013. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin. Simul. Nurs. 9 (9), e335–e342. Arthur, C., Levett-Jones, T., Kable, A., 2013. Quality indicators for the design and implementation of simulation experiences: a Delphi study. Nurse Educ. Today 33 (11), 1357–1361. http://dx.doi.org/10.1016/j.nedt.2012.07.012. Benner, P., Sutphen, M., Leonard, V., Day, L., 2010. Connecting Classroom and Clinical Through Integrative Teaching and Learning Educating Nurses. A call for Radical Transformation. Jossey-Bass, Stanford, CA. Bland, A.J., Topping, A., Wood, B., 2011. A concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Educ. Today 31 (7), 664–670. http://dx.doi.org/10.1016/j.nedt.2010.10.013. Brady, D.S., 2011. Using quality and safety education for nurses (QSEN) as a pedagogical structure for course redesign and content. Int. J. Nurs. Educ. Scholarsh. 8 (1). http:// dx.doi.org/10.2202/1548-923X.2147 (Article 5). Bremner, M.N., Aduddell, K., Bennett, D.N., VanGeest, J.B., 2006. The use of human patient simulators: best practice with novice nursing students. Nurse Educ. 31 (4), 170–174. Chronister, C., Brown, D., 2012. Comparison of simulation debriefing methods. Clin. Simul. Nurs. 8 (7), e281–e288. http://dx.doi.org/10.1016/j.ecns.2010.12.005. Cioffi, J., 2001. Clinical simulations: development and validation. Nurse Educ. Today 21, 477–486. Cooper, S., Kinsman, L., Buykx, P., McConnell-Henry, T., Endacott, R., Scholes, J., 2010. Managing the deteriorating patient in a simulated environment: nursing students' knowledge, skill and situation awareness. J. Clin. Nurses 19 (15–16), 2309–2318. http://dx. doi.org/10.1111/j.1365-2702.2009.03164.x.
711
Dubose, D., Sellinger-Karmel, L.D., Scoloveno, R.L., 2010. Baccalaureate nursing education. In: Nehring, M., Lashley, R. (Eds.), High-fidelity Patient Simulation in Nursing Education. Jones and Bartlett Publishers, MA, pp. 189–203. Fero, L.J., O'Donnell, J.M., Zullo, T.G., Dabbs, A.D., Kitutu, J., Samosky, J.T., Hoffman, L.A., 2010. Critical thinking skills in nursing students: comparison of simulation-based performance with metrics. J. Adv. Nurs. 66 (10), 2182–2193. http://dx.doi.org/10. 1111/j.1365-2648.2010.05385.x. Handwerker, S.M., 2012. Transforming nursing education: a review of current curricular practices in relation to Benner's latest work. Int. J. Nurs. Educ. Scholarsh. 9 (1). http://dx.doi.org/10.1515/1548-923X.2510 (Article 21). Hur, H.K., Park, S., 2012. Effects of simulation based education, for emergency care of patients with dyspnea, on knowledge and performance confidence of nursing students. J. Korean Acad. Soc. Nurs. Educ. 18 (1), 110–118. http://dx.doi.org/10.5977/jkasne. 2012.18.1.110. Hur, H.K., Park, S., Shin, Y.H., Lim, Y.M., Kim, G., Kim, K.K., Choi, H.O., Choi, J.H., 2013. Development and applicability evaluation of an emergent care management simulation practicum for nursing students. J. Korean Acad. Soc. Nurs. Educ. 19 (2), 228–240. http://dx.doi.org/10.5977/jkasne.2013.19.2.228. Jeffries, P.R., 2005. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs. Educ. Perspect. 26 (2), 96–103. Jeffries, P.R., Rogers, K.J., 2012. Theoretical framework for simulation design. In: Jeffries, P.R. (Ed.), Simulation in Nursing Education. National League for Nursing, New York, pp. 25–44. Kerr, B., Hawkins, T.L., Herman, R., Barnes, S., Kaufmann, S., Fraser, K., Ma, I.W., 2013. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med. Educ. Online 18 (18), 21312. http://dx.doi.org/10.3402/meo.v18i0.21312. Khalaila, R., 2014. Simulation in nursing education: an evaluation of students' outcomes at their first clinical practice combined with simulations. Nurse Educ. Today 34, 252–258. http://dx.doi.org/10.1016/j.nedt.2013.08.015. Kim, H., Min, Y.H., 2013. Development of curriculum and scenarios using constructive alignment theory for simulation-based education for nursing students to enhance clinical skills and nursing knowledge. Paper Presented at the International Proceedings of Economics Development & Research, Rome. Korean Accrediation Board of Nursing Education, 2012. Core Fundamental Nursing Skill for Nursing Education Accreditation. (Retrieved 04.30, 2013, from http://www. kabon.or.kr/HyAdmin/upload/goodFile/120120319112302.pdf.). Kwon, I.S., Lee, G.E., Kim, G.D., Kim, Y.H., Park, K.M., Park, H.S., Sohn, S.K., Lee, W.S., Jang, K.S., Chung, B.Y., 2006. Development of a critical thinking disposition scale for nursing students. J. Korean Acad. Nurs. 36 (6), 950–958. Landeen, L., Nielson, A., 2008. Focus on simulation-integrating simulation into teaching practice. J. Nurs. Educ. 47 (11), 487–488. Liaw, S.Y., Scherpbier, A., Rethans, J.J., Klainin-Yobas, P., 2012. Assessment for simulation learning outcomes: a comparison of knowledge and self-reported confidence with observed clinical performance. Nurse Educ. Today 32 (6), e35–e39. http://dx.doi. org/10.1016/j.nedt.2011.10.006. McCaughey, C.S., Traynor, M.K., 2010. The role of simulation in nurse education. Nurse Educ. Today 30 (8), 827–832. http://dx.doi.org/10.1016/j.nedt.2010.03.005. Smith, S.J., Roehrs, C.J., 2009. High-fidelity simulation: factors correlated with nursing student satisfaction and self-confidence. Nurs. Educ. Perspect. 30 (2), 74–78. Waxman, K.T., 2010. The development of evidence-based clinical simulation scenarios; guidelines for nurse educators. J. Nurs. Educ. 49 (1), 29–35. http://dx.doi.org/10. 3928/01484834-20090916-07. Weatherspoon, D.L., Wyatt, T.H., 2012. Testing computer-based simulation to enhance clinical judgment skills in senior nursing students. Nurs. Clin. North Am. 47 (4), 481–491. http://dx.doi.org/10.1016/j.cnur.2012.07.002. Whyte IV, J., Ward, P., Eccles, D.W., Harris, K.R., Nandagopal, K., Torof, J.M., 2012. Nurses' immediate response to the fall of a hospitalized patient: a comparison of actions and cognitions of experienced and novice nurses. Int. J. Nurs. Stud. 49 (9), 1054–1063. http://dx.doi.org/10.1016/j.ijnurstu.2012.03.005. Wilson, R.D., Klein, J.D., 2012. Design, implementation and evaluation of a nursing simulation: a design and development research study. J. Appl. Instr. Des. 2 (1), 57–68. Zulkosky, K.D., 2012. Simulation use in the classroom: impact on knowledge acquisition, satisfaction, and self-confidence. Clin. Simul. Nurs. 8 (1), e25–e33. http://dx.doi.org/ 10.1016/j.ecns.2010.06.003.
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
Implementation and outcome evaluation of high-fidelity simulation scenarios to integrate cognitive and psychomotor skills for Korean nursing students Heejung Ahn a,1, Hyun-Young Kim b,⁎ a b
Medical Research Collaborating Center, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 110-744, Republic of Korea College of Nursing, Eulji University, 77 Gyoryong-ro, 771 Beon-gil, Jung-gu, Daejeon 301-746, Republic of Korea
a r t i c l e
i n f o
Article history: Accepted 23 January 2015 Keywords: Nursing students Nursing education Patient simulations Program evaluation
s u m m a r y Objectives: This study is involved in designing high-fidelity simulations reflecting the Korean nursing education environment. In addition, it evaluated the simulations by nursing students' learning outcomes and perceptions of the simulation design features. Design: A quantitative design was used in two separate phases. Settings and participants: For the first phase, five nursing experts participated in verifying the appropriateness of two simulation scenarios that reflected the intended learning objectives. For the second phase, 69 nursing students in the third year of a bachelor's degree at a nursing school participated in evaluating the simulations and were randomized according to their previous course grades. Methods: The first phase verified the two simulation scenarios using a questionnaire. The second phase evaluated students' perceptions of the simulation design, self-confidence, and critical thinking skills using a quasiexperimental post-test design. ANCOVA was used to compare the experimental and control groups, and correlation coefficient analysis was used to determine the correlation among them. Results: We created 2 simulation scenarios to integrate cognitive and psychomotor skills according to the learning objectives and clinical environment in Korea. The experimental group had significantly higher scores on selfconfidence in the first scenario. The positive correlations between perceptions of the simulation design features, self-confidence, and critical thinking skill scores were statistically significant. Conclusions: Students with a more positive perception of the design features of the simulations had better learning outcomes. Based on this result, simulations need to be designed and implemented with more differentiation in order to be perceived more appropriately by students. © 2015 Elsevier Ltd. All rights reserved.
Introduction Nursing education puts emphasis on knowledge, skills, and attitudes. Nursing educators are responsible for improving nursing education while addressing changes in the medical environment, which stresses patient safety and quality of care (Brady, 2011; Handwerker, 2012). Because competent nurses are skilled at properly dealing with unexpected risks to patients (Whyte et al., 2012), nursing students must develop the abilities to incorporate cognitive skills, skilled practical knowledge, and ethical awareness into clinical practice during their professional nursing apprenticeships. The clinical practicum at clinical sites is intended to provide care to real patients and is a critical element in nursing education methods, as
⁎ Corresponding author. Tel.: +82 42 259 1716; fax: +82 42 259 1709. E-mail addresses: [email protected] (H. Ahn), fl[email protected] (H.-Y. Kim). 1 Tel.: +82 2 740 8934; fax: +82 2 766 8172.
http://dx.doi.org/10.1016/j.nedt.2015.01.021 0260-6917/© 2015 Elsevier Ltd. All rights reserved.
it is the process by which knowledge, skills, and attitudes are integrated. However, the limited number of clinical sites is a significant issue to be resolved in nursing education (Landeen and Nielson, 2008), particularly in Korean nursing education. The reason is that while many new nursing colleges have been established recently in Korea, the number of hospitals offering clinical education has not changed, so the opportunity for each student to participate in clinical practice has steadily decreased. Simulation-based learning is learner-centered and is based on constructivism to stimulate lively discussions on clinical practice. Students maximize the effects of their learning by experiencing mistakes and self-reflection on their own actions in a non-threatening environment (Benner et al., 2010; Handwerker, 2012). Prior studies showed that simulation-based learning resulted in nursing students' improvement in clinical judgment, self-efficacy, clinical abilities, and self-confidence. Furthermore, studies demonstrate that simulation can potentially resolve the issue of the limited number of clinical sites for clinical practice (Akhu-Zaheya et al., 2013; Khalaila, 2014; Weatherspoon and Wyatt, 2012).
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Two factors can be considered in the evaluation of simulation-based learning: The first factor that can be examined is learning outcomes that show how effective the simulation was for students, such as critical thinking with regard to the ability to integrate cognitive and psychomotor skills and self-confidence. The other factor is the simulation activity itself (Adamson and Rojers, 2012; Wilson and Klein, 2012). In simulation learning, higher student satisfaction results in better learning outcomes, and the design features of a simulation influence its learning outcomes (Smith and Roehrs, 2009), thus, that students' evaluation results reflect their education is essential. This study is a follow-up to a study determining the learning topics and objectives for simulation-based learning aiming to combine nursing knowledge and clinical skills in Korean nursing education (Kim and Min, 2013). In this study, we designed a simulation incorporating simulation design characteristics (Jeffries and Rogers, 2012), and examined its learning effects. We also analyzed the correlation between students' perception of the high-fidelity simulation design and their learning outcomes, following suggestions about the need to analyze the effect of students' experience with simulation in their learning outcomes (Khalaila, 2014; Zulkosky, 2012). Background
Study Aim The ultimate aim of this study was to implement two high-fidelity simulations in order to help nursing students integrate their cognitive and psychomotor skills. We first verified the appropriateness of simulation scenarios reflecting intended learning objectives for the third year nursing students pursuing a bachelor's degree. In addition, we intended to evaluate the students' simulation experience using the SDS and learning outcomes (self-confidence and critical thinking skills) by applying the simulation scenarios developed in this study. We also planned to analyze the correlation between students' simulation experience and these learning outcomes. Methods Study Design This study used a quantitative design in two separate phases. The first phase was to examine the validity of two scenarios developed in this study for designing a high-fidelity simulation. The second phase had a quasi-experimental, randomized comparison group using a post-test design in order to evaluate students' perceptions of simulation design and learning outcomes (Fig. 1). Sample characteristics To examine the validity of the scenarios in the first phase, convenience sampling was used. Five nursing experts from different hospitals, each with over ten years of clinical expertise in the adult nursing field, took part in this study. Using prior studies as a reference (Kerr et al., 2013; Zulkosky, 2012), with an effect size of 0.70, power of .80, and significance level (α) of .05, we needed 34 subjects in each group for the second phase. Convenience Randomization by course grades of the previous semester
Experimental group
Control group
Pre-test measurement General characteristics Self-confidence in 20 nursing skills Critical thinking skills Antibiotic preparation for a patient with newly diagnosed pneumonia
Group 1 (n=35) Simulation activity and debriefing
Tracheostomy suction to prevent increased intracranial pressure
The basic principal of simulation-based learning is to be studentcentered and outcome-focused. Students construct meaning from what they do for their own learning, and educators plan learning activities according to learning outcomes. Previous research shows that educators need to share learning objectives with students to maximize the effect of simulation-based learning, and they should determine simulation scenarios suitable to the students' level and learning objectives (Akhu-Zaheya et al., 2013; Handwerker, 2012). Simulation design features includes objectives/information, support/ cues, complexity of problem-solving, guided reflection/debriefing, and fidelity to reality (Jeffries and Rogers, 2012). In addition, a scenario includes events that are triggered to solve a nursing problem according to the learning objectives of the level of previous knowledge and skill of the learner. The designer documents decisions and concerns related as to how to trigger problem-solving that would need to be answered within the case scenario (Waxman, 2010; Wilson and Klein, 2012). Therefore, examination of the content validity of the scenarios described is crucial. Expert examination on whether the scenarios are accurate and valid for learning objectives needs to be done, and revisions according to the feedback need to be made as necessary (Cioffi, 2001; Waxman, 2010). One of the core elements for consideration when designing a simulation is the level of student support by educators with consideration of students' levels. Student support is given by means of the cues provided during the simulation as well as facilitation of guided reflection on their simulation activity during debriefing (Dubose et al., 2010). During the simulation experience, students and educators can interact with each other in various ways including student-driven, partially instructor-driven, or instructor-driven types of simulations, depending on the students' experience with classroom learning, clinical practice, and simulation-based learning (Dubose et al., 2010). Prior studies have suggested the need for additional research on how different design types and experiences change learning outcomes, reflecting such features (Khalaila, 2014; Zulkosky, 2012). In prior research, evaluations of learning outcomes have been common, while evaluations of the students' perception are rare. Therefore, we need to evaluate the simulation activity itself in simulation-based education and analyze the correlation between the quality of education and learning outcomes. One of the tools for evaluating a simulation activity itself is the Simulation Design Scale (SDS) developed by the National League for Nursing (NLN) based on Jeffries' Simulation Framework. It is a useful tool for evaluating students' perception of simulation design (Adamson and Rojers, 2012; Wilson and Klein, 2012).
707
Group 2 (n=34) Simulation activity and debriefing
Group 2 (n=34) Lecture and case studies
Measurement Self-confidence in scenario 1 Critical thinking skills Perception of simulation design (SDS)
Group 1 (n=35) Lecture and case studies
Measurement Self-confidence in scenario 2 Critical thinking skills Perception of simulation design (SDS)
Fig. 1. Process of the second phase.
708
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
sampling was used and we recruited students who wished to join the simulation, and 72 students in the third year of a bachelor's degree at a nursing school volunteered. We then randomly assigned the subjects to the experimental group (36 subjects) and control group (36 subjects) based on the students' grades in the adult health nursing and fundamentals of nursing courses from the previous semester. During the study, a total of three subjects withdrew (one from the experimental group and two from the control group), and the analysis was done on the remaining 69 subjects. Variables and Measurements Validity of scenarios. We used a 5-point Likert scale for verifying the appropriateness of the scenarios constructed based on clinical cases. Responses were scored on a scale from 1 (strongly disagree) to 5 (strongly agree). General characteristics of students. We collected data from the study participants, including demographic data such as sex and age, whether they had experience with high-fidelity simulation, and their grades in the previous semester. We also measured their self-confidence in the 20 core fundamental nursing skills as defined by the Korean Accreditation Board of Nursing (Korean Accrediation Board of Nursing Education, 2012). Perceptions of the simulation design. The Simulation Design Scale (SDS) was used to evaluate the perceptions of simulation design. The SDS includes 20 questions for assessing the objectives, problem-solving, student support (cues), guided reflection/debriefing, and fidelity to reality using a 5-point Likert scale. The estimate of the reliability was Cronbach's α = .94 at the time of the development of the scale, and Cronbach's α = .87 in this study. In addition, we asked the subjects to rate how appropriate the learning objectives were for third-year students' level. A 5-point Likert scale was used for the variable, and responses ranged from 1 (very inappropriate) to 5 (very appropriate). Self-confidence. Items for the questionnaire were constructed for measuring students' self-confidence with the behaviors the students were expected to perform during the simulation activities. There were two kinds of self-confidence questionnaires according to the intended learning objectives in each scenario. The content validity of the selfconfidence questionnaires on scenarios 1 and 2 were 8.9 and 9.1, respectively. Self-confidence was measured after each of the two scenarios was implemented. A 5-point Likert scale was used, and the responses ranged from 1 (strongly disagree) to 5 (strongly agree). Critical thinking skills. To gauge critical thinking skills, we used a tool developed to measure Korean nursing students' critical thinking tendency (Kwon et al., 2006). It is composed of 35 items in total, each measured on a 5-point scale. Critical thinking skills were measured three times (pre-test, after each of the two scenario implementations). Higher scores mean stronger critical thinking skills. The estimate of the reliability was Cronbach's α = .80 at the time of developing the scale, and Cronbach's α = .81 in this study.
antibiotic solution for AST (after the skin test)’. The main objectives for scenario 2 were ‘Explain the signs and symptoms of increased intracranial pressure’, ‘Evaluate the risk of developing increased intracranial pressure’, and ‘Demonstrate a tracheal suction while preventing increased intracranial pressure’. The two scenarios' learning objectives are shown in Appendices A and B. Problem-solving. In the two scenarios, we presented events that would trigger students' nursing care actions as one or two problem-solving elements, which can be performed in the 15 min assigned for simulation activities. Students' support (cues). Forty minutes was assigned for pre-briefing to share learning outcomes and to establish the nursing care plans. During this phase, students were to find cues for problem-solving and establish their nursing plans based on the information in the scenarios. The simulation activities were designed as partially instructor-driven type simulations. In other words, the subjects entered the simulation room and performed their plan without any interruption first. If the students needed additional directions on care planning beyond that established during the pre-briefing, an instructor could give verbal cues in order to help the students achieve their learning outcomes (Dubose et al., 2010). Guided reflection/debriefing. After the simulation activities were completed, debriefing was performed for 35 min by the group. If the students were not able to accurately recall their activities, video recordings were used to help the students reflect on their actions. Fidelity. We developed scenarios containing physiological parameters and verbal responses that can be applied to high-fidelity simulators (METI 3G) according to the learning objectives we set. Control group. The control group received an education through lectures and case conferences for an hour and a half, using the same scenarios as in the high-fidelity simulation education. Data analyses Data analyses were performed with IBM SPSS Statistics for Windows version 21.0 (IBM Corp., Armonk, NY, USA). In the first phase, we used means and standard deviations to examine the validity of the scenarios developed in this study. In the second phase, we conducted a t-test and χ2 test for testing the homogeneity of the study participants, and ANCOVA to examine the differences in self-confidence and critical thinking skills between the experimental and control groups. Finally, we analyzed the correlation between the perception of simulation design, self-confidence and critical thinking skills using Pearson's correlation coefficient. All P-values are two-sided and P b .05 was considered statistically significant.
Design of High-fidelity Simulation
Ethical considerations
Objectives. Before developing the scenarios, we set detailed learning objectives for the simulation learning topics that had been chosen by reflecting on the curriculum of the first and second semesters in the third year (Kim and Min, 2013). On the basis of previous studies, we determined four to five primary learning objectives for each scenario (Waxman, 2010). The main objectives for scenario 1 were ‘Explain the signs and symptoms of CAP’, ‘Establish the prioritization of the nursing intervention in which the patient has a high fever’, and ‘Prepare the
The study procedure of this research project was approved by the Institutional Review Board in our university. All the information was handled anonymously and confidentially, and was never used for any other purpose. We explained to the study participants the goal of the study, the fact that they were free to withdraw from the study at any stage, and that they would not be at a disadvantage if they did not participate in the study. We proceeded with the study after receiving informed consent for study participation and video recording from the participants.
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711 Table 1 The validity of scenario 1: antibiotic preparation for a patient with newly diagnosed pneumonia (Appendix A), and scenario 2: tracheostomy suction to prevent increased intracranial pressure (Appendix B). Variables evaluated
Scenario 1
Scenario 2
Mean ± SD
Mean ± SD
Situations in the scenario occur frequently in clinical practice. The physiological characteristics such as the patient's vital signs properly reflect a real clinical situation. The treatment process such as the doctor's orders properly reflects a real clinical situation. The students' behaviors are appropriate for a clinical nursing plan. The students' actions indicate that the student is able to perform the required duties.
5.00 ± 0.00 4.60 ± 0.55 4.75 ± 0.50 4.17 ± 0.41 4.75 ± 0.50 4.17 ± 0.41
709
Table 3 Mean and standard deviation for students' perception of the simulation design and learning objectives (n = 69). Variables
Simulation design
Objectives Problem-solving Student support Guided reflection Fidelity Total Overall appropriateness of the learning objectives
Scenario 1
Scenario 2
Mean ± SD
Mean ± SD
3.59 ± 0.60 3.33 ± 0.50 3.54 ± 0.61 4.35 ± 0.54 3.88 ± 0.85 3.69 ± 0.44 4.24 ± 0.63
3.40 ± 0.76 3.29 ± 0.62 3.16 ± 0.66 4.06 ± 0.92 3.56 ± 0.97 3.47 ± 0.62 4.43 ± 0.55
4.50 ± 0.58 4.33 ± 0.52 5.00 ± 0.00 4.83 ± 0.41
Results
On the other hand, no statistically significant difference was found between the experimental and control groups in critical thinking skills in scenario 1. The experimental group (group 2) demonstrated significantly higher critical thinking skills after the scenario 2 simulation than the control group (group 1) did; however, the actual mean difference was found not to be significant (Table 5).
Validity of Scenarios The clinical validity of each scenario (Appendices A and B) was tested by evaluating the variables in Tables 1. In particular, the test of whether the situations in the scenario occurred frequently in clinical practice and whether the students' actions indicated that the student was able to perform the required duties had high average scores in both scenarios (Tables 1).
Correlation Between the Perceptions of the Simulation Design and the Learning Outcomes A significant weak positive correlation between the perceptions of the simulation design and self-confidence as well as critical thinking skills was found in the experimental group (Table 6). Discussion
Socio-demographic Characteristics of the Nursing Students No statistically significant difference was found between the two groups for the students' grades in the previous semester or selfconfidence in the 20 core fundamental skills. Moreover, all students had experience with a simulation lecture. Thus, the two groups can be considered homogeneous (Table 2). Students' Perception of the Simulation Design and Learning Objectives Using the SDS, five factors incorporated into the simulation design have been represented in Table 3. For the design of both scenarios, the students rated guided reflection and fidelity highly. In addition, most students agreed or strongly agreed that the learning objectives were appropriate for third-year students considering the required curriculum of the simulation class. Self-confidence and Critical Thinking Skills The experimental group (group 1) demonstrated significantly higher self-confidence than the control group (group 2) did in scenario 1. However, no significant difference was found in the level of selfconfidence after the scenario 2 simulation between the experimental group (group 2) and control group (group 1) (Table 4).
In this study, two high-fidelity simulations based on the learning objectives representative of a third-year nursing student's curriculum (from a 4-year nursing college) were evaluated. Importantly, learning objectives is one of the characteristics of simulation design (Jeffries and Rogers, 2012), and clear objectives should be integrated within the curriculum because the objectives guide all of the aspects of simulation design (Arthur et al., 2013). In this study, high-fidelity simulation design features constructed to fit specific learning objectives demonstrated significant positive correlations with certain learning outcomes. Therefore, improved performance in course material may be achievable with the addition of rigorous simulation design. Learning objectives need to be based on students' prior knowledge and skills, include intended outcomes and expected behaviors, and be presented specifically enough for students to effectively participate in simulation activities (Jeffries and Rogers, 2012). Simulation-based education on a human patient may limit the student's ability to achieve all of their learning objectives due to technical issues and potential complexities in working with a human simulator. Therefore, scenarios need to be clearly connected with the objectives of the curriculum (Bremner et al., 2006), and educators need to choose or develop scenarios suitable for the students' level with consideration of the fidelity and problem-solving elements of the simulation design (Akhu-Zaheya et al., 2013; Jeffries and Rogers, 2012). To enhance the integrity of a
Table 2 Demographic characteristics of participants in the second phase. Characteristics
Age (years) Previous semester scores from adult health nursing Previous semester scores from fundamentals of nursing Self-confidence in the 20 core fundamental nursing skills Critical thinking Gender Female Male Participated in the simulation lecture Yes No
Group 1 (n = 35)
Group 2 (n = 34)
N (%) or Mean ± SD
N (%) or Mean ± SD
20.13 ± 1.24 84.75 ± 8.27 87.03 ± 4.58 2.92 ± 0.64 3.22 ± 0.68 32 (91.4) 3 (8.6) 32 (100) 0 (0)
20.81 ± 2.65 84.28 ± 8.60 86.69 ± 5.18 3.18 ± 0.49 3.36 ± 0.37 32 (94.1) 2 (5.9) 32 (100) 0 (0)
χ2 or t
p-value
−1.332 −0.222 0.281 −1.834 −1.085 .186
.190 .825 .779 .071 .282 .327
710
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Table 4 Mean and difference of self-confidence (n = 69).
Scenario 1 Scenario 2
Group 1
Group 2
Mean ± SD
Mean ± SD
Experimental group 4.05 ± 0.48 Control group 3.37 ± 0.41
Control group 3.86 ± 0.53 Experimental group 3.56 ± 0.34
Fa
p-Value
Table 6 Correlations between the perceptions of the simulation design and the two learning outcomes (n = 69). Perceptions of simulation design Self confidence
4.674
.034
2.908
.093
Critical thinking skills
Pearson r p-Value Pearson r p-Value
.252 .038 .315 .021
a ANCOVA, covariate = self-confidence in the 20 core fundamental nursing skills as defined by the Korean Accreditation Board of Nursing at pre-test.
simulation, scenario developers must have relevant clinical expertise to describe the course of events and intervention responses according to learning objectives accurately (Waxman, 2010). The scenarios used in this study were developed to facilitate students' learning on clinical cases that frequently occur in Korean university hospitals, and, in fact, both scenarios properly replicated clinical situations according to the experts. On the other hand, the students rated problem-solving of simulation design from 3.29 to 3.33 points; thus, they were not as satisfied with their ability to solve problems in the simulation. Two previous studies reported similar findings when nursing students at a 4-year college performed poorly or were not able to achieve expected results in a problem-solving and decision-making process after undergoing basic assessments in cases about pulmonary embolism and hypovolemia (Cooper et al., 2010; Fero et al., 2010). In these studies, students were thought to already have the required knowledge but failed to apply it to the given situations thus, educators should support them by using obvious cues during simulation activity. Educators who are planning simulations should consider when and how they can support their students before the simulation (Jeffries and Rogers, 2012). In this study, a partial-instructor driven type simulation was used to provide some student support (Dubose et al., 2010). To guarantee that students actively participated in the simulation, problem-solving was initiated using cues provided in the proctor's scripts. For example, scenario 1 tested the student's ability to decide on the order of tasks in administering antibiotics and antipyretics and performing a blood culture in a patient with a newly developed high fever. In addition, the students had to provide the rationale for their decisions, which tested their problem-solving skills. Additional information was provided to the students by an instructor when decisions or judgments during the simulation activities were questionable. However, the students rated the level of support that was provided between 3.16 and 3.54, which may mean that the students had hoped for more support during the simulation. Our findings are similar to the results of a previous study with a similar study design. They found that fourth-year nursing students in Korea perceived a very low level of student support being provided while they conducted an emergency care management simulation (Hur et al., 2013). In addition, newly graduated nurses perceived a similar level of student support as in our study while completing a simulation (Wilson and Klein, 2012). The student felt supported by the instructor's assistance during the simulation activity, but when cues were delivered, they tended to be reported inconsistently and often lacked clarity (Jeffries, 2005). Considering that the results of previous studies and this study found that student support scored
Table 5 Mean and difference of critical thinking skills (n = 69).
Scenario 1 Scenario 2 a
Group 1
Group 2
Mean ± SD
Mean ± SD
Experimental group 3.31 ± 0.31 Control group 3.37 ± 0.35
Control group 3.34 ± 0.27 Experimental group 3.27 ± 0.33
ANCOVA, covariate = critical thinking skills at pre-test.
lower than other factors of the simulation design did, efficient support should be provided to third- and fourth-year nursing students during simulation activities. In simulation-based education, a guided reflection process after the simulation is important so students can thoroughly reconstruct their previous cognitive framework since reflection during simulation is typically impossible due to time constraints (Bland et al., 2011). In particular, video recording can be an effective means of reflection because subjects can easily remember their actions/decisions and evaluate them objectively (Chronister and Brown, 2012). Guided reflection may have been rated the highest in our study because students were fully able to understand their experiences using a video-assisted verbal debriefing. We randomized students to obtain unbiased results (Adler et al., 2007; Khalaila, 2014), in order to analyze whether a correlation existed between their perception of the simulation design and learning outcomes. The experimental group had significantly higher scores than the control group did for measures of self-confidence such as being able to establish a nursing care plan for a patient with high fever in scenario 1. For the simulation in scenario 2 to evaluate a client's risk of developing increased intracranial pressure, while there was no significant statistical difference in self-confidence between the experimental and control groups, the mean score of the experimental group was much higher than the control group. There has been controversy over whether simulation education can be expected to increase the self-confidence of nursing students. The results of other studies show that no significant changes in self-confidence were found after simulating on a hemodynamic assessment (Liaw et al., 2012). On the other hand, nursing students' confidence improved significantly in emergency care, which was considered a relatively difficult task (Hur and Park, 2012). Considering these results, researchers might have to consider the level of difficulty or significance of the tasks being tested in the simulation when the students realize their mistakes during high-fidelity simulation (McCaughey and Traynor, 2010). We have discussed the significance of how students' perceptions of a simulation design contribute to their learning experience and its correlation with self-confidence and critical thinking skills as learning outcomes. The most significant implication of this study is that there was a significant, positive correlation between the students' perceived appropriateness of the simulation design and the learning outcomes. This agrees with the result of a study that showed a significantly positive correlation between perceived appropriateness and post-education self-confidence in an emergency care management simulation (Hur et al., 2013) and verifies the need for more rigorous study designs based on the recommendations of future studies (Khalaila, 2014; Zulkosky, 2012). Study Limitations
Fa
p-Value
0.001
.971
4.093
.047
Two limitations of this study were the use of convenience sampling as the sampling method and the cumulative effect of simulation education that could not be evaluated. Students' self-confidence and nursing performance are expected to improve gradually with more frequent exposure to simulations, so more notable effects should be detected with the application of the method for a greater period of time (Khalaila, 2014).
H. Ahn, H.-Y. Kim / Nurse Education Today 35 (2015) 706–711
Conclusions and Implications In this study, we selected topics relevant to third-year nursing students in a 4-year nursing program while taking into account the clinical practicum curriculum of the nursing program. After the students completed their simulations, we examined the correlation between the components of the simulation design and the student's educational outcomes. A significant, positive correlation between the simulation design and learning outcomes was revealed. In particular, nursing students' self-confidence improved after practicing problem-solving tasks based on the learning objectives in a simulation scenario. More research is needed to decide the most appropriate type of simulation design suitable for a student's level toward the development of a more effective simulation program. Acknowledgment This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MEST) (Grant number: 2012-2012R1A1A1007988). Appendix A. Supplementary data Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.nedt.2015.01.021. References Adamson, K.A., Rojers, K.J., 2012. Evaluation: a critical step in simulation practice and research. In: Jeffries, P.R. (Ed.), Simulation in Nursing Education: From Conceptualization to Evaluation. National League for Nursing, New York, pp. 131–161. Adler, M.D., Trainor, J.L., Siddall, V.J., McGaghie, W.C., 2007. Development and evaluation of high-fidelity simulation case scenarios for pediatric resident education. Ambul. Pediatr. 7 (2), 182–186. Akhu-Zaheya, L.M., Gharaibeh, M.K., Alostaz, Z.M., 2013. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin. Simul. Nurs. 9 (9), e335–e342. Arthur, C., Levett-Jones, T., Kable, A., 2013. Quality indicators for the design and implementation of simulation experiences: a Delphi study. Nurse Educ. Today 33 (11), 1357–1361. http://dx.doi.org/10.1016/j.nedt.2012.07.012. Benner, P., Sutphen, M., Leonard, V., Day, L., 2010. Connecting Classroom and Clinical Through Integrative Teaching and Learning Educating Nurses. A call for Radical Transformation. Jossey-Bass, Stanford, CA. Bland, A.J., Topping, A., Wood, B., 2011. A concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Educ. Today 31 (7), 664–670. http://dx.doi.org/10.1016/j.nedt.2010.10.013. Brady, D.S., 2011. Using quality and safety education for nurses (QSEN) as a pedagogical structure for course redesign and content. Int. J. Nurs. Educ. Scholarsh. 8 (1). http:// dx.doi.org/10.2202/1548-923X.2147 (Article 5). Bremner, M.N., Aduddell, K., Bennett, D.N., VanGeest, J.B., 2006. The use of human patient simulators: best practice with novice nursing students. Nurse Educ. 31 (4), 170–174. Chronister, C., Brown, D., 2012. Comparison of simulation debriefing methods. Clin. Simul. Nurs. 8 (7), e281–e288. http://dx.doi.org/10.1016/j.ecns.2010.12.005. Cioffi, J., 2001. Clinical simulations: development and validation. Nurse Educ. Today 21, 477–486. Cooper, S., Kinsman, L., Buykx, P., McConnell-Henry, T., Endacott, R., Scholes, J., 2010. Managing the deteriorating patient in a simulated environment: nursing students' knowledge, skill and situation awareness. J. Clin. Nurses 19 (15–16), 2309–2318. http://dx. doi.org/10.1111/j.1365-2702.2009.03164.x.
711
Dubose, D., Sellinger-Karmel, L.D., Scoloveno, R.L., 2010. Baccalaureate nursing education. In: Nehring, M., Lashley, R. (Eds.), High-fidelity Patient Simulation in Nursing Education. Jones and Bartlett Publishers, MA, pp. 189–203. Fero, L.J., O'Donnell, J.M., Zullo, T.G., Dabbs, A.D., Kitutu, J., Samosky, J.T., Hoffman, L.A., 2010. Critical thinking skills in nursing students: comparison of simulation-based performance with metrics. J. Adv. Nurs. 66 (10), 2182–2193. http://dx.doi.org/10. 1111/j.1365-2648.2010.05385.x. Handwerker, S.M., 2012. Transforming nursing education: a review of current curricular practices in relation to Benner's latest work. Int. J. Nurs. Educ. Scholarsh. 9 (1). http://dx.doi.org/10.1515/1548-923X.2510 (Article 21). Hur, H.K., Park, S., 2012. Effects of simulation based education, for emergency care of patients with dyspnea, on knowledge and performance confidence of nursing students. J. Korean Acad. Soc. Nurs. Educ. 18 (1), 110–118. http://dx.doi.org/10.5977/jkasne. 2012.18.1.110. Hur, H.K., Park, S., Shin, Y.H., Lim, Y.M., Kim, G., Kim, K.K., Choi, H.O., Choi, J.H., 2013. Development and applicability evaluation of an emergent care management simulation practicum for nursing students. J. Korean Acad. Soc. Nurs. Educ. 19 (2), 228–240. http://dx.doi.org/10.5977/jkasne.2013.19.2.228. Jeffries, P.R., 2005. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs. Educ. Perspect. 26 (2), 96–103. Jeffries, P.R., Rogers, K.J., 2012. Theoretical framework for simulation design. In: Jeffries, P.R. (Ed.), Simulation in Nursing Education. National League for Nursing, New York, pp. 25–44. Kerr, B., Hawkins, T.L., Herman, R., Barnes, S., Kaufmann, S., Fraser, K., Ma, I.W., 2013. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med. Educ. Online 18 (18), 21312. http://dx.doi.org/10.3402/meo.v18i0.21312. Khalaila, R., 2014. Simulation in nursing education: an evaluation of students' outcomes at their first clinical practice combined with simulations. Nurse Educ. Today 34, 252–258. http://dx.doi.org/10.1016/j.nedt.2013.08.015. Kim, H., Min, Y.H., 2013. Development of curriculum and scenarios using constructive alignment theory for simulation-based education for nursing students to enhance clinical skills and nursing knowledge. Paper Presented at the International Proceedings of Economics Development & Research, Rome. Korean Accrediation Board of Nursing Education, 2012. Core Fundamental Nursing Skill for Nursing Education Accreditation. (Retrieved 04.30, 2013, from http://www. kabon.or.kr/HyAdmin/upload/goodFile/120120319112302.pdf.). Kwon, I.S., Lee, G.E., Kim, G.D., Kim, Y.H., Park, K.M., Park, H.S., Sohn, S.K., Lee, W.S., Jang, K.S., Chung, B.Y., 2006. Development of a critical thinking disposition scale for nursing students. J. Korean Acad. Nurs. 36 (6), 950–958. Landeen, L., Nielson, A., 2008. Focus on simulation-integrating simulation into teaching practice. J. Nurs. Educ. 47 (11), 487–488. Liaw, S.Y., Scherpbier, A., Rethans, J.J., Klainin-Yobas, P., 2012. Assessment for simulation learning outcomes: a comparison of knowledge and self-reported confidence with observed clinical performance. Nurse Educ. Today 32 (6), e35–e39. http://dx.doi. org/10.1016/j.nedt.2011.10.006. McCaughey, C.S., Traynor, M.K., 2010. The role of simulation in nurse education. Nurse Educ. Today 30 (8), 827–832. http://dx.doi.org/10.1016/j.nedt.2010.03.005. Smith, S.J., Roehrs, C.J., 2009. High-fidelity simulation: factors correlated with nursing student satisfaction and self-confidence. Nurs. Educ. Perspect. 30 (2), 74–78. Waxman, K.T., 2010. The development of evidence-based clinical simulation scenarios; guidelines for nurse educators. J. Nurs. Educ. 49 (1), 29–35. http://dx.doi.org/10. 3928/01484834-20090916-07. Weatherspoon, D.L., Wyatt, T.H., 2012. Testing computer-based simulation to enhance clinical judgment skills in senior nursing students. Nurs. Clin. North Am. 47 (4), 481–491. http://dx.doi.org/10.1016/j.cnur.2012.07.002. Whyte IV, J., Ward, P., Eccles, D.W., Harris, K.R., Nandagopal, K., Torof, J.M., 2012. Nurses' immediate response to the fall of a hospitalized patient: a comparison of actions and cognitions of experienced and novice nurses. Int. J. Nurs. Stud. 49 (9), 1054–1063. http://dx.doi.org/10.1016/j.ijnurstu.2012.03.005. Wilson, R.D., Klein, J.D., 2012. Design, implementation and evaluation of a nursing simulation: a design and development research study. J. Appl. Instr. Des. 2 (1), 57–68. Zulkosky, K.D., 2012. Simulation use in the classroom: impact on knowledge acquisition, satisfaction, and self-confidence. Clin. Simul. Nurs. 8 (1), e25–e33. http://dx.doi.org/ 10.1016/j.ecns.2010.06.003.
