Copyright © eContent Management Pty Ltd. Contemporary Nurse (2014) 49: 4–14.
A psychometric evaluation of the Korean version of the evidence-based practice questionnaire for nurses
Youn-Jung Son, Youngshin Song*, So-Youn Park+ and Jong-Im Kim* Department of Nursing, Soonchunhyang University, Cheonan, Republic of Korea; *College of Nursing, Chungnam National University, Daejeon, Republic of Korea; +Department of Social Welfare, Kyonggi University, Suwon-si, Republic of Korea
Abstract: Background: Measurement of evidence-based practice (EBP) competence is a challenge in the health care
setting. Aim: The purpose of this study was to evaluate the psychometric properties of the Korean version of the EvidenceBased Practice Questionnaire (EBPQ). Methods/Design: Data were collected from Korean clinical nurses (N = 801) who worked in acute care settings. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to assess its construct validity. Concurrent validity and internal consistence were also assessed. Results: Findings identified that EFA revealed a three-factor solution with 64.4% of total variance explained. The factor loadings ranged from 0.431 to 0.900 for the 24 items. The instrument showed good reliability and concurrent validity with the Critical Thinking Disposition scale. CFA revealed that the Korean version of the EBPQ was acceptable. Conclusion: The Korean version of the EBPQ is a valid and reliable evaluation measure of EBP competencies of nurses in South Korea.
Keywords: factor analysis, evidence-based practice, psychometrics
E
vidence-based practice (EBP) has become increasingly important in health care settings throughout the world. It has provided a framework for clinical problem-solving and decisionmaking activities, allowing practitioners to keep up-to-date with the best practice in their field. To engage in EBP, health-care providers should be able to interpret and apply the current best evidence to a given clinical situation (Weng et al., 2013). It is essential to have knowledge and an understanding of the effectiveness of clinical practices (Upton & Upton, 2005). Patients who receive clinical care based on the most current and best evidence generated by research can experience better outcomes in clinical situations (Doran & Sidani, 2007). Numerous studies have emphasised the importance of nurses’ EBP competence in asking clinical questions, searching for evidence, and using critical appraisal techniques (Bostrom, Ehrenberg, Gustavsson, & Wallin, 2009; Bostrom, Rudman, Ehrenberg, Gustavsson, & Wallin, 2013; Chang, Russell, & Jones, 2010; Johnston, Leung, Fielding, Tin, & Ho, 2003). Meanwhile, the importance of competence in EBP has emerged in clinical settings and efforts to measure competence in EBP have been made by using various instruments. The Evidence-Based Practice Questionnaire (EBPQ) was designed to 4
CN
Volume 49, December 2014
measure EBP practice, knowledge/skills, and attitudes for hospitals and community nurses (Upton & Upton, 2006). As part of the effort to develop competence in EBP, educational curricula and clinical training of students, health-care providers have focussed on measuring their perceptions and behaviours related to EBP (Gerrish et al., 2011; GonzalezTorrente et al., 2012; Hadley, Hassan, & Khan, 2008; Johnston et al., 2003; Leo, Peterson, Haas, LeFebvre, & Bhalerao, 2012). Despite those endeavours, findings from the literature have revealed that health-care providers, such as physicians, staff nurses, nurse practitioners, and social workers differ not only in how they facilitate EBP but also how familiar they are with the term, EBP (Lim et al., 2011; Melnyk, Fineout-Overholt, & Mays, 2008; Son, Kim, Park, Lee, & Lee, 2012). A study by Chang et al. (2010) reported that the most frequently cited barriers to EBP were related to insufficient authority to change practice, difficulty understanding statistical analyses, and perceived isolation from knowledgeable colleagues with whom to discuss research (Chang et al., 2010). Prior to extracting the factors related to EBP implementation, various instruments have been used to measure EBP in relation to perception
Psychometric evaluation of the EBPQ (Korean) and behaviour, such as the attitudes, beliefs, and knowledge of health-care providers (RuzafaMartinez, Lopez-Iborra, Moreno-Casbas, & Madrigal-Torres, 2013). Well known instruments selected to measure these outcomes include the EBPQ (Upton & Upton, 2006), the EBP Belief Scale (Melnyk et al., 2008), the Nurses Attitudes Towards EBP Scale (Thiel & Ghosh, 2008), the EBP Capability Belief Scale (Wallin, Bostrom, & Gustavsson, 2012), and the EBP Competence Questionnaire (Ruzafa-Martinez et al., 2013). The EBPQ, developed by Upton and Upton (2006), has had reported use to evaluate nurses’ self-perceived competency in EBP, in English and Spanish (de Pedro Gomez et al., 2009; RuzafaMartinez et al., 2013; Upton & Upton, 2006). Critical thinking is vital to evidence-based nursing practice (Profetto-McGrath, 2005). Along with EBP, nurses with a critical thinking disposition (CTD) have a tendency to engage in self-correction, and are therefore, likely to base their practice on evidence rather than on personal preferences or beliefs (Facchiano & Snyder, 2012). Nurses using critical thinking skills assess the situation by considering the context of the patient’s culture and characteristics, recognising the patient’s uniqueness, abandoning ineffective interventions, and considering multiple approaches to problem-solving (Upton & Upton, 2006). Therefore, a CTD might be a reliable predictor of EBP. Despite the global expansion of EBP over the past 20 years, few studies about nurses’ perceptions and behaviours related to EBP have been conducted in South Korea (Lim et al., 2011; Son et al., 2012). Little is known about Korean nurses’ views of EBP, and studies of South Korean nurses report that they are not competent at interpreting research findings to guide treatment in the clinical setting (Lim et al., 2011; Son et al., 2012). In recent studies by Lim et al. (2011) and Son et al. (2012), competence in EBP was assessed using the EBPQ without considering its cultural appropriateness and validity (Lim et al., 2011; Son et al., 2012). Therefore, a psychometric evaluation of the instrument was deemed necessary before accepting the results of previous studies and its use in South Korea. To meet this need, we
© eContent Management Pty Ltd
CN
examined the psychometric properties of the current version of the EBPQ in Korean health care settings. Methods Study design, samples, and data collection A cross-sectional survey of Korean nurses was used for our psychometric evaluation of the EBPQ (Upton & Upton, 2006). The inclusion criteria were registered nurses who (a) worked in designated university hospitals and (b) agreed with the research protocols. Multiple data set is recommended when examining the scale for refining and generating to the sample from a relevant population (Netemeyer, Bearden, & Sharma, 2003). Thus, this study used data that were collected as part of two previous studies. We collected and combined the two different data sets to try to represent the samples and to analyse two different factor analyse while two studies were similar study structure in terms of study design, sampling method, qualification of respondents and survey context. The first author independently conducted Study 1 (Son et al., 2012). In that study, 492 nurses were recruited from four university hospitals located in metropolitan areas of South Korea, and the data were collected between November and December of 2011. Details of the data collection protocol can be found in the published report (Son et al., 2012). Study 2 was conducted by the second author between August and October 2013. Data for Study 2 were collected by mail from 309 nurses who worked in three university hospitals (different from those in Study 1). Prior to collecting the data, we selected university hospitals that were similar to the ones used in Study 1 in terms of size and staff characteristics such as the number of nurses and qualifications of staff nurses. We then contacted the nursing departments of the hospitals and presented the research aims and procedures, including voluntary participation, withdrawal, anonymous data collection, and the sole use of data for research purposes. Three out of five hospitals agreed to participate in the survey. After receiving the organisational agreement, 320 questionnaires were mailed to the three hospitals
Volume 49, December 2014
CN
5
CN
Youn-Jung Son et al.
and the questionnaires were self-administered using a paper-and-pencil format. Of the 320 questionnaires, 309 were returned by mail and included in the data set. To increase the response rate, we sent reminders to those who did not yet responded. Moreover, participants who completed it were given $10 as a token of appreciation. The response rate for this study was 96.5%. Figure 1 shows the process of data reconstruction. A total of 801 nurses comprised the final sample. Using a computer statistics IBM SPSS 20.0 version program, the combined data from Studies 1 and 2 were randomly assigned to two different data sets to conduct exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The two samples used for EFA and CFA are referred to as subsample 1 and subsample 2, respectively. The sample sizes of subsample 1 and subsample 2 were 404 and 397 nurses, respectively. The required sample size for the EFA was determined by calculating the ratio of study
Study 1 492 nurses from 4 University Hospitals
Study 2 309 nurses from 3 University Hospitals
participants to the number of items in the scale, ensuring that the ratio exceeded 10:1 (Costello & Osborne, 2005). Since the EBPQ consists of 24 items, a minimum of 240 participants were needed to perform the factor analysis. For the CFA analysis, at least 200 participants were needed (Marsh, Hau, Balla, & Grayson, 1998). Thus the number of participants in the EFA group (n = 404) and CFA (n = 397) groups far exceeded the requirements to perform factor analysis. Linguistic translation and cultural adaptation of the EBPQ to Korean To evaluate the psychometric properties of the EBPQ, it was translated to Korean by bilingual researchers from Korea, and then back translated to English by independent researchers who were also bilingual. After completing the forward and backward translation processes, all researchers evaluated the translated Korean version of the EBPQ by comparing the original instrument with the Korean version (EBPQ-K) that was back translated. The EBPQ-K was piloted with the nurses to ensure that its cultural and linguistic adaptations were appropriate. There were no reported problems in understanding the scale, nor was cultural rewording necessary.
Combined Data Set, N=801 Use of a computerized random assignment method
Subsample 1 n=404
Subsample 2 n=397
EFA
CFA
Figure 1: The process of data reconstruction
6
CN
Volume 49, December 2014
Instruments Evidence-based practice questionnaire The EBPQ was developed by Upton and Upton (2006) using a sample of 751 nurses in the UK to identify the determinants of EBP. The 24-item EBPQ consists of three subscales: Practice of EBP, attitude towards EBP, and knowledge/skills of EBP. The response options for each item uses a seven-point Likert scale, ranging from 1 (poor/never) to 7 (best/ frequently). The practice of EBP subscale consists of six items and is associated with individual patient care (Upton & Upton, 2006). An
© eContent Management Pty Ltd
Psychometric evaluation of the EBPQ (Korean) example of a question from this subscale is, ‘How often have you used critical appraisal against set criteria, and literature you have researched’? The scores range from 7 to 42, with higher scores indicating higher levels of the practice of EBP. The subscale measuring attitude towards EBP consists of four items, including perceived barriers, such as workload and personal judgements of the value of EBP. The scores range from 4 to 28, with higher scores indicating a more positive attitude towards EBP. Examples of questions from this subscale include, ‘My workload is too great for me to keep up-to-date with all the new evidence’ and ‘EBP is a waste of time.’ The subscale measuring knowledge/skills of EBP consists of 14 items with scores ranging from 14 to 98, with higher scores representing a higher level of knowledge/skills of EBP. Examples of items include, ‘Monitoring and reviewing of practice skills’ and ‘Ability to determine how useful (clinically applicable) the material is.’ Cronbach’s alpha levels for the three subscales were reported to be greater than 0.7, indicating good reliabilities (Upton & Upton, 2006). The total reliability of the EBPQ-K was 0.95 (Cronbach’s alpha) in our study. Critical thinking disposition scale The CTD scale, developed by Yoon (2004), was used to test the concurrent validity of the EBPQ in this study. The CTD scale consists of 27 items with a five-point Likert scale ranging from 1 (not at all true) to 5 (extremely true). Seven subcategories include systematicity, intellectual fairness, healthy scepticism, objectivity, intellectual eagerness/curiosity, prudence, and self-confidence (Yoon, 2008). A higher score indicates a higher level of CTD. The validity and reliability of the CTD were tested in several studies of samples of nursing students and nurses (Choi & Cho, 2011; Kim, 2012; Song, 2009). Cronbach’s alpha for the present study was 0.78. Data analysis Multiple methods of data analyses were performed to evaluate the EBPQ’s psychometric properties. Item analyses, including Cronbach’s alpha and the corrected item-total correlations were performed to test the reliability of subsample 1
© eContent Management Pty Ltd
CN
(n = 404). We checked the value of each itemtotal correlation to ensure that it was at least 0.30 (Nunnally & Bernstein, 1994) and eliminated those below 0.30. Next, we assessed sampling adequacy using the Kaiser–Meyer–Olkin (KMO) index and Bartlett’s test for EFA. The KMO index is a comparison of the magnitude of the measured correlation coefficients with the magnitude of the partial correlation coefficients (Nunnally & Bernstein, 1994). Values between 0.5 and 1.0 indicate that factor analysis is appropriate. Bartlett’s test of sphericity was also used to examine the hypothesis that the variables are uncorrelated in the population. That is, the population correlation matrix is an identity matrix (each variable correlates perfectly with itself [r = 1]; there is no correlation with the other variables [r = 0]). If the probability level (p) is greater than 0.05, factor analysis is not appropriate (Nunnally & Bernstein, 1994). Exploratory factor analysis and CFA were then used to assess the EBPQ’s construct validity. EFA was performed using principal axis factoring with a direct oblimin rotation in subsample 1, because this method effectively identifies the component parts of a scale according to the correlations between the factors (Costello & Osborne, 2005). Factors were initially extracted based on their eigenvalues (>1.0) and the scree plot (Thompson & Daniel, 1996). Consistent with Costello and Osborne’s (2005) guidelines, after rotation, we selected items with loadings above 0.32 and ensured that there were no items with cross-loadings of over 0.20 on more than two factors (Costello & Osborne, 2005). Confirmatory factor analysis was performed using the Analysis of Moment Structure (AMOS) program with subsample 2 to confirm the model fit the chosen factors. Chi-square goodness of fit (χ2/df ), the Normed Fit Index (NFI), the Standardised Root Mean Square Residual (SRMR), and the Comparative Fit Index (CFI) were used to estimate the model’s fit. The Chisquare Goodness of Fit test reveals the difference between the observed covariance matrix and the one predicted by the specified model. However, this index is sensitive to sample size; therefore, the relative goodness of fit indices, such as the NFI,
Volume 49, December 2014
CN
7
CN
Youn-Jung Son et al.
CFI, and SRMR, are strongly recommended (Bentler, 1990). A low Chi-square value that is not significant indicates that the hypothesised model is a good fit (Schaufeli, Salanova, González-romá, & Bakker, 2002). An acceptable fit of other indices are NFI ≥ 0.80, CFI ≥ 0.80, and SRMR ≤ 0.08 (Hu & Bentler, 1999). Finally, to test the concurrent validity of the EBPQ-K, Pearson’s correlation coefficient between the EBPQ-K and the CTD was calculated using the combined data set. The IBM SPSS Statistics 20 and SPSS AMOS 21 statistical software programs (SPSS Inc., Chicago, IL, USA) were used for data analyses and a significance level of 5% was applied. Ethical considerations The approval for Study 1 was obtained from the Institutional Review Board of Soonchunhyang University in South Korea (2011-10-04-02) and the Institutional Review Board of Chungnam National University’s College of Nursing approved Study 2 (2013–27). All participants provided written informed consent by mail prior to their participation. The nurses received a structured questionnaire, pen, and envelope with instructions to place the completed survey inside of it. They were instructed to seal the envelope to ensure that the survey information would be kept confidential when it was returned to the researchers. Results Demographics of participants Table 1 summarises the sample’s characteristics, including mean age, work experience (in months), gender, education level, religion, marital status, and RN position. Data from the total sample (N = 801) were randomly divided into subsample 1 (n = 404) for the EFA group and subsample 2 (n = 397) for the CFA group. The mean age of the EFA and CFA subsamples were 30.7 (SD = 7.32) and 30.7 (SD = 7.30), respectively. The majority of the nurses (about 98%) were female in both the samples (EFA and CFA) and the mean age of subsamples 1 and 2 were 30.7 and 30.5 years, respectively. 8
CN
Volume 49, December 2014
Table 1: Each subsample characteristics (N = 801) Characteristics
Subsample 1 Subsample 2 (n = 404) (n = 397)
Age (yrs): Mean ± SD Work experience (months): Mean ± SD Female, N (%) Education level, N (%) University degree Over master’s degree Having religion, N (%) Married, N (%) Position (RN), N (%)
30.7 ± 7.32 97.2 ± 88.98
30.7 ± 7.30 103.3 ± 88.49
395 (97.8)
392 (98.7)
291 (72.2) 113 (27.8) 198 (49.0) 133 (34.8) 318 (78.7)
300 (75.6) 97 (24.4) 208 (52.4) 135 (35.7) 298 (75.3)
Item analysis and reliability The results of the item-total score correlation showed that none of the items had a correlation of less than 0.3 among the 24 items; therefore, all of the items were retained. Scores on the itemtotal correlation of the 24 items ranged from 0.48 to 0.78. Cronbach’s alpha was used to confirm the internal consistency of the Korean version of the (EBPQ-K). Cronbach’s alpha for the 24-item EBPQ-K was 0.94. For the total study sample, Cronbach’s alphas for the three subscales measuring practice of, attitude towards, and knowledge/ skills of EBP were 0.92, 0.82, and 0.95, respectively (Table 2). Construct validity Exploratory factor analysis The KMO value was 0.94 and Bartlett’s test of sphericity was
A psychometric evaluation of the Korean version of the evidence-based practice questionnaire for nurses
Youn-Jung Son, Youngshin Song*, So-Youn Park+ and Jong-Im Kim* Department of Nursing, Soonchunhyang University, Cheonan, Republic of Korea; *College of Nursing, Chungnam National University, Daejeon, Republic of Korea; +Department of Social Welfare, Kyonggi University, Suwon-si, Republic of Korea
Abstract: Background: Measurement of evidence-based practice (EBP) competence is a challenge in the health care
setting. Aim: The purpose of this study was to evaluate the psychometric properties of the Korean version of the EvidenceBased Practice Questionnaire (EBPQ). Methods/Design: Data were collected from Korean clinical nurses (N = 801) who worked in acute care settings. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to assess its construct validity. Concurrent validity and internal consistence were also assessed. Results: Findings identified that EFA revealed a three-factor solution with 64.4% of total variance explained. The factor loadings ranged from 0.431 to 0.900 for the 24 items. The instrument showed good reliability and concurrent validity with the Critical Thinking Disposition scale. CFA revealed that the Korean version of the EBPQ was acceptable. Conclusion: The Korean version of the EBPQ is a valid and reliable evaluation measure of EBP competencies of nurses in South Korea.
Keywords: factor analysis, evidence-based practice, psychometrics
E
vidence-based practice (EBP) has become increasingly important in health care settings throughout the world. It has provided a framework for clinical problem-solving and decisionmaking activities, allowing practitioners to keep up-to-date with the best practice in their field. To engage in EBP, health-care providers should be able to interpret and apply the current best evidence to a given clinical situation (Weng et al., 2013). It is essential to have knowledge and an understanding of the effectiveness of clinical practices (Upton & Upton, 2005). Patients who receive clinical care based on the most current and best evidence generated by research can experience better outcomes in clinical situations (Doran & Sidani, 2007). Numerous studies have emphasised the importance of nurses’ EBP competence in asking clinical questions, searching for evidence, and using critical appraisal techniques (Bostrom, Ehrenberg, Gustavsson, & Wallin, 2009; Bostrom, Rudman, Ehrenberg, Gustavsson, & Wallin, 2013; Chang, Russell, & Jones, 2010; Johnston, Leung, Fielding, Tin, & Ho, 2003). Meanwhile, the importance of competence in EBP has emerged in clinical settings and efforts to measure competence in EBP have been made by using various instruments. The Evidence-Based Practice Questionnaire (EBPQ) was designed to 4
CN
Volume 49, December 2014
measure EBP practice, knowledge/skills, and attitudes for hospitals and community nurses (Upton & Upton, 2006). As part of the effort to develop competence in EBP, educational curricula and clinical training of students, health-care providers have focussed on measuring their perceptions and behaviours related to EBP (Gerrish et al., 2011; GonzalezTorrente et al., 2012; Hadley, Hassan, & Khan, 2008; Johnston et al., 2003; Leo, Peterson, Haas, LeFebvre, & Bhalerao, 2012). Despite those endeavours, findings from the literature have revealed that health-care providers, such as physicians, staff nurses, nurse practitioners, and social workers differ not only in how they facilitate EBP but also how familiar they are with the term, EBP (Lim et al., 2011; Melnyk, Fineout-Overholt, & Mays, 2008; Son, Kim, Park, Lee, & Lee, 2012). A study by Chang et al. (2010) reported that the most frequently cited barriers to EBP were related to insufficient authority to change practice, difficulty understanding statistical analyses, and perceived isolation from knowledgeable colleagues with whom to discuss research (Chang et al., 2010). Prior to extracting the factors related to EBP implementation, various instruments have been used to measure EBP in relation to perception
Psychometric evaluation of the EBPQ (Korean) and behaviour, such as the attitudes, beliefs, and knowledge of health-care providers (RuzafaMartinez, Lopez-Iborra, Moreno-Casbas, & Madrigal-Torres, 2013). Well known instruments selected to measure these outcomes include the EBPQ (Upton & Upton, 2006), the EBP Belief Scale (Melnyk et al., 2008), the Nurses Attitudes Towards EBP Scale (Thiel & Ghosh, 2008), the EBP Capability Belief Scale (Wallin, Bostrom, & Gustavsson, 2012), and the EBP Competence Questionnaire (Ruzafa-Martinez et al., 2013). The EBPQ, developed by Upton and Upton (2006), has had reported use to evaluate nurses’ self-perceived competency in EBP, in English and Spanish (de Pedro Gomez et al., 2009; RuzafaMartinez et al., 2013; Upton & Upton, 2006). Critical thinking is vital to evidence-based nursing practice (Profetto-McGrath, 2005). Along with EBP, nurses with a critical thinking disposition (CTD) have a tendency to engage in self-correction, and are therefore, likely to base their practice on evidence rather than on personal preferences or beliefs (Facchiano & Snyder, 2012). Nurses using critical thinking skills assess the situation by considering the context of the patient’s culture and characteristics, recognising the patient’s uniqueness, abandoning ineffective interventions, and considering multiple approaches to problem-solving (Upton & Upton, 2006). Therefore, a CTD might be a reliable predictor of EBP. Despite the global expansion of EBP over the past 20 years, few studies about nurses’ perceptions and behaviours related to EBP have been conducted in South Korea (Lim et al., 2011; Son et al., 2012). Little is known about Korean nurses’ views of EBP, and studies of South Korean nurses report that they are not competent at interpreting research findings to guide treatment in the clinical setting (Lim et al., 2011; Son et al., 2012). In recent studies by Lim et al. (2011) and Son et al. (2012), competence in EBP was assessed using the EBPQ without considering its cultural appropriateness and validity (Lim et al., 2011; Son et al., 2012). Therefore, a psychometric evaluation of the instrument was deemed necessary before accepting the results of previous studies and its use in South Korea. To meet this need, we
© eContent Management Pty Ltd
CN
examined the psychometric properties of the current version of the EBPQ in Korean health care settings. Methods Study design, samples, and data collection A cross-sectional survey of Korean nurses was used for our psychometric evaluation of the EBPQ (Upton & Upton, 2006). The inclusion criteria were registered nurses who (a) worked in designated university hospitals and (b) agreed with the research protocols. Multiple data set is recommended when examining the scale for refining and generating to the sample from a relevant population (Netemeyer, Bearden, & Sharma, 2003). Thus, this study used data that were collected as part of two previous studies. We collected and combined the two different data sets to try to represent the samples and to analyse two different factor analyse while two studies were similar study structure in terms of study design, sampling method, qualification of respondents and survey context. The first author independently conducted Study 1 (Son et al., 2012). In that study, 492 nurses were recruited from four university hospitals located in metropolitan areas of South Korea, and the data were collected between November and December of 2011. Details of the data collection protocol can be found in the published report (Son et al., 2012). Study 2 was conducted by the second author between August and October 2013. Data for Study 2 were collected by mail from 309 nurses who worked in three university hospitals (different from those in Study 1). Prior to collecting the data, we selected university hospitals that were similar to the ones used in Study 1 in terms of size and staff characteristics such as the number of nurses and qualifications of staff nurses. We then contacted the nursing departments of the hospitals and presented the research aims and procedures, including voluntary participation, withdrawal, anonymous data collection, and the sole use of data for research purposes. Three out of five hospitals agreed to participate in the survey. After receiving the organisational agreement, 320 questionnaires were mailed to the three hospitals
Volume 49, December 2014
CN
5
CN
Youn-Jung Son et al.
and the questionnaires were self-administered using a paper-and-pencil format. Of the 320 questionnaires, 309 were returned by mail and included in the data set. To increase the response rate, we sent reminders to those who did not yet responded. Moreover, participants who completed it were given $10 as a token of appreciation. The response rate for this study was 96.5%. Figure 1 shows the process of data reconstruction. A total of 801 nurses comprised the final sample. Using a computer statistics IBM SPSS 20.0 version program, the combined data from Studies 1 and 2 were randomly assigned to two different data sets to conduct exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The two samples used for EFA and CFA are referred to as subsample 1 and subsample 2, respectively. The sample sizes of subsample 1 and subsample 2 were 404 and 397 nurses, respectively. The required sample size for the EFA was determined by calculating the ratio of study
Study 1 492 nurses from 4 University Hospitals
Study 2 309 nurses from 3 University Hospitals
participants to the number of items in the scale, ensuring that the ratio exceeded 10:1 (Costello & Osborne, 2005). Since the EBPQ consists of 24 items, a minimum of 240 participants were needed to perform the factor analysis. For the CFA analysis, at least 200 participants were needed (Marsh, Hau, Balla, & Grayson, 1998). Thus the number of participants in the EFA group (n = 404) and CFA (n = 397) groups far exceeded the requirements to perform factor analysis. Linguistic translation and cultural adaptation of the EBPQ to Korean To evaluate the psychometric properties of the EBPQ, it was translated to Korean by bilingual researchers from Korea, and then back translated to English by independent researchers who were also bilingual. After completing the forward and backward translation processes, all researchers evaluated the translated Korean version of the EBPQ by comparing the original instrument with the Korean version (EBPQ-K) that was back translated. The EBPQ-K was piloted with the nurses to ensure that its cultural and linguistic adaptations were appropriate. There were no reported problems in understanding the scale, nor was cultural rewording necessary.
Combined Data Set, N=801 Use of a computerized random assignment method
Subsample 1 n=404
Subsample 2 n=397
EFA
CFA
Figure 1: The process of data reconstruction
6
CN
Volume 49, December 2014
Instruments Evidence-based practice questionnaire The EBPQ was developed by Upton and Upton (2006) using a sample of 751 nurses in the UK to identify the determinants of EBP. The 24-item EBPQ consists of three subscales: Practice of EBP, attitude towards EBP, and knowledge/skills of EBP. The response options for each item uses a seven-point Likert scale, ranging from 1 (poor/never) to 7 (best/ frequently). The practice of EBP subscale consists of six items and is associated with individual patient care (Upton & Upton, 2006). An
© eContent Management Pty Ltd
Psychometric evaluation of the EBPQ (Korean) example of a question from this subscale is, ‘How often have you used critical appraisal against set criteria, and literature you have researched’? The scores range from 7 to 42, with higher scores indicating higher levels of the practice of EBP. The subscale measuring attitude towards EBP consists of four items, including perceived barriers, such as workload and personal judgements of the value of EBP. The scores range from 4 to 28, with higher scores indicating a more positive attitude towards EBP. Examples of questions from this subscale include, ‘My workload is too great for me to keep up-to-date with all the new evidence’ and ‘EBP is a waste of time.’ The subscale measuring knowledge/skills of EBP consists of 14 items with scores ranging from 14 to 98, with higher scores representing a higher level of knowledge/skills of EBP. Examples of items include, ‘Monitoring and reviewing of practice skills’ and ‘Ability to determine how useful (clinically applicable) the material is.’ Cronbach’s alpha levels for the three subscales were reported to be greater than 0.7, indicating good reliabilities (Upton & Upton, 2006). The total reliability of the EBPQ-K was 0.95 (Cronbach’s alpha) in our study. Critical thinking disposition scale The CTD scale, developed by Yoon (2004), was used to test the concurrent validity of the EBPQ in this study. The CTD scale consists of 27 items with a five-point Likert scale ranging from 1 (not at all true) to 5 (extremely true). Seven subcategories include systematicity, intellectual fairness, healthy scepticism, objectivity, intellectual eagerness/curiosity, prudence, and self-confidence (Yoon, 2008). A higher score indicates a higher level of CTD. The validity and reliability of the CTD were tested in several studies of samples of nursing students and nurses (Choi & Cho, 2011; Kim, 2012; Song, 2009). Cronbach’s alpha for the present study was 0.78. Data analysis Multiple methods of data analyses were performed to evaluate the EBPQ’s psychometric properties. Item analyses, including Cronbach’s alpha and the corrected item-total correlations were performed to test the reliability of subsample 1
© eContent Management Pty Ltd
CN
(n = 404). We checked the value of each itemtotal correlation to ensure that it was at least 0.30 (Nunnally & Bernstein, 1994) and eliminated those below 0.30. Next, we assessed sampling adequacy using the Kaiser–Meyer–Olkin (KMO) index and Bartlett’s test for EFA. The KMO index is a comparison of the magnitude of the measured correlation coefficients with the magnitude of the partial correlation coefficients (Nunnally & Bernstein, 1994). Values between 0.5 and 1.0 indicate that factor analysis is appropriate. Bartlett’s test of sphericity was also used to examine the hypothesis that the variables are uncorrelated in the population. That is, the population correlation matrix is an identity matrix (each variable correlates perfectly with itself [r = 1]; there is no correlation with the other variables [r = 0]). If the probability level (p) is greater than 0.05, factor analysis is not appropriate (Nunnally & Bernstein, 1994). Exploratory factor analysis and CFA were then used to assess the EBPQ’s construct validity. EFA was performed using principal axis factoring with a direct oblimin rotation in subsample 1, because this method effectively identifies the component parts of a scale according to the correlations between the factors (Costello & Osborne, 2005). Factors were initially extracted based on their eigenvalues (>1.0) and the scree plot (Thompson & Daniel, 1996). Consistent with Costello and Osborne’s (2005) guidelines, after rotation, we selected items with loadings above 0.32 and ensured that there were no items with cross-loadings of over 0.20 on more than two factors (Costello & Osborne, 2005). Confirmatory factor analysis was performed using the Analysis of Moment Structure (AMOS) program with subsample 2 to confirm the model fit the chosen factors. Chi-square goodness of fit (χ2/df ), the Normed Fit Index (NFI), the Standardised Root Mean Square Residual (SRMR), and the Comparative Fit Index (CFI) were used to estimate the model’s fit. The Chisquare Goodness of Fit test reveals the difference between the observed covariance matrix and the one predicted by the specified model. However, this index is sensitive to sample size; therefore, the relative goodness of fit indices, such as the NFI,
Volume 49, December 2014
CN
7
CN
Youn-Jung Son et al.
CFI, and SRMR, are strongly recommended (Bentler, 1990). A low Chi-square value that is not significant indicates that the hypothesised model is a good fit (Schaufeli, Salanova, González-romá, & Bakker, 2002). An acceptable fit of other indices are NFI ≥ 0.80, CFI ≥ 0.80, and SRMR ≤ 0.08 (Hu & Bentler, 1999). Finally, to test the concurrent validity of the EBPQ-K, Pearson’s correlation coefficient between the EBPQ-K and the CTD was calculated using the combined data set. The IBM SPSS Statistics 20 and SPSS AMOS 21 statistical software programs (SPSS Inc., Chicago, IL, USA) were used for data analyses and a significance level of 5% was applied. Ethical considerations The approval for Study 1 was obtained from the Institutional Review Board of Soonchunhyang University in South Korea (2011-10-04-02) and the Institutional Review Board of Chungnam National University’s College of Nursing approved Study 2 (2013–27). All participants provided written informed consent by mail prior to their participation. The nurses received a structured questionnaire, pen, and envelope with instructions to place the completed survey inside of it. They were instructed to seal the envelope to ensure that the survey information would be kept confidential when it was returned to the researchers. Results Demographics of participants Table 1 summarises the sample’s characteristics, including mean age, work experience (in months), gender, education level, religion, marital status, and RN position. Data from the total sample (N = 801) were randomly divided into subsample 1 (n = 404) for the EFA group and subsample 2 (n = 397) for the CFA group. The mean age of the EFA and CFA subsamples were 30.7 (SD = 7.32) and 30.7 (SD = 7.30), respectively. The majority of the nurses (about 98%) were female in both the samples (EFA and CFA) and the mean age of subsamples 1 and 2 were 30.7 and 30.5 years, respectively. 8
CN
Volume 49, December 2014
Table 1: Each subsample characteristics (N = 801) Characteristics
Subsample 1 Subsample 2 (n = 404) (n = 397)
Age (yrs): Mean ± SD Work experience (months): Mean ± SD Female, N (%) Education level, N (%) University degree Over master’s degree Having religion, N (%) Married, N (%) Position (RN), N (%)
30.7 ± 7.32 97.2 ± 88.98
30.7 ± 7.30 103.3 ± 88.49
395 (97.8)
392 (98.7)
291 (72.2) 113 (27.8) 198 (49.0) 133 (34.8) 318 (78.7)
300 (75.6) 97 (24.4) 208 (52.4) 135 (35.7) 298 (75.3)
Item analysis and reliability The results of the item-total score correlation showed that none of the items had a correlation of less than 0.3 among the 24 items; therefore, all of the items were retained. Scores on the itemtotal correlation of the 24 items ranged from 0.48 to 0.78. Cronbach’s alpha was used to confirm the internal consistency of the Korean version of the (EBPQ-K). Cronbach’s alpha for the 24-item EBPQ-K was 0.94. For the total study sample, Cronbach’s alphas for the three subscales measuring practice of, attitude towards, and knowledge/ skills of EBP were 0.92, 0.82, and 0.95, respectively (Table 2). Construct validity Exploratory factor analysis The KMO value was 0.94 and Bartlett’s test of sphericity was
