Teaching EBP Column
Teaching Systematic Searching in a Baccalaureate Nursing Research Course Kimberly J. Whalen, MLIS • Suzanne E. Zentz, DNP, RN, CNE
This column shares the best evidence-based strategies and innovative ideas on how to facilitate the learning of EBP principles and processes by clinicians as well as nursing and interprofessional students. Guidelines for submission are available at http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1741-6787
BACKGROUND Graduates of baccalaureate nursing programs must be able to effectively identify, analyze, and synthesize evidence (American Association of Colleges of Nursing, 2008). In the context of nursing, these skills are considered foundational steps in the evidence-based practice (EBP) process. Although numerous EBP models exist, the basic process is consistent: (Step 0) ignite a spirit of inquiry, (Step 1) identify a PICO(T) clinical question, (Step 2) search for relevant evidence, (Step 3) critically appraise the evidence, (Step 4) integrate the evidence with a clinician’s expertise and a patient’s preferences and values, then implement evidence into practice, (Step 5) evaluate the outcomes, and (Step 6) disseminate the outcomes (Melnyk & Fineout-Overholt, 2014). Each step in the process is essential. When any step is improperly performed, the outcome of the entire EBP process is negatively affected. To successfully perform Step 2 of the EBP process, students must be able to develop keywords and subject headings, select appropriate information resources, apply search limiters, and navigate information resource interfaces. Most undergraduate students have grown up with easy access to information technology and an abundance of information sources. Students are comfortable typing a few words into a search engine on their phone, laptop, or computer and receiving thousands of search results. However, research suggests that students are often overconfident in their ability to search for and access the most appropriate and relevant information needed for their work (Denison & Montgomery, 2012; Holman, 2011). Nursing students and nurses struggle with selecting keywords and subject headings, combining search terms, and using limiters (Carlock & Anderson, 2007; Fineout-Overholt, Hofstetter, Shell, & Johnston, 2005; Jacobsen & Andenaes, 2011; Klem & Weiss, 2005). A worksheet or template that combines an outline of the search process with guided questions to promote reflection
246
can be a useful tool to facilitate the systematic search process of novice searchers. Craven and Levay (2011) focus on the importance of recording database search strategies for systematic reviews. They recommend a template to document the name of the database or website, database platform, version, date searched, fields utilized, and number of search results. In addition to completing the worksheet, they recommend searchers include a paragraph description that explains the changes made during the search process. The authors believe that utilizing this strategy not only records the location of evidence, but also documents the rationale for the search techniques used and evidence selected. A systematic review of reflective practice in health education by Mann, Gordon, and MacLeod (2009) found that when used as a learning strategy, reflection connects and integrates new learning into existing knowledge and skills. As part of the requirements for an introductory nursing research or EBP course within a BSN program, senior-level nursing students completed an EBP group project. The purpose of the project was to explore evidence available regarding a clinical problem. Groups performed a systematic search of the literature addressing an assigned clinical problem, selected relevant evidence, analyzed the evidence, and compiled an evidence summary. Groups were taught to initially search for higher levels of evidence such as systematic reviews and clinical practice guidelines, before moving on to lower levels of evidence such as single studies and expert opinions. To enhance EBP skills, a nursing professor and health sciences librarian collaborated to develop a tool aimed at facilitating development of systematic searching skills.
STRATEGY Beginning in the spring of 2012, students were provided a worksheet which included an outline of the systematic search process and reflective questions (see Figure 1). Initially, groups Worldviews on Evidence-Based Nursing, 2015; 12:4, 246–248. C 2015 Sigma Theta Tau International
Teaching EBP Column A systematic search of the literature requires coordination among group members. Though individuals can start the process with their own preliminary search, it is important for everyone in the group to use the same PICOT, keywords, subject headings, and limiters in the final group search. A systematic search consistent among group members will help identify the best available evidence for your project. Step 1 1. 2. 3. 4. Step 2 1. 2. 3.
Draft PICOT based on clinical problem. What are appropriate keywords and subject headings? What limiters (e.g., date range, language) are appropriate? What databases/resources will be preliminarily searched and who will search each? Regroup after an initial search is completed. Evaluate preliminary evidence located. Assess the databases and resources searched. Which were useful? Which were not? Assess keywords, subject headings, and limiters used. Which were useful? Which were not?
Step 3 1. Refine PICOT question based on preliminary evidence. 2. As a group, decide keywords, subject headings, and limiters for group’s final search. 3. Organize keywords, subject headings, and limiters in a final search. Identify appropriate Boolean Operators (AND, OR, NOT). Should search terms or subject headings be limited within a title, abstract, subject, or all text? Step 4 1. As a group, decide which databases/resources will be searched for final search. 2. As a group, decide who will search each database/resource. Run finalized searches in databases/resources selected. Step 5 1. Document databases/resources searched and number of results found for each source. Keep in mind that there will be an overlap of identified results in different databases. Review the titles and abstracts of results to determine which full-text to retrieve, read, and appraise. Not all of the articles retrieved and read will be used in the final analysis. 2. Note why evidence was not selected for final analysis. Generalized comments based on the inclusion and exclusion criteria established by the group are appropriate. 3. Note why evidence was selected for final analysis. List specific reasons for each piece of evidence selected. 4. Document the final databases and resources searched and the final results selected from each for the evidence summary. 5. Document what was learned by group members during the search process. What were the difficulties? How were they resolved? 6. Document what searchers will do differently next time.
Figure 1. Systematic Search Worksheet. were instructed to document their group’s search process on the worksheet. As part of their final EBP project, students formally typed up and submitted their group’s systematic search process as a research log. Research logs described the group’s systematic search noting databases, keywords, subject headings, limiters, and database search results. Additionally, rationale for evidence selection, explanation of difficulties encountered during the search process, and suggested adjustments for subsequent searches were included.
METHODS USED TO EVALUATE OUTCOMES To determine the effect of implementing systematic worksheets and research logs on students’ EBP projects, students’ evidence summary scores over four semesters were examined. Student groups selected, analyzed, and presented evidence in table format. These tables were labeled evidence summaries. Worldviews on Evidence-Based Nursing, 2015; 12:4, 246–248. C 2015 Sigma Theta Tau International
Evidence summary scores of student groups from Fall 2011 (prior to implementation of worksheets and research logs) were used as baseline data and compared to evidence summary scores of student groups from subsequent terms (Spring 2012, Fall 2012, and Spring 2013) when systematic search worksheets and research logs were implemented using one-way ANOVA.
RESULTS Over 250 students comprising 39 EBP project groups were compared across four terms. Univariate ANOVA indicated that evidence summary scores were significantly improved by use of the worksheets and logs (F(3.35) = 14.116, p = .000) (see Table 1.) Tukey’s HSD was used to determine the nature of differences among groups by term. Students in Fall 2011 who did not use systematic search worksheets and research logs scored significantly lower on evidence summaries than students in Spring 2012, Fall 2012, and Spring 2013.
247
Teaching Systematic Searching
Table 1. ANOVA Results of Evidence Summary Scores
Sample size groups (students)
M (SD)
Fall 2011
12(66)
62.5(11.3)
Spring 2012
8(49)
77.8(8.8)
Fall 2012
9(62)
85.1(6.2)
Spring 2013
10(76)
83.8(8.7)
Total
39(253)
df
F
p value
3
14.116*
.000
*p
Teaching Systematic Searching in a Baccalaureate Nursing Research Course Kimberly J. Whalen, MLIS • Suzanne E. Zentz, DNP, RN, CNE
This column shares the best evidence-based strategies and innovative ideas on how to facilitate the learning of EBP principles and processes by clinicians as well as nursing and interprofessional students. Guidelines for submission are available at http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1741-6787
BACKGROUND Graduates of baccalaureate nursing programs must be able to effectively identify, analyze, and synthesize evidence (American Association of Colleges of Nursing, 2008). In the context of nursing, these skills are considered foundational steps in the evidence-based practice (EBP) process. Although numerous EBP models exist, the basic process is consistent: (Step 0) ignite a spirit of inquiry, (Step 1) identify a PICO(T) clinical question, (Step 2) search for relevant evidence, (Step 3) critically appraise the evidence, (Step 4) integrate the evidence with a clinician’s expertise and a patient’s preferences and values, then implement evidence into practice, (Step 5) evaluate the outcomes, and (Step 6) disseminate the outcomes (Melnyk & Fineout-Overholt, 2014). Each step in the process is essential. When any step is improperly performed, the outcome of the entire EBP process is negatively affected. To successfully perform Step 2 of the EBP process, students must be able to develop keywords and subject headings, select appropriate information resources, apply search limiters, and navigate information resource interfaces. Most undergraduate students have grown up with easy access to information technology and an abundance of information sources. Students are comfortable typing a few words into a search engine on their phone, laptop, or computer and receiving thousands of search results. However, research suggests that students are often overconfident in their ability to search for and access the most appropriate and relevant information needed for their work (Denison & Montgomery, 2012; Holman, 2011). Nursing students and nurses struggle with selecting keywords and subject headings, combining search terms, and using limiters (Carlock & Anderson, 2007; Fineout-Overholt, Hofstetter, Shell, & Johnston, 2005; Jacobsen & Andenaes, 2011; Klem & Weiss, 2005). A worksheet or template that combines an outline of the search process with guided questions to promote reflection
246
can be a useful tool to facilitate the systematic search process of novice searchers. Craven and Levay (2011) focus on the importance of recording database search strategies for systematic reviews. They recommend a template to document the name of the database or website, database platform, version, date searched, fields utilized, and number of search results. In addition to completing the worksheet, they recommend searchers include a paragraph description that explains the changes made during the search process. The authors believe that utilizing this strategy not only records the location of evidence, but also documents the rationale for the search techniques used and evidence selected. A systematic review of reflective practice in health education by Mann, Gordon, and MacLeod (2009) found that when used as a learning strategy, reflection connects and integrates new learning into existing knowledge and skills. As part of the requirements for an introductory nursing research or EBP course within a BSN program, senior-level nursing students completed an EBP group project. The purpose of the project was to explore evidence available regarding a clinical problem. Groups performed a systematic search of the literature addressing an assigned clinical problem, selected relevant evidence, analyzed the evidence, and compiled an evidence summary. Groups were taught to initially search for higher levels of evidence such as systematic reviews and clinical practice guidelines, before moving on to lower levels of evidence such as single studies and expert opinions. To enhance EBP skills, a nursing professor and health sciences librarian collaborated to develop a tool aimed at facilitating development of systematic searching skills.
STRATEGY Beginning in the spring of 2012, students were provided a worksheet which included an outline of the systematic search process and reflective questions (see Figure 1). Initially, groups Worldviews on Evidence-Based Nursing, 2015; 12:4, 246–248. C 2015 Sigma Theta Tau International
Teaching EBP Column A systematic search of the literature requires coordination among group members. Though individuals can start the process with their own preliminary search, it is important for everyone in the group to use the same PICOT, keywords, subject headings, and limiters in the final group search. A systematic search consistent among group members will help identify the best available evidence for your project. Step 1 1. 2. 3. 4. Step 2 1. 2. 3.
Draft PICOT based on clinical problem. What are appropriate keywords and subject headings? What limiters (e.g., date range, language) are appropriate? What databases/resources will be preliminarily searched and who will search each? Regroup after an initial search is completed. Evaluate preliminary evidence located. Assess the databases and resources searched. Which were useful? Which were not? Assess keywords, subject headings, and limiters used. Which were useful? Which were not?
Step 3 1. Refine PICOT question based on preliminary evidence. 2. As a group, decide keywords, subject headings, and limiters for group’s final search. 3. Organize keywords, subject headings, and limiters in a final search. Identify appropriate Boolean Operators (AND, OR, NOT). Should search terms or subject headings be limited within a title, abstract, subject, or all text? Step 4 1. As a group, decide which databases/resources will be searched for final search. 2. As a group, decide who will search each database/resource. Run finalized searches in databases/resources selected. Step 5 1. Document databases/resources searched and number of results found for each source. Keep in mind that there will be an overlap of identified results in different databases. Review the titles and abstracts of results to determine which full-text to retrieve, read, and appraise. Not all of the articles retrieved and read will be used in the final analysis. 2. Note why evidence was not selected for final analysis. Generalized comments based on the inclusion and exclusion criteria established by the group are appropriate. 3. Note why evidence was selected for final analysis. List specific reasons for each piece of evidence selected. 4. Document the final databases and resources searched and the final results selected from each for the evidence summary. 5. Document what was learned by group members during the search process. What were the difficulties? How were they resolved? 6. Document what searchers will do differently next time.
Figure 1. Systematic Search Worksheet. were instructed to document their group’s search process on the worksheet. As part of their final EBP project, students formally typed up and submitted their group’s systematic search process as a research log. Research logs described the group’s systematic search noting databases, keywords, subject headings, limiters, and database search results. Additionally, rationale for evidence selection, explanation of difficulties encountered during the search process, and suggested adjustments for subsequent searches were included.
METHODS USED TO EVALUATE OUTCOMES To determine the effect of implementing systematic worksheets and research logs on students’ EBP projects, students’ evidence summary scores over four semesters were examined. Student groups selected, analyzed, and presented evidence in table format. These tables were labeled evidence summaries. Worldviews on Evidence-Based Nursing, 2015; 12:4, 246–248. C 2015 Sigma Theta Tau International
Evidence summary scores of student groups from Fall 2011 (prior to implementation of worksheets and research logs) were used as baseline data and compared to evidence summary scores of student groups from subsequent terms (Spring 2012, Fall 2012, and Spring 2013) when systematic search worksheets and research logs were implemented using one-way ANOVA.
RESULTS Over 250 students comprising 39 EBP project groups were compared across four terms. Univariate ANOVA indicated that evidence summary scores were significantly improved by use of the worksheets and logs (F(3.35) = 14.116, p = .000) (see Table 1.) Tukey’s HSD was used to determine the nature of differences among groups by term. Students in Fall 2011 who did not use systematic search worksheets and research logs scored significantly lower on evidence summaries than students in Spring 2012, Fall 2012, and Spring 2013.
247
Teaching Systematic Searching
Table 1. ANOVA Results of Evidence Summary Scores
Sample size groups (students)
M (SD)
Fall 2011
12(66)
62.5(11.3)
Spring 2012
8(49)
77.8(8.8)
Fall 2012
9(62)
85.1(6.2)
Spring 2013
10(76)
83.8(8.7)
Total
39(253)
df
F
p value
3
14.116*
.000
*p
