Original Research
Effectiveness of Test-Enhanced Learning in Continuing Health Sciences Education: A Randomized Controlled Trial
MEGHAN M. MCCONNELL, PHD; KHALID AZZAM, MBBS, FACP; TED XENODEMETROPOULOS, MD, MSC, FRCPC; AKBAR PANJU, MD, FRCPC Background: Increasing concerns over the effectiveness and quality of continuing medical education (CME) programs has encouraged educators to use theoretically driven empirical research to improve the educational value of these activities. Within cognitive psychology, theories of learning and knowledge acquisition, such as test-enhanced learning (TEL), may be used to enhance the effectiveness of CME protocols and delivery. The present study examined whether the pedagogical benefits of testing can be observed in practicing physicians. Methods: A total of 83 physicians were recruited following an education session on constipation management and were randomized to either (a) the test condition (n = 43), where learners wrote a short test consisting of 10 short-answer questions (SAQs), or (b) the study condition (n = 40), where learners studied the same information. Four weeks later, 56 (68%) physicians completed a final test with 10 new SAQs, with 27 being from the initial test condition and 29 belonging to the initial study condition. Results: Performance on the final SAQ test was equivalent for both test (42.5%) and study-only (41.2%) conditions (p = .71). Discussion: The null findings in the present study are inconsistent with previous research showing the pedagogical benefits of testing relative to studying. Given that most TEL research focuses on novice learners, who lack strong associative memory networks, it is possible that TEL is specific to novices and not generalizable to experts. Alternative explanations focus on the importance of repeated, distributed testing with feedback. Key Words: evaluation-educational intervention, experimental/quasi-experimental design, innovative educational intervention, assessment, learning, continuing medical education, testing effect, test-enhanced learning
Introduction Continuing medical education (CME) activities come in many different formats, such as didactic lectures, workshops, problem-based learning, reflective learning, and so forth. With such a wide range of learning methods, continuing medical educators are expected to determine the overall effectiveness of these CME activities and whether they are Disclosures: The authors report none. Dr. McConnell: Assistant Professor, McMaster University; Dr. Azzam: Associate Professor, McMaster University; Dr. Xenodemetropoulos: Assistant Professor, McMaster University; Dr. Panju: Professor, McMaster University. Correspondence: Meghan McConnell, McMaster University, MDCL 3510, Hamilton, ON; e-mail: [email protected]. © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education • Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.21293
associated with desired changes in physician knowledge, behaviours and practices.1 Recent calls for accountability and concerns regarding the effectiveness and quality of CME programs has promoted the use of theoretically driven empirical research to better understand how physicians learn. Within cognitive psychology, theories on learning and knowledge acquisition may be used to enhance the effectiveness of CME events. Indeed, nearly all of the core concepts in cognitive science (e.g., attention, memory, learning, problem solving, metacognition, etc.) are intimately involved in the educational processes. Over the past couple of decades, a large body of literature in cognitive psychology and related fields has shown that repeated testing is an extremely effective and powerful way to enhance learning,2–4 an effect commonly referred to as “testenhanced learning” (or TEL). The benefits of testing over studying on long-term retention of information are highly robust. Within medical education settings, TEL has been observed in a wide range of settings (eg, undergraduate5–7 and postgraduate4 learners) and with a variety of different test
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 35(2):119–122, 2015
McConnell et al.
materials (eg, quizzes,4,5 standardized patients,7 electronic flashcards6 ). The benefits of testing relative to studying on memory and learning are typically explained in the context of retrieval practice models. According to such models, learning relies on the ability to retrieve appropriate information from past experiences to solve the current problem at hand (for a recent review, see Karpicke and Grimaldi, 20128 ). In this way, studying and testing influence memory in different ways; while studying enhances memory acquisition (the “entering” of information into memory), testing enhances the retrieval process itself (the ability to access and “exit” or use information from memory). As an individual repeatedly retrieves information from memory, for example, through repeated testing, it is more likely that he or she will successfully retrieve such information in future contexts. It is important to note, however, that most of the research on TEL has been conducted on novice learners—that is, individuals who are in the process of developing novel knowledge and skills. If TEL enhances learning by increasing retrieval pathways in memory, then the effects may not be transferable to CME contexts, as practicing clinicians have had an abundance of natural practice retrieving relevant information (eg, in clinical settings). Therefore, the present study sought to examine whether TEL can be observed in practicing physicians within the context of a CME activity.
Methods Participants and Context The present study was run in conjunction with the 6th McMaster University Review Course in Internal Medicine. This is an annual CME course co-developed by the Department of Medicine and Continuing Health Sciences Education (CHSE) Program at McMaster. The target audience for this course includes general internists, subspecialists, ER physicians, hospitalists, family physicians, and general practitioners. The course spans over 3 days and is composed of 52 twenty-minute lectures. To examine TEL within a CME context, we selected a single lecture on constipation management. A total of 83 physicians were recruited following the education session on constipation management. Following an education session (described below), physicians were randomized to either (a) the test condition (n = 43), where learners wrote a short test consisting of 10 short answer questions (SAQs), or (b) the study condition (n = 40), where learners studied the same information. One month later, all participants were invited to complete a final test with 10 new SAQs. Of this initial pool of participants, 56 (68%) consented to participate in the second test, with 27 being from the initial test condition and 29 belonging to the initial study condition. 120
Participants who completed both the initial educational intervention (eg, quiz or study activity) and the final quiz received a $50 gift care as compensation. Materials The speaker (TX) of the aforementioned CME lecture on constipation management identified 10 learning objectives, which were used to generate educational materials for the present study. For each learning objective, 3 educational activities were created: ●
●
Two short answer questions (SAQs)—one used on the initial quiz and the other used on the final quiz activity. We opted to use SAQs, as these items have been shown to promote test-enhanced learning more so than questions in other formats (eg, true-false, multiple-choice questions4,8–10 ). One study item, which consisted of a detailed explanation of the correct answer and served as the control condition.
Procedure A between-subject design was used to investigate the effect of test-enhanced learning in practicing physicians. Participation took place during the lunch break. Upon entering the study room, participants were handed an envelope that contained either a short quiz (test condition) or a study sheet (study condition). Those in the test condition were asked to complete the quiz, while those in the study condition were asked to review the information provided on the study sheet. Individuals in the quiz condition were not given feedback on the accuracy of their responses. Individuals were informed that participation would involve an initial educational intervention, and 4 weeks later they would receive an e-mail that would direct them to an online quiz activity. Participants could withdrawal at any point of the experiment. To examine whether writing a short quiz promoted memory retention relative to studying the same material, 56 of the 83 total participants completed a short quiz consisting of 10 new SAQs 4–6 weeks after the CME event and initial quiz/study activity. These new SAQs were matched in content to the initial quiz/study material. The speaker of the event (TX) graded all quizzes. Results We conducted a between-subjects t-test to compare performance on the final quiz as a function of the initial educational activity (test condition vs study condition). The dependent variable is the proportions of correct answers on the final quiz activity. As illustrated in FIGURE 1, the present study did not find a significant testing effect. Mean proportion correct on the final quiz was equivalent for clinicians who were tested on
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
Test-Enhanced Learning
Lessons for Practice ●
●
FIGURE 1. Mean Proportion Correct Score on the Final Quiz as a Function of Initial Educational Activity
material presented during the CME event relative to those who studied the same material (t(54) = .374, p = .71). Discussion The present study sought to examine the potential benefits of testing on learning within the context of a CME event. While TEL has been observed in a variety of educational contexts, the present study did not observe TEL in practicing clinicians after a CME event. Such findings have theoretical and practical implications. Retrieval-based practice models postulate that testing enhances learning by promoting the active retrieval of information from memory.8 As an individual repeatedly retrieves information from memory, for example, through repeated testing, it is more likely that he or she will successfully retrieve such information in future contexts. Based on such theories, it is possible that the benefits of testing on learning is more pronounced on individuals learning new concepts and skills compared to individuals who are adding additional knowledge to an already extensive knowledge network, such as practicing clinicians. In other words, if TEL enhances memory and learning by increasing retrieval pathways in memory, the effects of TEL may not be as substantial in CME contexts, where participants have a lot of practice retrieving information in clinical settings. An alternative explanation of the null findings in the present research lies in the study design. Research has shown that the benefits of testing on long-term retention is proportional to the number tests taken, up to a point.11 Indeed, much of the TEL studies in medical education have used
It is important for educators to examine the potential benefits of test-enhanced learning in continuing education settings. The importance of feedback, repeated testing opportunities, and spacing of testing events all likely influence the effect of testing on practicing clinicians.
multiple tests.5–7 The results of the current study suggest that a single test may not be sufficient to promote long-term retention in practicing clinicians. Furthermore, research in experimental psychology has shown long-term retention of materials is enhanced when learning events are separated temporally from one another. For example, Cepeda et al12 found that long-term retention of information was enhanced when learning activities were spaced over days or weeks. Because our educational interventions (eg, initial quiz vs study material) were provided immediately following the CME activity, our design did not incorporate the important aspect of spacing in our design, which may account for our null findings. And, finally, it is important to point out that participants in the testing condition did not receive any feedback on their answers to the initial learning quiz. While testing alone has been shown to improve long-term retention, the provision of feedback greatly enhances TEL effects.13 Taken together, these findings illustrate the need for more research on the extent to which tests promote learning within CME contexts. The challenge for medical educators is to provide learning methods that promote long-term memory for information that can be readily accessed in multiple clinical settings. More TEL research is needed in the context of CME events, with an emphasis on incorporating important pedagogical principles such as repeated testing, spacing effects, and feedback. As data begin to accumulate regarding the effectiveness of testing in CME contexts, medical educators will be better informed as to whether CME events should be structured in a way that utilizes TEL effects, or whether the use of testing is not as beneficial as it is in other educational settings. To conclude, within the context of the present study, we failed to observe TEL effects within practicing clinicians following a CME event. However, more research is needed to examine the generalizability of such findings. Our findings demonstrate the importance of studying the educational methods within a variety of contexts to fully understand how to design learning activities that promote long-term retention of information in CME activities.
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
121
McConnell et al.
References 1. Mazmanian PE, Davis D. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057–1060. 2. McDaniel MA. Applying cognitive psychology to education. Psych Bull Rev. 2007;14:185–186. 3. Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long- term retention. Psych Sci. 2006;17:249–255. 4. Larsen DP, Butler AC, Roediger HL. Repeated testing improves longterm retention relative to repeated study: a randomized controlled trial. Med Educ. 2009;43:1174–1181. 5. McConnell MM, St-Onge C, Young ME. The benefits of testing for learning on later performance. Adv Health Sci Educ. 2014;1–16. 6. Schmidmaier R, Ebersbach R, Schiller M, Hege I, Holzer M, Fischer MR. Using electronic flashcards to promote learning in medical students: Retesting versus restudying. Med Educ. 2011;45:1101–1110. 7. Larsen DP, Bulter AC, Lawson AL, Roediger HL. The importance of seeing the patient: test-enhanced learning with standardized patients
122
8. 9. 10.
11.
12.
13.
and written tests improve clinical application of knowledge. Adv Health Sci Educ. 2013;18:409–425. Karpicke JD, Grimaldi, PJ. Retrieval-based learning: a perspective for enhancing meaningful learning. Educ Psychol Rev. 2012;24:401–418. Butler AC, Roediger HL. Testing improves long-term retention in a simulated classroom setting. Eur J Cog Psychol. 2007;19:514–527. Kang SHK, McDermott KB, Roediger HL. Test format and corrective feedback modulate the effect of testing on long-term retention. Eur J Cogn Psychol. 2007;19:528–558. Rawson KA, Dunlosky J. Optimizing schedules of retrieval practice for durable and efficient learning: how much is enough? J Exp Psychol Gen. 2011;140:283–302. Cepeda NJ, Coburn N, Rohrer D, Wixted JT, Mozer MC, Pashler H. Spacing effects in learning: a temporal ridgeline of optimal retention. Psych Sci. 2008;19:1095–1102. Butler AC, Karpicke JD, Roediger HL. Correcting a metacognitive error: feedback increases retention of low-confidence correct responses. J Exp Psychol Learn Mem Cogn. 2008;34:918–928.
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
Effectiveness of Test-Enhanced Learning in Continuing Health Sciences Education: A Randomized Controlled Trial
MEGHAN M. MCCONNELL, PHD; KHALID AZZAM, MBBS, FACP; TED XENODEMETROPOULOS, MD, MSC, FRCPC; AKBAR PANJU, MD, FRCPC Background: Increasing concerns over the effectiveness and quality of continuing medical education (CME) programs has encouraged educators to use theoretically driven empirical research to improve the educational value of these activities. Within cognitive psychology, theories of learning and knowledge acquisition, such as test-enhanced learning (TEL), may be used to enhance the effectiveness of CME protocols and delivery. The present study examined whether the pedagogical benefits of testing can be observed in practicing physicians. Methods: A total of 83 physicians were recruited following an education session on constipation management and were randomized to either (a) the test condition (n = 43), where learners wrote a short test consisting of 10 short-answer questions (SAQs), or (b) the study condition (n = 40), where learners studied the same information. Four weeks later, 56 (68%) physicians completed a final test with 10 new SAQs, with 27 being from the initial test condition and 29 belonging to the initial study condition. Results: Performance on the final SAQ test was equivalent for both test (42.5%) and study-only (41.2%) conditions (p = .71). Discussion: The null findings in the present study are inconsistent with previous research showing the pedagogical benefits of testing relative to studying. Given that most TEL research focuses on novice learners, who lack strong associative memory networks, it is possible that TEL is specific to novices and not generalizable to experts. Alternative explanations focus on the importance of repeated, distributed testing with feedback. Key Words: evaluation-educational intervention, experimental/quasi-experimental design, innovative educational intervention, assessment, learning, continuing medical education, testing effect, test-enhanced learning
Introduction Continuing medical education (CME) activities come in many different formats, such as didactic lectures, workshops, problem-based learning, reflective learning, and so forth. With such a wide range of learning methods, continuing medical educators are expected to determine the overall effectiveness of these CME activities and whether they are Disclosures: The authors report none. Dr. McConnell: Assistant Professor, McMaster University; Dr. Azzam: Associate Professor, McMaster University; Dr. Xenodemetropoulos: Assistant Professor, McMaster University; Dr. Panju: Professor, McMaster University. Correspondence: Meghan McConnell, McMaster University, MDCL 3510, Hamilton, ON; e-mail: [email protected]. © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education • Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.21293
associated with desired changes in physician knowledge, behaviours and practices.1 Recent calls for accountability and concerns regarding the effectiveness and quality of CME programs has promoted the use of theoretically driven empirical research to better understand how physicians learn. Within cognitive psychology, theories on learning and knowledge acquisition may be used to enhance the effectiveness of CME events. Indeed, nearly all of the core concepts in cognitive science (e.g., attention, memory, learning, problem solving, metacognition, etc.) are intimately involved in the educational processes. Over the past couple of decades, a large body of literature in cognitive psychology and related fields has shown that repeated testing is an extremely effective and powerful way to enhance learning,2–4 an effect commonly referred to as “testenhanced learning” (or TEL). The benefits of testing over studying on long-term retention of information are highly robust. Within medical education settings, TEL has been observed in a wide range of settings (eg, undergraduate5–7 and postgraduate4 learners) and with a variety of different test
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 35(2):119–122, 2015
McConnell et al.
materials (eg, quizzes,4,5 standardized patients,7 electronic flashcards6 ). The benefits of testing relative to studying on memory and learning are typically explained in the context of retrieval practice models. According to such models, learning relies on the ability to retrieve appropriate information from past experiences to solve the current problem at hand (for a recent review, see Karpicke and Grimaldi, 20128 ). In this way, studying and testing influence memory in different ways; while studying enhances memory acquisition (the “entering” of information into memory), testing enhances the retrieval process itself (the ability to access and “exit” or use information from memory). As an individual repeatedly retrieves information from memory, for example, through repeated testing, it is more likely that he or she will successfully retrieve such information in future contexts. It is important to note, however, that most of the research on TEL has been conducted on novice learners—that is, individuals who are in the process of developing novel knowledge and skills. If TEL enhances learning by increasing retrieval pathways in memory, then the effects may not be transferable to CME contexts, as practicing clinicians have had an abundance of natural practice retrieving relevant information (eg, in clinical settings). Therefore, the present study sought to examine whether TEL can be observed in practicing physicians within the context of a CME activity.
Methods Participants and Context The present study was run in conjunction with the 6th McMaster University Review Course in Internal Medicine. This is an annual CME course co-developed by the Department of Medicine and Continuing Health Sciences Education (CHSE) Program at McMaster. The target audience for this course includes general internists, subspecialists, ER physicians, hospitalists, family physicians, and general practitioners. The course spans over 3 days and is composed of 52 twenty-minute lectures. To examine TEL within a CME context, we selected a single lecture on constipation management. A total of 83 physicians were recruited following the education session on constipation management. Following an education session (described below), physicians were randomized to either (a) the test condition (n = 43), where learners wrote a short test consisting of 10 short answer questions (SAQs), or (b) the study condition (n = 40), where learners studied the same information. One month later, all participants were invited to complete a final test with 10 new SAQs. Of this initial pool of participants, 56 (68%) consented to participate in the second test, with 27 being from the initial test condition and 29 belonging to the initial study condition. 120
Participants who completed both the initial educational intervention (eg, quiz or study activity) and the final quiz received a $50 gift care as compensation. Materials The speaker (TX) of the aforementioned CME lecture on constipation management identified 10 learning objectives, which were used to generate educational materials for the present study. For each learning objective, 3 educational activities were created: ●
●
Two short answer questions (SAQs)—one used on the initial quiz and the other used on the final quiz activity. We opted to use SAQs, as these items have been shown to promote test-enhanced learning more so than questions in other formats (eg, true-false, multiple-choice questions4,8–10 ). One study item, which consisted of a detailed explanation of the correct answer and served as the control condition.
Procedure A between-subject design was used to investigate the effect of test-enhanced learning in practicing physicians. Participation took place during the lunch break. Upon entering the study room, participants were handed an envelope that contained either a short quiz (test condition) or a study sheet (study condition). Those in the test condition were asked to complete the quiz, while those in the study condition were asked to review the information provided on the study sheet. Individuals in the quiz condition were not given feedback on the accuracy of their responses. Individuals were informed that participation would involve an initial educational intervention, and 4 weeks later they would receive an e-mail that would direct them to an online quiz activity. Participants could withdrawal at any point of the experiment. To examine whether writing a short quiz promoted memory retention relative to studying the same material, 56 of the 83 total participants completed a short quiz consisting of 10 new SAQs 4–6 weeks after the CME event and initial quiz/study activity. These new SAQs were matched in content to the initial quiz/study material. The speaker of the event (TX) graded all quizzes. Results We conducted a between-subjects t-test to compare performance on the final quiz as a function of the initial educational activity (test condition vs study condition). The dependent variable is the proportions of correct answers on the final quiz activity. As illustrated in FIGURE 1, the present study did not find a significant testing effect. Mean proportion correct on the final quiz was equivalent for clinicians who were tested on
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
Test-Enhanced Learning
Lessons for Practice ●
●
FIGURE 1. Mean Proportion Correct Score on the Final Quiz as a Function of Initial Educational Activity
material presented during the CME event relative to those who studied the same material (t(54) = .374, p = .71). Discussion The present study sought to examine the potential benefits of testing on learning within the context of a CME event. While TEL has been observed in a variety of educational contexts, the present study did not observe TEL in practicing clinicians after a CME event. Such findings have theoretical and practical implications. Retrieval-based practice models postulate that testing enhances learning by promoting the active retrieval of information from memory.8 As an individual repeatedly retrieves information from memory, for example, through repeated testing, it is more likely that he or she will successfully retrieve such information in future contexts. Based on such theories, it is possible that the benefits of testing on learning is more pronounced on individuals learning new concepts and skills compared to individuals who are adding additional knowledge to an already extensive knowledge network, such as practicing clinicians. In other words, if TEL enhances memory and learning by increasing retrieval pathways in memory, the effects of TEL may not be as substantial in CME contexts, where participants have a lot of practice retrieving information in clinical settings. An alternative explanation of the null findings in the present research lies in the study design. Research has shown that the benefits of testing on long-term retention is proportional to the number tests taken, up to a point.11 Indeed, much of the TEL studies in medical education have used
It is important for educators to examine the potential benefits of test-enhanced learning in continuing education settings. The importance of feedback, repeated testing opportunities, and spacing of testing events all likely influence the effect of testing on practicing clinicians.
multiple tests.5–7 The results of the current study suggest that a single test may not be sufficient to promote long-term retention in practicing clinicians. Furthermore, research in experimental psychology has shown long-term retention of materials is enhanced when learning events are separated temporally from one another. For example, Cepeda et al12 found that long-term retention of information was enhanced when learning activities were spaced over days or weeks. Because our educational interventions (eg, initial quiz vs study material) were provided immediately following the CME activity, our design did not incorporate the important aspect of spacing in our design, which may account for our null findings. And, finally, it is important to point out that participants in the testing condition did not receive any feedback on their answers to the initial learning quiz. While testing alone has been shown to improve long-term retention, the provision of feedback greatly enhances TEL effects.13 Taken together, these findings illustrate the need for more research on the extent to which tests promote learning within CME contexts. The challenge for medical educators is to provide learning methods that promote long-term memory for information that can be readily accessed in multiple clinical settings. More TEL research is needed in the context of CME events, with an emphasis on incorporating important pedagogical principles such as repeated testing, spacing effects, and feedback. As data begin to accumulate regarding the effectiveness of testing in CME contexts, medical educators will be better informed as to whether CME events should be structured in a way that utilizes TEL effects, or whether the use of testing is not as beneficial as it is in other educational settings. To conclude, within the context of the present study, we failed to observe TEL effects within practicing clinicians following a CME event. However, more research is needed to examine the generalizability of such findings. Our findings demonstrate the importance of studying the educational methods within a variety of contexts to fully understand how to design learning activities that promote long-term retention of information in CME activities.
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
121
McConnell et al.
References 1. Mazmanian PE, Davis D. Continuing medical education and the physician as a learner: guide to the evidence. JAMA. 2002;288:1057–1060. 2. McDaniel MA. Applying cognitive psychology to education. Psych Bull Rev. 2007;14:185–186. 3. Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long- term retention. Psych Sci. 2006;17:249–255. 4. Larsen DP, Butler AC, Roediger HL. Repeated testing improves longterm retention relative to repeated study: a randomized controlled trial. Med Educ. 2009;43:1174–1181. 5. McConnell MM, St-Onge C, Young ME. The benefits of testing for learning on later performance. Adv Health Sci Educ. 2014;1–16. 6. Schmidmaier R, Ebersbach R, Schiller M, Hege I, Holzer M, Fischer MR. Using electronic flashcards to promote learning in medical students: Retesting versus restudying. Med Educ. 2011;45:1101–1110. 7. Larsen DP, Bulter AC, Lawson AL, Roediger HL. The importance of seeing the patient: test-enhanced learning with standardized patients
122
8. 9. 10.
11.
12.
13.
and written tests improve clinical application of knowledge. Adv Health Sci Educ. 2013;18:409–425. Karpicke JD, Grimaldi, PJ. Retrieval-based learning: a perspective for enhancing meaningful learning. Educ Psychol Rev. 2012;24:401–418. Butler AC, Roediger HL. Testing improves long-term retention in a simulated classroom setting. Eur J Cog Psychol. 2007;19:514–527. Kang SHK, McDermott KB, Roediger HL. Test format and corrective feedback modulate the effect of testing on long-term retention. Eur J Cogn Psychol. 2007;19:528–558. Rawson KA, Dunlosky J. Optimizing schedules of retrieval practice for durable and efficient learning: how much is enough? J Exp Psychol Gen. 2011;140:283–302. Cepeda NJ, Coburn N, Rohrer D, Wixted JT, Mozer MC, Pashler H. Spacing effects in learning: a temporal ridgeline of optimal retention. Psych Sci. 2008;19:1095–1102. Butler AC, Karpicke JD, Roediger HL. Correcting a metacognitive error: feedback increases retention of low-confidence correct responses. J Exp Psychol Learn Mem Cogn. 2008;34:918–928.
JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp
