Clinical Trials http://ctj.sagepub.com/
Clinician-trialist rounds: 24. Modernizing your introductory graduate course in clinical trials. Part 1: Commonsense meets evidence Geoffrey R Norman and David L Sackett Clin Trials 2014 11: 681 originally published online 16 July 2014 DOI: 10.1177/1740774514542622 The online version of this article can be found at: http://ctj.sagepub.com/content/11/6/681
Published by: http://www.sagepublications.com
On behalf of:
The Society for Clinical Trials
Additional services and information for Clinical Trials can be found at: Email Alerts: http://ctj.sagepub.com/cgi/alerts Subscriptions: http://ctj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Nov 10, 2014 OnlineFirst Version of Record - Jul 16, 2014 What is This?
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
CLINICAL TRIALS
Column
Clinician-trialist rounds: 24. Modernizing your introductory graduate course in clinical trials. Part 1: Commonsense meets evidence
Clinical Trials 2014, Vol. 11(6) 681–684 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514542622 ctj.sagepub.com
Geoffrey R Norman1* and David L Sackett2
Today’s case You have become a highly successful mid-career clinician-trialist: promoted and tenured, with an enviable publication record, an international reputation, a great team of research staff and students and mentees, a supportive clinical team, several clinical teaching awards, and a work-life balance that is the envy of all. You didn’t accomplish this entirely on your own, but thanks also to great mentoring, rigorous time-management, and saying ‘‘no’’ to the myriad dys-opportunities that could have consumed too much of your time and talents. Taking stock, you decide it’s time for some payback, and are looking for opportunities to redirect some of your time, energy, and skills to serve the graduate programme that gave you your start. As luck would have it, the introductory course in RCTs is looking for a new director, and once you’ve determined that the scutwork of finding rooms and tutors will remain in a staff member’s capable hands, you offer to take it on. Determined to make it the flagship of the graduate program, you begin by setting down four ‘‘commonsense axioms for teaching and learning’’ based on your experiences as both student and teacher, supplemented by reading a couple of popular books about education: 1.
The course will recognize that individual students have different approaches to learning, and take these individual learning styles into account. You’ll use lots of graphics and flow charts for visual learners, and point-form step-by-step details for verbal learners, conceptual explanations for conceptual learners and examples for practical learners. You’ll take advantage of the state of the art in visual graphics,
*Dr Norman is a Cognitive Psychologist, a Professor in the Department of Clinical Epidemiology & Biostatistics, and an Associate Member of the Department of Psychology, all at McMaster University in Hamilton Ontario.
2.
3.
4.
using dynamic presentations and virtual reality to engage the learners. Recognizing that ‘‘core-knowledge’’ about trial methodology is a rapidly changing field, this course will deliberately favor the acquisition of critical thinking skills over current core knowledge. Instead of overloading students with a bunch of multiple choice tests, the students will be evaluated in part by self-directed ‘‘reflection’’ exercises, so they can learn how to test themselves, and become successful lifelong learners about RCTs as well as keeping their clinical skills up to date. The course will capitalize on the highly effective multi-tasking skills of todays’ learners (e.g. Wi-Fi in every class and seminar room will permit simultaneous access to readings and slideshows) so that learning will be more effective and efficient.
You present your proposal to the departmental education committee, anticipating their shared excitement, praise and hearty endorsement. Instead, they begin to chuckle over your ‘‘axioms for teaching and learning’’ and one of them, an educational psychologist, gently tells you that your axioms possess roughly the same validity as the 1950s treatment of heart attacks (5 weeks of
1
Program for Educational Research and Development, McMaster University, Hamilton, ON, Canada 2 Trout Research & Education Centre at Irish Lake, Markdale, ON, Canada Corresponding author: Geoffrey R Norman, Program for Educational Research and Development, McMaster University, Room 3519, MDCL, 1200 Main St. West, Hamilton, ON, L8N 3Z5, Canada. Email: [email protected] David L Sackett, Trout Research & Education Centre at Irish Lake, RR 1, Markdale, ON, N0C 1H0, Canada. Email: [email protected]
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
682
Clinical Trials 11(6)
absolute bed rest) and the 1960s treatment of peptic ulcer (gastric freezing). Scrambling to recover lost ground, you exclaim that at least your PowerPoints will be up-to-date with the latest in catchy background music, animation, and eye-catching action. Alas, more chuckles. Never a quitter (‘tho sorely tempted here), you humbly ask for an ‘‘educational consultation’’ to learn why and how your commonsense proposals aren’t sensible after all.
Discussion The proposals from our mid-career trialist colleague could be forgiven; much of current educational practice does result from ideas that are perfectly commonsensical. Alas, it is not based on sound evidence. As Roediger1 puts it: The field of education seems particularly susceptible to the allure of plausible but untested ideas and fads (especially ones that are lucrative for their inventors). One could write an interesting history of ideas based on either plausible theory or somewhat flimsy research that have come and gone over the years. And . once an idea takes hold, it is hard to root out.
This sad state didn’t arise because educators are thicker than the rest of us; it’s simply because the history of science is replete with ideas that made perfect sense, but when subjected to empirical research, just didn’t hold up. And that’s just as true for learning styles and self-assessment as it was for freezing the acid-producing cells of ulcerated stomachs and keeping heart attacks flat on their backs until their infarcts were bridged by stout fibrous tissue. In fact, the situation may be even worse in education, for a couple of reasons. First, while we wouldn’t invoke common sense solutions to solving the mysteries of quantum mechanics or non-chromosomal inheritance, all of us have had years of experience being taught and trying to learn, so we feel authorized to offer strong opinions on what works, what doesn’t, and how and why. Second, although the history of formal medical research extends back several centuries, formal research in education and in its basic science, cognitive psychology (more on this in a moment), is a much more recent endeavor. The acknowledged founder of medical education research was George Miller, who was a teacher of Dave Sackett’s at the U of Illinois, which shows how recent it is. Cognitive psychology, the branch of psychology that studies processes like perception, learning, problem-solving also had its origins in the 1960s, although scientific psychology is much older.
A brief history of the ‘‘science of learning’’ Before about 1920, there were the widely accepted educational philosophies of John Dewey (‘‘experiential learning’’) and Edward Thorndike (‘‘active learning’’), but not much experimental evidence that they worked. In this era in psychology, phrenology (reading personality from bumps on the head) and psychoanalysis had equal claims to legitimacy.y Then along came behaviorism (knowledge is the product of stimulus/response conditioning), which abandoned studies of people in favor of rats and pigeons, and was epitomized by the mid-century work of B. F. Skinner in its attempt to put psychology on a scientific footing.z Behaviourism spawned ‘‘behavioral objectives’’ (what the student was expected to be able to do as a result of learning) which reduced learning to mastering hundreds and hundreds of little goals like ‘‘Be able to list 6 causes of anemia.’’ The 1960s spelled the end of this foolishness,§ and none too soon. Society was changing around them, and educators rebelled at the mindless, paternalistic, mechanistic approaches dictated by behaviorism. It was time to return to authentic experience and skill development as a basis for learning and assessment. Illich’s De-Schooling Society2 and Knowles’s The Adult Learner—A Neglected Species3 challenged these teacher-focused approaches and encouraged educators to refocus their efforts on the learner, not the teacher. In general, it was a time to celebrate individuality, in society and in education. In medical education, the sixties brought a radical new approach to undergraduate education (pioneered, we are pleased to say, by McMaster) that focused of acquiring knowledge by solving problems. Instead of mastering long lists of facts, students were encouraged to learn the concepts so they could use them subsequently to solve problems. The educators who devised these revolutionary approaches correctly rejected what passed for the science of learning in those days. However this parting of the ways between educators and psychologists is now in need of reconciliation, because behaviorism died in psychology about the same time as it did in education. The new discipline, cognitive psychology, rejected the proposition that all learning amounted to learning simple associations, and realized that much of human learning—from perception to motor skills—was and should be an object of deeper experimental scientific scrutiny. And in the course of studying these processes, y
To be fair, Osler’s contemporaneous Textbook of Medicine was still recommending blood-letting for pneumonia. z Internal mammary artery ligation for angina pectoris reached its peak at about this same time. § And also delivered the RCT showing for the first time that the treatment of diastolic blood pressures between 115 and 129 did more good than harm.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
Norman and Sackett
683
psychologists have identified a number of highly effective teaching strategies.
So why is the proposed course so very wrong? 1.
Today’s ‘‘science of learning’’ Cognitive psychological research has revealed that we initiate ‘‘thinking’’ by using our ‘‘short-term’’ or ‘‘working memory’’ (WM) which, although of very limited capacity (literally one byte), can access a long-term memory (LTM) of almost infinite capacity (greater even than that of an external hard drive) with incredible speed. It is the changes in our LTM that constitute learning. For example, think about solving these two problems: 1.
2.
What is 12 x 12? Most of our brains can access our LTM and generate the correct answer to this problem in about 200 milliseconds. Google can solve it in about 190 milliseconds, too, but only after searching through over 25 billion possible answers. What is 17 x 17? Interestingly, we know that we don’t know the answer to this problem in about those same 200 milliseconds. We clearly can’t have sequentially searched the sum of all LTM and come up dry. Accordingly, our brain’s LTM must be qualitatively different from the sum of all WM.
Cognitive psychologists characterize this qualitative difference by inserting an important adjective into the term. It’s called Long Term Associative Memory (LTAM), in recognition that the basic process is an association between new information and what is already in long term memory. As Bjork and his colleagues say . we do not store information in our LTM by making any kind of literal recording of that information, but, instead, we do so by relating new information to what we already know. We store new information in terms of its meaning to us, as defined by its relationships and semantic associations to information that already exists in our long term associative memory.4
These two simple, yet powerful ideas, short-term working memory (WM) and long-term associative memory (LTAM) lead to some very profound insights about teaching and learning which in turn lead to very powerful teaching interventions. And we do mean powerful; many of these interventions can increase learning by up to 50% with no increase in instructional or learning time! That is, teach it one way and the average mark on the final is 60%; teach it using some of these strategies and it’s 90%. Or in the language of systematic reviews, effect sizes** of around 1.0 are the norm.5
**Effect size here = (Treatment Effect) divided by its (Standard Deviation).
2.
3. 4.
5.
6.
While it’s easy to measure individual differences in things like learning style, extraversion, conscientiousness (and height and weight for that matter), these have very little to do with the changes in the LTAM that constitutes learning. Despite our intuitions, the major determinant of learning is what is to be learned, not who is doing the learning.6 General skills like problem-solving or critical thinking are very weak predictors of performance.7,8 In medicine, although we can measure ‘‘skills’’ like history-taking or clinical reasoning reliably with tools like the Objective Structured Clinical Examination, they do not generalize from one case to another (hence are not skill-like),9 and do not predict performance in practice.10,11 Skills like self-assessment, meta-cognition or reflection have no relation to actual performance.12,13 Periodic well-designed tests, instead of interfering with learning, can actually enhance learning by encouraging students to repeatedly retrieve information from LTAM.14 Given the serious size limit of short-term WM, all those whistles and bells in your powerpoint will have a large negative impact on learning (that is, on converting short-term WM to LTAM). Finally, surprise surprise. Today’s kids are using the same brain as we are. Evolution doesn’t move that quickly. And since their WM is as limited as ours, they multi-task at their expense. Actually they don’t multi-task at all—they task-switch. And it comes with a cost.15
Just as evidence-based medicine has been called a ‘‘paradigm shift,’’ it is not an exaggeration to proclaim that this new pedagogical discipline, derived from scientific study of the nature of memory and cognition, amounts to a paradigm shift in our understanding of the interplay between teaching and learning. Which brings us to the next issue—how ought this understanding of the nature of learning change the nature of teaching? More particularly, what should our protagonist’s ‘‘Intro to RCT’s’’ course look like? That’s for part 2 where we will explore how this new pedagogical evidence leads to different approaches to curriculum in the context of their proposed RCT course. References 1. Roediger HL. Applying cognitive psychology to education translational educational science. Psychol Sci Public Interest 2013; 14(1): 1–3. 2. Illich I. Deschooling society. New York: Harper & Row, 1971. 3. Knowles MS. The adult learner: A neglected species. Houston, TX: Gulf Publishing, 1973.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
684
Clinical Trials 11(6)
4. Bjork R, Dunlosky J and Kornell N. Self-regulated learning: beliefs, techniques and illusions. Annu Rev Psychol 2013; 64: 417–444. 5. Mayer RE. Applying the science of learning to medical education. Med Educ 2010; 44(6): 543–549. 6. Pashler H, McDaniel M, Rohrer D, et al. Learning styles: concepts and evidence. Psychol Sci Public Interest 2008; 9(3): 105–119. 7. Perkins DN and Salomon G. Are cognitive skills contextbound? Educ Res 1989; 18(1): 16–25. 8. Glaser R. Education and thinking: the role of knowledge. Am Psychol 1984; 39(2): 93. 9. Elstein AS, Shulman LS and Sprafka SA. Medical problem solving: an analysis of clinical reasoning. Cambridge, MA: Harvard University Press, 1978. 10. Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA 2007; 298(9): 993–001.
11. Wenghofer E, Klass D, Abrahamowicz M, et al. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ 2009; 43(12): 1166–1173. 12. Eva KW and Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80(10): S46–S54. 13. Davis DA, Mazmanian PE, Fordis M, et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 296(9): 1094–1102. 14. Larsen DP, Butler AC and Roediger HL III. Testenhanced learning in medical education. Med Educ 2008; 42(10): 959–966. 15. Kirschner PA and van Merrie¨nboer JJ. Do learners really know best? Urban legends in education. Educ Psychol 2013; 48(3): 169–183.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
Clinician-trialist rounds: 24. Modernizing your introductory graduate course in clinical trials. Part 1: Commonsense meets evidence Geoffrey R Norman and David L Sackett Clin Trials 2014 11: 681 originally published online 16 July 2014 DOI: 10.1177/1740774514542622 The online version of this article can be found at: http://ctj.sagepub.com/content/11/6/681
Published by: http://www.sagepublications.com
On behalf of:
The Society for Clinical Trials
Additional services and information for Clinical Trials can be found at: Email Alerts: http://ctj.sagepub.com/cgi/alerts Subscriptions: http://ctj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Nov 10, 2014 OnlineFirst Version of Record - Jul 16, 2014 What is This?
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
CLINICAL TRIALS
Column
Clinician-trialist rounds: 24. Modernizing your introductory graduate course in clinical trials. Part 1: Commonsense meets evidence
Clinical Trials 2014, Vol. 11(6) 681–684 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514542622 ctj.sagepub.com
Geoffrey R Norman1* and David L Sackett2
Today’s case You have become a highly successful mid-career clinician-trialist: promoted and tenured, with an enviable publication record, an international reputation, a great team of research staff and students and mentees, a supportive clinical team, several clinical teaching awards, and a work-life balance that is the envy of all. You didn’t accomplish this entirely on your own, but thanks also to great mentoring, rigorous time-management, and saying ‘‘no’’ to the myriad dys-opportunities that could have consumed too much of your time and talents. Taking stock, you decide it’s time for some payback, and are looking for opportunities to redirect some of your time, energy, and skills to serve the graduate programme that gave you your start. As luck would have it, the introductory course in RCTs is looking for a new director, and once you’ve determined that the scutwork of finding rooms and tutors will remain in a staff member’s capable hands, you offer to take it on. Determined to make it the flagship of the graduate program, you begin by setting down four ‘‘commonsense axioms for teaching and learning’’ based on your experiences as both student and teacher, supplemented by reading a couple of popular books about education: 1.
The course will recognize that individual students have different approaches to learning, and take these individual learning styles into account. You’ll use lots of graphics and flow charts for visual learners, and point-form step-by-step details for verbal learners, conceptual explanations for conceptual learners and examples for practical learners. You’ll take advantage of the state of the art in visual graphics,
*Dr Norman is a Cognitive Psychologist, a Professor in the Department of Clinical Epidemiology & Biostatistics, and an Associate Member of the Department of Psychology, all at McMaster University in Hamilton Ontario.
2.
3.
4.
using dynamic presentations and virtual reality to engage the learners. Recognizing that ‘‘core-knowledge’’ about trial methodology is a rapidly changing field, this course will deliberately favor the acquisition of critical thinking skills over current core knowledge. Instead of overloading students with a bunch of multiple choice tests, the students will be evaluated in part by self-directed ‘‘reflection’’ exercises, so they can learn how to test themselves, and become successful lifelong learners about RCTs as well as keeping their clinical skills up to date. The course will capitalize on the highly effective multi-tasking skills of todays’ learners (e.g. Wi-Fi in every class and seminar room will permit simultaneous access to readings and slideshows) so that learning will be more effective and efficient.
You present your proposal to the departmental education committee, anticipating their shared excitement, praise and hearty endorsement. Instead, they begin to chuckle over your ‘‘axioms for teaching and learning’’ and one of them, an educational psychologist, gently tells you that your axioms possess roughly the same validity as the 1950s treatment of heart attacks (5 weeks of
1
Program for Educational Research and Development, McMaster University, Hamilton, ON, Canada 2 Trout Research & Education Centre at Irish Lake, Markdale, ON, Canada Corresponding author: Geoffrey R Norman, Program for Educational Research and Development, McMaster University, Room 3519, MDCL, 1200 Main St. West, Hamilton, ON, L8N 3Z5, Canada. Email: [email protected] David L Sackett, Trout Research & Education Centre at Irish Lake, RR 1, Markdale, ON, N0C 1H0, Canada. Email: [email protected]
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
682
Clinical Trials 11(6)
absolute bed rest) and the 1960s treatment of peptic ulcer (gastric freezing). Scrambling to recover lost ground, you exclaim that at least your PowerPoints will be up-to-date with the latest in catchy background music, animation, and eye-catching action. Alas, more chuckles. Never a quitter (‘tho sorely tempted here), you humbly ask for an ‘‘educational consultation’’ to learn why and how your commonsense proposals aren’t sensible after all.
Discussion The proposals from our mid-career trialist colleague could be forgiven; much of current educational practice does result from ideas that are perfectly commonsensical. Alas, it is not based on sound evidence. As Roediger1 puts it: The field of education seems particularly susceptible to the allure of plausible but untested ideas and fads (especially ones that are lucrative for their inventors). One could write an interesting history of ideas based on either plausible theory or somewhat flimsy research that have come and gone over the years. And . once an idea takes hold, it is hard to root out.
This sad state didn’t arise because educators are thicker than the rest of us; it’s simply because the history of science is replete with ideas that made perfect sense, but when subjected to empirical research, just didn’t hold up. And that’s just as true for learning styles and self-assessment as it was for freezing the acid-producing cells of ulcerated stomachs and keeping heart attacks flat on their backs until their infarcts were bridged by stout fibrous tissue. In fact, the situation may be even worse in education, for a couple of reasons. First, while we wouldn’t invoke common sense solutions to solving the mysteries of quantum mechanics or non-chromosomal inheritance, all of us have had years of experience being taught and trying to learn, so we feel authorized to offer strong opinions on what works, what doesn’t, and how and why. Second, although the history of formal medical research extends back several centuries, formal research in education and in its basic science, cognitive psychology (more on this in a moment), is a much more recent endeavor. The acknowledged founder of medical education research was George Miller, who was a teacher of Dave Sackett’s at the U of Illinois, which shows how recent it is. Cognitive psychology, the branch of psychology that studies processes like perception, learning, problem-solving also had its origins in the 1960s, although scientific psychology is much older.
A brief history of the ‘‘science of learning’’ Before about 1920, there were the widely accepted educational philosophies of John Dewey (‘‘experiential learning’’) and Edward Thorndike (‘‘active learning’’), but not much experimental evidence that they worked. In this era in psychology, phrenology (reading personality from bumps on the head) and psychoanalysis had equal claims to legitimacy.y Then along came behaviorism (knowledge is the product of stimulus/response conditioning), which abandoned studies of people in favor of rats and pigeons, and was epitomized by the mid-century work of B. F. Skinner in its attempt to put psychology on a scientific footing.z Behaviourism spawned ‘‘behavioral objectives’’ (what the student was expected to be able to do as a result of learning) which reduced learning to mastering hundreds and hundreds of little goals like ‘‘Be able to list 6 causes of anemia.’’ The 1960s spelled the end of this foolishness,§ and none too soon. Society was changing around them, and educators rebelled at the mindless, paternalistic, mechanistic approaches dictated by behaviorism. It was time to return to authentic experience and skill development as a basis for learning and assessment. Illich’s De-Schooling Society2 and Knowles’s The Adult Learner—A Neglected Species3 challenged these teacher-focused approaches and encouraged educators to refocus their efforts on the learner, not the teacher. In general, it was a time to celebrate individuality, in society and in education. In medical education, the sixties brought a radical new approach to undergraduate education (pioneered, we are pleased to say, by McMaster) that focused of acquiring knowledge by solving problems. Instead of mastering long lists of facts, students were encouraged to learn the concepts so they could use them subsequently to solve problems. The educators who devised these revolutionary approaches correctly rejected what passed for the science of learning in those days. However this parting of the ways between educators and psychologists is now in need of reconciliation, because behaviorism died in psychology about the same time as it did in education. The new discipline, cognitive psychology, rejected the proposition that all learning amounted to learning simple associations, and realized that much of human learning—from perception to motor skills—was and should be an object of deeper experimental scientific scrutiny. And in the course of studying these processes, y
To be fair, Osler’s contemporaneous Textbook of Medicine was still recommending blood-letting for pneumonia. z Internal mammary artery ligation for angina pectoris reached its peak at about this same time. § And also delivered the RCT showing for the first time that the treatment of diastolic blood pressures between 115 and 129 did more good than harm.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
Norman and Sackett
683
psychologists have identified a number of highly effective teaching strategies.
So why is the proposed course so very wrong? 1.
Today’s ‘‘science of learning’’ Cognitive psychological research has revealed that we initiate ‘‘thinking’’ by using our ‘‘short-term’’ or ‘‘working memory’’ (WM) which, although of very limited capacity (literally one byte), can access a long-term memory (LTM) of almost infinite capacity (greater even than that of an external hard drive) with incredible speed. It is the changes in our LTM that constitute learning. For example, think about solving these two problems: 1.
2.
What is 12 x 12? Most of our brains can access our LTM and generate the correct answer to this problem in about 200 milliseconds. Google can solve it in about 190 milliseconds, too, but only after searching through over 25 billion possible answers. What is 17 x 17? Interestingly, we know that we don’t know the answer to this problem in about those same 200 milliseconds. We clearly can’t have sequentially searched the sum of all LTM and come up dry. Accordingly, our brain’s LTM must be qualitatively different from the sum of all WM.
Cognitive psychologists characterize this qualitative difference by inserting an important adjective into the term. It’s called Long Term Associative Memory (LTAM), in recognition that the basic process is an association between new information and what is already in long term memory. As Bjork and his colleagues say . we do not store information in our LTM by making any kind of literal recording of that information, but, instead, we do so by relating new information to what we already know. We store new information in terms of its meaning to us, as defined by its relationships and semantic associations to information that already exists in our long term associative memory.4
These two simple, yet powerful ideas, short-term working memory (WM) and long-term associative memory (LTAM) lead to some very profound insights about teaching and learning which in turn lead to very powerful teaching interventions. And we do mean powerful; many of these interventions can increase learning by up to 50% with no increase in instructional or learning time! That is, teach it one way and the average mark on the final is 60%; teach it using some of these strategies and it’s 90%. Or in the language of systematic reviews, effect sizes** of around 1.0 are the norm.5
**Effect size here = (Treatment Effect) divided by its (Standard Deviation).
2.
3. 4.
5.
6.
While it’s easy to measure individual differences in things like learning style, extraversion, conscientiousness (and height and weight for that matter), these have very little to do with the changes in the LTAM that constitutes learning. Despite our intuitions, the major determinant of learning is what is to be learned, not who is doing the learning.6 General skills like problem-solving or critical thinking are very weak predictors of performance.7,8 In medicine, although we can measure ‘‘skills’’ like history-taking or clinical reasoning reliably with tools like the Objective Structured Clinical Examination, they do not generalize from one case to another (hence are not skill-like),9 and do not predict performance in practice.10,11 Skills like self-assessment, meta-cognition or reflection have no relation to actual performance.12,13 Periodic well-designed tests, instead of interfering with learning, can actually enhance learning by encouraging students to repeatedly retrieve information from LTAM.14 Given the serious size limit of short-term WM, all those whistles and bells in your powerpoint will have a large negative impact on learning (that is, on converting short-term WM to LTAM). Finally, surprise surprise. Today’s kids are using the same brain as we are. Evolution doesn’t move that quickly. And since their WM is as limited as ours, they multi-task at their expense. Actually they don’t multi-task at all—they task-switch. And it comes with a cost.15
Just as evidence-based medicine has been called a ‘‘paradigm shift,’’ it is not an exaggeration to proclaim that this new pedagogical discipline, derived from scientific study of the nature of memory and cognition, amounts to a paradigm shift in our understanding of the interplay between teaching and learning. Which brings us to the next issue—how ought this understanding of the nature of learning change the nature of teaching? More particularly, what should our protagonist’s ‘‘Intro to RCT’s’’ course look like? That’s for part 2 where we will explore how this new pedagogical evidence leads to different approaches to curriculum in the context of their proposed RCT course. References 1. Roediger HL. Applying cognitive psychology to education translational educational science. Psychol Sci Public Interest 2013; 14(1): 1–3. 2. Illich I. Deschooling society. New York: Harper & Row, 1971. 3. Knowles MS. The adult learner: A neglected species. Houston, TX: Gulf Publishing, 1973.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
684
Clinical Trials 11(6)
4. Bjork R, Dunlosky J and Kornell N. Self-regulated learning: beliefs, techniques and illusions. Annu Rev Psychol 2013; 64: 417–444. 5. Mayer RE. Applying the science of learning to medical education. Med Educ 2010; 44(6): 543–549. 6. Pashler H, McDaniel M, Rohrer D, et al. Learning styles: concepts and evidence. Psychol Sci Public Interest 2008; 9(3): 105–119. 7. Perkins DN and Salomon G. Are cognitive skills contextbound? Educ Res 1989; 18(1): 16–25. 8. Glaser R. Education and thinking: the role of knowledge. Am Psychol 1984; 39(2): 93. 9. Elstein AS, Shulman LS and Sprafka SA. Medical problem solving: an analysis of clinical reasoning. Cambridge, MA: Harvard University Press, 1978. 10. Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA 2007; 298(9): 993–001.
11. Wenghofer E, Klass D, Abrahamowicz M, et al. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ 2009; 43(12): 1166–1173. 12. Eva KW and Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80(10): S46–S54. 13. Davis DA, Mazmanian PE, Fordis M, et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 296(9): 1094–1102. 14. Larsen DP, Butler AC and Roediger HL III. Testenhanced learning in medical education. Med Educ 2008; 42(10): 959–966. 15. Kirschner PA and van Merrie¨nboer JJ. Do learners really know best? Urban legends in education. Educ Psychol 2013; 48(3): 169–183.
Downloaded from ctj.sagepub.com at UNIVERSITE LAVAL on November 29, 2014
