Research Report
Affordances of Knowledge Translation in Medical Education: A Qualitative Exploration of Empirical Knowledge Use Among Medical Educators Betty Onyura, PhD, France Légaré, MD, PhD, Lindsay Baker, MEd, Scott Reeves, PhD, Jay Rosenfield, MD, MEd, Simon Kitto, PhD, Brian Hodges, MD, PhD, Ivan Silver, MD, MEd, Vernon Curran, MEd, PhD, Heather Armson, MD, MCE, and Karen Leslie, MD, MEd
Abstract Purpose Little is known about knowledge translation processes within medical education. Specifically, there is scant research on how and whether faculty incorporate empirical medical education knowledge into their educational practices. The authors use the conceptual framework of affordances to examine factors within the medical education practice environment that influence faculty utilization of empirical knowledge. Method In 2012, the authors, using a purposive sampling strategy, recruited medical education leaders in undergraduate medical education from a Canadian university. Recruits all had direct teaching
Medical education research has
drawn on diverse disciplines to produce a large body of empirical knowledge. Educators and researchers alike need to take stock of how this research interacts with practice and review the status of knowledge translation processes within medical education. Whereas our understanding of knowledge translation—the dynamic process by which research is synthesized, exchanged, and disseminated in order to inform decision making and implementation1— has grown steadily in the clinical realm,2,3 Please see the end of this article for information about the authors. Correspondence should be addressed to Dr. Onyura, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, 209 Victoria St., Toronto, ON, M5B 1T8T; telephone: (416) 864-6060 ext. 77420; fax: (416) 864-5929; e-mail: [email protected]. Acad Med. 2015;90:518–524. First published online December 2, 2014 doi: 10.1097/ACM.0000000000000590
518
and curricular development roles in either preclinical or clinical courses across the four years of the undergraduate curriculum. Data were collected through individual semistructured interviews on participants’ use of empirical evidence, as well as the factors that influence integration of empirical knowledge into practice. Data were analyzed using thematic analysis. Results Fifteen medical educators participated. The authors identified both constraining and facilitating affordances of empirical medical education knowledge use. Constraining affordances included poor quality and availability of evidence, inadequate knowledge delivery approaches, work and role
we still know little about knowledge translation in medical education. Indeed, there is concern that there is a gap between research and practice and that widely used educational methods are not always those supported by empirical data.4–8 Education researchers, educators, and policy makers have speculated about the cause of this gap. Some researchers contend that educators are not good consumers of science or lack the skills to understand research implications,8 whereas educators maintain that empirical research is often inaccessible or inapplicable in real-world settings.6,8,9 In the knowledge translation discourse, factors that contribute to research– practice discordance are identified as barriers in the knowledge-to-action (KTA) cycle.10 The cyclical KTA model illustrates processes involved in moving knowledge from creation to application. It positions knowledge creation at
overload, faculty and student change resistance, and resource limitations. Facilitating affordances included faculty development, peer recommendations, and local involvement in medical education knowledge creation. Conclusions Affordances of the medical education practice environment influence empirical knowledge use. Developing strategies for effective knowledge translation thus requires careful assessment of contextual factors that can enable, constrain, or inhibit evidence use. Empirical knowledge use is most likely to occur among medical educators who are afforded rich, facilitative opportunities for participation in creating, seeking, and implementing knowledge.
the cycle’s center; created knowledge then necessitates transformation into knowledge products (e.g., clinical guidelines) before it is fed into an action cycle for uptake into practice.11 This model positions both barriers and facilitators to knowledge translation as somewhat external to the knowledge creation process—thus implying that these factors exist within the realm of the practitioner and intended “knowledge user.” Scant attention has been directed to understanding how contextual factors interact with empirical knowledge use at various points of the KTA cycle. In this study, we aimed to identify and examine features of medical educators’ environments that might facilitate, constrain, or inhibit their use of empirical knowledge. To do so, we adopted the unifying conceptual framework of “affordances.” This framework, borrowed from the psychology of perception and prevalent in the workplace
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
learning literature,12–14 emphasizes the interaction between individuals and their environments. It describes how features of the learning or practice environment, as perceived by individuals, affect specified behaviors (such as the use of empirical knowledge).13 Affordances can be physical, informational, or social features, and they range along an “enablement gradient,” from facilitating to constraining to inhibiting the execution of specific behaviors and goals (see Table 1).15 This framework is a powerful, psychologically relevant way to evaluate environmental influences on individual behavior, and, we contend, affordances can be situated at any and all phases of the KTA cycle—from knowledge creation through knowledge implementation. We aimed to identify and describe the affordances of medical educators’ use of empirical knowledge. Method
We conducted this research in 2012. The project received ethical approval from the University of Toronto research ethics board. Participants Using a purposive sampling strategy, we recruited faculty in undergraduate medical education at a Canadian medical school. First, we examined institutional records to identify faculty in education leadership roles across preclinical and clinical courses who were actively involved in undergraduate teaching and curricular development. We sent e-mail requests to the 43 faculty members who met these criteria. We intentionally sampled the 21 respondents (49%) for diversity in years of experience and
formal training in medical education, continuing until saturation was reached. The 15 faculty members (10 women and 5 men) who completed the study were drawn from preclerkship (8) and clerkship (7) courses in the undergraduate medical education program; 14 were clinical faculty, and 1 was nonclinical faculty. They had worked in medical education between 1 and 36 years (mean = 15.93 years). Their formal training in education was diverse: 5 had completed either a master of education or a formal education scholarship program, while the other 10 had no formal training in education. Data collection One author (B.O.) collected data using semistructured interviews with individual participants. For this purpose, we designed an interview guide (see Appendix 1) consisting of nonleading questions about the knowledge sources that participants relied on for their educational practices, including curricular development, teaching, and assessment practices. Follow-up probes ascertained whether faculty relied on empirical knowledge sources and generated discussion on the facilitating and constraining affordances of evidence seeking and utilization in their educational practices. B.O. conducted the interviews using the guide to generate semistructured discussion with participants. Interviews lasted 20 to 65 minutes (mean = 45 minutes). Analytic approach and data analysis Adopting a critical realist orientation, we used thematic analysis18,19 to identify, analyze, and report data patterns. While
16,17
Table 1 Conceptual Definitions of Affordances Along an Enablement Gradienta as Used in a Qualitative Study of 15 Medical Educators at One Canadian Medical School, 2012 Placement on enablement gradient Facilitating affordances Constraining affordances
Inhibiting affordances
Definition Engage the skills and/or volition of an actor within a given context to resource his/her optimal performance of a given activity. Limit an individual’s ability and/or volition to engage his/her skills within the given context, thereby resulting in suboptimal performance of a given activity. Bring an individual to a permanent standstill or stalemate within the given context and thereby prevent even minimal engagement of their skills and volitional effort toward performing a given activity.
The enablement gradient ranges from facilitating (resource intensive) to completely blocking or inhibiting (resource deficient) state of the workers’ environment. Source: Cronshaw SF, Ong PY, Chappell DB. Workers’ adaptation enables work functioning. Psychol Rep. 2007;100:1043–1064.
a
Academic Medicine, Vol. 90, No. 4 / April 2015
acknowledging an inherent subjectivity in the production of knowledge, critical realism views social actors’ cognitions and emotions as real phenomena.16,17 The analytic approach allowed us to engage with the data in a quasi-deductive fashion in order to add theoretical depth.19 It is noteworthy that this study does not aim to make generalizable statements; rather, it aims to identify key issues and develop hypotheses about knowledge translation processes in medical education that may be tested in research with larger sample sizes and/or alternate methodologies. The interviews were audio-recorded and transcribed verbatim. We used NVivo Version 9 (QSR International Pty Ltd., Melbourne, Australia) as our data management tool. Data analytic methods were similar to those outlined in Braun and Clarke.18 Data coding was informed by the affordances framework. Specifically, we coded the transcripts for physical, social, and informational features of participants’ environments that they perceived to facilitate, constrain, or inhibit the search for and use of empirical knowledge. We developed preliminary themes by collating and assembling coded data extracts into various combinations. The confirmability of the results was supported by the reemergence of the identified themes across several transcripts. We reached a saturation of themes after conducting and analyzing 15 interviews. Results
The participants in our study reported that they occasionally engaged with empirical medical education knowledge. Many did not actively seek research evidence to inform their educational practices, relying instead on integrated experiential, historical, and institutional knowledge. List 1 highlights those features of the participants’ environments that they perceived as either constraining or facilitating their engagement with empirical knowledge. In the following sections, we share extracts from the interviews that we thought to be vivid, exemplary representations of the themes. Constraining and inhibiting affordances The participating faculty members reported several factors that limited their capacity or willingness to use empirical knowledge. These constraining (and sometimes entirely inhibiting)
519
Research Report
List 1 Themes and Subthemes Relating to the Affordances of Empirical Knowledge Use in Medical Education, Identified in a Qualitative Study of 15 Medical Educators at One Canadian Medical School, 2012 Constraining or inhibiting affordances • Poor evidence quality or availability • Inadequate knowledge delivery approaches • Work and role overload • Faculty resistance to change • Financial and staffing resource limitations • Student resistance to change Facilitating affordances • Faculty development • Peer recommendations • Local involvement in knowledge creation
affordances operated at various phases of the KTA cycle, from knowledge creation (e.g., poor availability of evidence) to knowledge implementation (e.g., faculty resistance to change). Poor quality and availability of evidence. The participants perceived empirical medical education knowledge as unavailable, weak, or otherwise limited in the extent to which it could impact their practices. Some argued that the quality of available literature was poor, and thus did not provide definitive answers to practitioners on the efficacy of various teaching approaches. A lot of … research evidence in medical education, for the most part, has been descriptive, it’s been about evaluation, but there’s not a whole lot on actual teaching methods, like, which methods actually work better.
They also thought that, in contrast with new evidence pertaining to clinical practice, some research in medical education (e.g., on soft skills such as giving feedback) was commonsensical and thus limited in the extent to which it could impact their educational practices. I’ve got to be fairly selective in terms of where I spend my time.… In a lot of ways I find that the more clinically focused journals, it actually changes my clinical practice. With the educational ones, I kind of read them over and I realize I’m kind of doing a lot of that stuff already. It’s not telling me anything new or novel.
Perceptions of evidence quality were, for some, influenced by the difficulty
520
inherent in interpreting education research conducted under epistemological traditions different from conventional clinical research. Indeed, some faculty acknowledged experiencing challenges using qualitative, as opposed to quantitative, evidence in medical education. Most of my training in the understanding of evidence has been quantitative and has been in the context of clinical studies and clinical evidence. And so, that’s where I have a huge foundational history and that’s where, I think, a lot of my deep foundational beliefs have come from.… In a classic sense, we’ve been taught that qualitative evidence is simply exploratory, it’s hypothesis generating and it’s not the final answer. And I must say, I think I’m still sighted in that paradigm and I need to learn more and see more and wait to see whether qualitative evidence can go beyond that, beyond hypothesis generation.
Several participants also lamented the lack of available empirical knowledge in their specific areas of medical education. Inadequate knowledge delivery approaches. A consistent theme across the interviews was that existing approaches to delivering empirical knowledge are inadequate. The participants wanted empirical evidence to be disseminated using approaches that are highly accessible to educators, in part through simple synthesis and brief presentation. I generally try to find [research sources] where [evidence] is fairly digested and fairly synthesized already.… They wrote the teachers’ handbook, which I think was a great idea and it’s a wonderful resource, but like all big fat books, it’s daunting and not that useable.
Some unfavorably compared the approaches to delivering knowledge in medical education with the much more effective approaches used in clinical research. There isn’t this tradition in the medical education literature that there is in the clinical literature of regular updates of evidence. So, for example, in the treatment of a disease, you can look for reviews of the current evidence for the treatment of that disease and they come out every year so you get updates. There’s no update in the medical education literature with review articles on the evidence for the effectiveness of a lecture every year.
Work and role overload. A significant part of the participants’ reflections
during the interviews had to do with their work environment and the organizational context within which they performed their educational roles. They described medical education faculty roles as having exceedingly high job demands. Many medical education leads perform multiple roles including teaching, advising students, managing curricula, and executing administrative functions. Clinical faculty must perform their educational and scholarly duties in addition to maintaining their clinical practices. Most participants found that they could not satisfy even the basic demands of the job within their allotted work hours, even when provided protected time. Consequentially, they simply could not prioritize time for seeking empirical knowledge in education. I do think that I don’t have enough time to really think critically about evidence, because I’m busy holding meetings and writing curricula and dealing with students, and so the priority isn’t as high as I’d like it to be.
Effects of this overload are compounded by the fact that work frequently spills over into other areas of the participants’ lives. Reports of suboptimal work–life balance were common as participants shared how work-related activities consume what should be personal time. People forget that we’re clinicians and we have a limited amount of time to put into this … it kind of envelops into our own personal time because it has to.… The things that they expect [us] to do fill up so much of your time that to actually go and do reading on education research? Like I said, there’s not enough time during the day.
The absence of discretionary time at work due to work and role overload completely inhibits many educators’ engagement with empirical knowledge. Empirical evidence seeking was thus relegated to an extra-role behavior that there was never time to perform. Extra-role behaviors are actions that promote organizational goals, but are not specified or formally prescribed role requirements.20 Faculty resistance to change. The participants described being unable to use empirical knowledge due to reluctance by fellow faculty to modify their existing educational practices. This constrained the extent to which new ideas
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
could be introduced or integrated into curricula. I feel there’s a lot of resistance to any kind of new things. I did a course recently … and I looked at the utility of [a certain instructional approach]. I reviewed all the literature and found out all this stuff. And then I thought well, we should maybe change how we’re [doing this]. And I went to the committee and basically got my head chopped off because they didn’t want to change anything. They just sat back and just said, no. We’ve been doing this, this way, for 20 years and we don’t want to change it.… I think that all they see is a person who’s coming with all these ideas and all it can mean to them is more work and they don’t want it.
The participants attributed part of this resistance to the perception that changes add demands to already-demanding jobs. They also attributed it to the fact that, for courses rated highly by students, faculty feel little motivation to modify existing practices. The challenge, really, is to get all the people that are involved in the course behind you in terms of changes you want to make.… It’s usually individuals who believe that, things aren’t broken, why fix them? Our course is rated very highly so that’s a real concern.
Financial and staff resource limitations. The participants experienced challenges in accessing the resources necessary to implement desired educational programming. These resource limitations are, in part, related to securing adequate staffing. I find it hard to find people who want to do stuff.… I have a sense of frustration trying to get help. I was trying to get [an educational program] off the ground for years.… You are handing someone an opportunity for scholarship and I could not find someone.
Sometimes, ideas for advancing innovative educational practices cannot be implemented due to a lack of funds. There are lots of things I could fix with more money and more space.… I could be more creative and inventive.… I can get more [equipment] that students find useful to learn from.
Students’ resistance to change. Some participants described students’ engagement as essential to successfully implementing empirically supported instructional approaches. If students
Academic Medicine, Vol. 90, No. 4 / April 2015
are not receptive, educators have a hard time introducing new teaching methods. This is particularly true for methods that require significant self-direction. Whenever you try to get into more selfdirection, usually the students balk. They want a syllabus, they want it in black and white, and they want to know what’s on the exam. When we started with PBL it was sort of the flavor of the month at the time and there was a lot of good rationale around why PBL was useful.… But it turned out the students just went through the case and they just wanted the answers and they were off as quickly as possible.
Facilitating affordances The participants also identified several factors that help make them able and willing to seek and implement empirical evidence in medical education. These facilitating affordances also operate at various phases of the KTA cycle, from creating knowledge (e.g., local involvement in knowledge creation) to knowledge access and implementation (e.g., peer recommendations). Faculty development. Some participants reported relying on faculty development resources for direction on emerging research that should be integrated into their curricula. We tend to [use empirical education research sources] if there is faculty development. I wouldn’t use it without having a certain amount of instruction coming for me in one particular venue.
In addition, several participants availed themselves of empirical knowledge on educational practices through faculty development programming. Such programs enabled them to more competently participate in discourse about education and comprehend relevant research. I realized afterwards, after I came on staff, that everybody really was getting real training in education, which I never had and I felt, I can’t possibly keep up. I can’t participate in conversations about these things or understand research or do things the right way without some more training. So, that was why I did the [faculty development program].
Peer recommendations. Recommendations on empirical resources from peers emerged as a facilitating affordance. A number of participants reported that they were encouraged to review empirical research written or referred to by colleagues.
Sometimes there’s some very good work in other journals that I don’t know.… I got an article sent to me by a colleague from a journal I’ve never heard of, but it was a phenomenal article on validating new assessment methods. I read [medical education research articles] from time to time, and to be honest with you, it’s if I have colleagues that have written something and said, hey, check this out.
Local involvement in knowledge creation. Those participants who were actively involved in conducting medical education research within their own work units reported using their findings to facilitate their educational practices. Well, we do medical education research and we use some of those findings to develop our curriculum. We’re doing that right now, so we do use some research that we are involved in to help us with that.
Discussion
Our research is a novel examination of the processes of knowledge translation within medical education. Using the affordances model, we identified features of medical educators’ practice environments that were perceived as constraining or facilitating the use of empirical knowledge in their educational practices. The findings empirically support the belief that environmental and contextual factors have an impact on evidence-based practice.21,22 Indeed, to successfully promote the use of empirical knowledge in medical education, attention must be paid to these affordances. Our findings have implications for different phases of the knowledge translation process, from creation through to implementation. Regarding knowledge creation, medical educators expressed concerns about the rigor and value of empirical knowledge in medical education. Some of these concerns relate to deficiencies in study design and execution; others to the poor availability of sound empirical knowledge in areas of direct relevance to educators’ educational practices. Indeed, it has been reported that few studies employ robust empirical designs to assess educational activities23; this view may partially explain why some medical educators do not believe existing research provides incisive answers to their questions. It is critical to note that differences in epistemological
521
Research Report
traditions between clinical and education research may underlie some of the tensions relating to evidence quality. The experimental control that is highly sought in clinical research realms is often unattainable or undesirable in education research. Educational interventions are heavily influenced by context, culture, and the emergent characteristics of students. These differences may call for means of interpreting and integrating research and practice that may be unfamiliar to faculty. Epistemological tensions need not paralyze the sharing and exchange of knowledge. As in clinical research, efforts should be extended toward synthesizing best available empirical knowledge into accessible, contextually appropriate knowledge tools (e.g., guidelines)24 so that knowledge can be effectively delivered to broad audiences. Even where knowledge creation is successful, institutional and organizational constraints can impede knowledge use and implementation. High job demands leave faculty little time for seeking and integrating empirical knowledge. Difficulties with securing institutional support can relegate the use of empirical knowledge to a low-priority, extra-role status as available resources are focused on activities critical to day-to-day operations. To counter these constraints, academic health systems should provide financial resources and related supports, including protected time and administrative aid for clinical faculty. This type of institutional support may moderate faculty resistance to new initiatives. It is worthwhile to note that resistance to change may be intensified by the collective decision-making processes required for integrating evidence into medical education. In contrast to the clinical environment, where empirical knowledge often informs individual decisions, medical educators often make these decisions in group settings, such as educational committees, where complex group dynamics, differences of opinion, and conflicts of interest must be negotiated before curricular changes can be enacted. Failure to engage decision makers and overcome resistance may constrain or inhibit the use of empirical knowledge. The facilitating affordances identified in our study shed light on factors that can be leveraged to enhance the success of
522
knowledge translation strategies. Faculty development, for example, can both promote empirical knowledge use and function as a key point of access. Indeed, faculty development programs are well situated as knowledge brokers, bridging the gap between those who research medical education and those who practice it. Effective knowledge brokers understand the complexities of both research and end-user cultures, and effectively exert the influence required to facilitate knowledge transfer between the groups.25–27 Knowledge brokers can also enable the use of researchin-practice and capacity building through skills training.28 Faculty developers are uniquely positioned to identify the empirical knowledge needs of the medical educators who use their services, and then develop strategies to address those needs.29 Faculty development programs, particularly those in leadership development, might also enable medical educators to promote the use of empirical knowledge by enhancing their ability to engage others, including students and colleagues. Our finding that peers are often conduits for disseminating empirical knowledge highlights the relevance of supportive workplace relationships to knowledge translation. Indeed, knowledge sharing through relationships has been described as a powerful tool in the knowledge transfer arsenal.26,27 Health professionals are more likely to consult a colleague they perceive as an expert than they are to review published research.30 Strategies that capitalize on workplace relationships, such as communities of practice,31 can leverage those relationships to promote the use of empirical knowledge. Local involvement in empirical research is a facilitating affordance with interesting implications for knowledge inquiry processes in medical education. Indeed, poor translation of research knowledge has been attributed, in part, to a separation between the research and practitioner communities.28,32 Conversely, reciprocal relationships between researchers and practitioners can generate empirical knowledge that is more effectively translated into practice.33 Involving medical educators in the creation of empirical knowledge relevant to their areas of expertise may be a critical strategy for successful knowledge translation. It is worthwhile to note that the Canadian Institutes of Health Research has defined an approach
to research, called integrated knowledge translation, in which researchers and practitioners share control, working together to shape research questions, methodologies, and applications.11,34 Integrated knowledge translation includes activities such as developing collaborative learning groups of researchers and educators/practitioners, a useful approach for encouraging the bidirectional exchange of knowledge, which, in turn, informs both the seeking and use of empirical knowledge. Future directions Future research might examine the success of various strategies in moderating the negative impact of constraining affordances and leveraging facilitating affordances to enhance the use of empirical knowledge translation. Additional research could use alternative methodologies to explore the contextual factors that influence educators’ integration of empirical knowledge, as well as draw on this study’s findings to develop a large-scale survey to gain a broader understanding of these issues regionally and internationally. Strengths and limitations To our knowledge, this is the first study to explore how medical educators interact with empirical evidence in their educational practices. Although we do not aim to generalize these findings, future research involving other regions and institutions may increase the transferability of our findings. It is important to note that individual faculty members’ accounts of the features of their practice environments may not accurately reflect the physical and social realities of their workplace. However, the conceptual framework of affordances highlights the power of individuals’ perceptions of their work environment in influencing their behavior. Conclusion
The findings of our preliminary investigation shed light on the challenges inherent in promoting empirical knowledge use within the demanding environment of medical education practice. Developing effective knowledge translation strategies necessitates careful assessment of environmental factors that can enable, constrain, or inhibit evidence use. Those medical educators who are afforded rich, facilitative
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
opportunities are the ones most likely to seek, implement, and value empirical knowledge. Ultimately, reduction of the research–practice gap in medical education is only tenable through collaborative efforts of research and practice communities within supportive institutional environments. Funding/Support: Funded by a Knowledge Translation Canada (Canadian Institutes of Health Research) Seed Funding Grant. Other disclosures: Dr. Onyura, Ms. Baker, and Dr. Leslie work in faculty development research and programming within the Centre for Faculty Development in Toronto. Ethical approval: This project received ethical approval from the University of Toronto research ethics board (protocol reference number: 26822). Previous presentations: Parts of this work were presented at the Qualitative Health Research Conference of the International Institute of Qualitative Methodology, in Montreal, Quebec, Canada, October 2012; at the Canadian Conference on Medical Education, in Quebec City, Canada, April 2013; and at the International Conference on Faculty Development in the Health Professions, in Prague, Czech Republic, August 2013. Dr. Onyura is research and evaluation consultant, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, Toronto, Ontario, Canada. Dr. Le´gare´ is Canada Research Chair in Implementation of Shared Decision Making in Primary Care and professor, Department of Family Medicine and Emergency Medicine, Laval University, Québec, Québec, Canada. Ms. Baker is research and education consultant, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, Toronto, Ontario, Canada. Dr. Reeves is professor of interprofessional research, Faculty of Health, Social Care and Education, Kingston University and St. Georges, University of London, London, United Kingdom. Dr. Rosenfield is professor, Faculty of Medicine, and vice dean of undergraduate medical education, University of Toronto, Toronto, Ontario, Canada. Dr. Kitto is a medical sociologist and assistant professor, Department of Surgery, University of Toronto, Toronto, Ontario, Canada. Dr. Hodges is professor, Faculty of Medicine and Faculty of Education, University of Toronto, and vice president of education, University Health Network, Toronto, Ontario, Canada. Dr. Silver is professor, Faculty of Medicine, University of Toronto, and vice president of education, Centre for Addiction and Mental Health, Toronto, Ontario, Canada. Dr. Curran is professor, Faculty of Medicine, Memorial University of Newfoundland, St. John’s, Newfoundland, Canada.
Academic Medicine, Vol. 90, No. 4 / April 2015
Dr. Armson is director of research, Foundation of Medical Practice Education, and professor, University of Calgary, Calgary, Alberta, Canada. Dr. Leslie is director, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, and associate professor, Department of Pediatrics, University of Toronto, Faculty of Medicine, Toronto, Ontario, Canada
References 1 Straus SE, Tetroe J, Graham I. Defining knowledge translation. CMAJ. 2009;181:165–168. 2 Baumbusch JL, Kirkham SR, Khan KB, et al. Pursuing common agendas: A collaborative model for knowledge translation between research and practice in clinical settings. Res Nurs Health. 2008;31:130–140. 3 Doran DM, Sidani S. Outcomes-focused knowledge translation: A framework for knowledge translation and patient outcomes improvement. Worldviews Evid Based Nurs. 2007;4:3–13. 4 Harden M, Grant J, Buckley G, Hart IR. BEME guide no. 1: Best evidence medical education. Med Teach. 1999;21:553–562. 5 Kaufman DM. ABC of learning and teaching in medicine: Applying educational theory in practice. BMJ. 2003;326:213–216. 6 Carnine D. Bridging the research-to-practice gap. J Except Child. 1997;63:513–521. 7 Stoiber KC, Kratochwill TR. Empirically supported interventions and school psychology: Rationale and methodological issues—part I. Sch Psychol Q. 2000;15:75–105. 8 Spencer TD, Detrich R, Slocum TA. Evidence-based practice: A framework for making effective decisions. Educ Treat Child. 2012;35:127–151. 9 Greenwood CR, Abbott M. The research to practice gap in special education. Teach Educ Spec Educ. 2001;24:276–289. 10 Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: Time for a map? J Contin Educ Health Prof. 2006;26:13–24. 11 Menear M, Grindrod K, Clouston K, Norton P, Légaré F. Advancing knowledge translation in primary care. Can Fam Physician. 2012;58:623–627, e302. 12 Billet S. Learning through work: Workplace affordances and individual engagement. J Workplace Learn. 2001;13:209–214. 13 Laurillard D, Stratfold M, Luckin R, Plowman L, Taylor J. Affordances for learning in a non-linear narrative medium. J Interact Media Educ. 2000;2:1–19. 14 Gibson JJ. The Ecological Approach to Visual Perception. Boston, Mass: Houghton Mifflin; 1979. 15 Cronshaw SF, Ong PY, Chappell DB. Workers’ adaptation enables work functioning. Psychol Rep. 2007;100(3 pt 2):1043–1064. 16 Maxwell JA, Mittappalli K. Realism as a stance for mixed method research. In: Tashakkori A, Teddlie C, eds. Sage Handbook of Mixed Methods in Social and Behavioural Research. 2nd ed. Thousand Oaks, Calif: Sage; 2010:241–272. 17 Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. Thousand Oaks, Calif: Sage; 1994.
18 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101. 19 Hayes N. Theory-led thematic analysis: Social identification in small companies. In: Hayes N, ed. Doing Qualitative Analysis in Psychology. Hove, UK: Psychology Press; 1997:93–114. 20 Organ DW, Konovsky M. Cognitive versus affective determinants of organizational citizenship behavior. J Appl Psychol. 1989;74:157–164. 21 Kitson AL. The need for systems change: Reflections on knowledge translation and organizational change. J Adv Nurs. 2009;65:217–228. 22 McCormack B, Kitson A, Harvey G, RycroftMalone J, Titchen A, Seers K. Getting evidence into practice: The meaning of “context.” J Adv Nurs. 2002;38:94–104. 23 Mazmanian PE. Advancing the body of knowledge: Evidence and study design for quality improvement. In: Davis DA, Barnes B, Fox R, eds. The Continuing Professional Development of Physicians: From Research to Practice. Chicago, Ill: American Medical Association Press; 2003:317–339. 24 Veldhuijzen W, Ram PM, van der Weijden T, van der Vleuten CP. Communication guidelines as a learning tool: An exploration of user preferences in general practice. Patient Educ Couns. 2013;90:213–219. 25 Dobbins M, Robeson P, Ciliska D, et al. A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci. 2009;4:23. 26 Metzler MJ, Metz GA. Analyzing the barriers and supports of knowledge translation using the PEO model. Can J Occup Ther. 2010;77:151–158. 27 Metzler MJ, Metz GA. Translating knowledge to practice: An occupational therapy perspective. Aust Occup Ther J. 2010;57:373–379. 28 Ward VL, House AO, Hamer S. Knowledge brokering: Exploring the process of transferring knowledge into action. BMC Health Serv Res. 2009;9:12. 29 Hodges B. Faculty development for research capacity building. In: Steinert Y, ed. Faculty Development in the Health Professions. Dordrecht, the Netherlands: Springer Verlag; 2014:79–96. 30 Rappolt S, Tassone M. How rehabilitation therapists gather, evaluate, and implement new knowledge. J Contin Educ Health Prof. 2002;22:170–180. 31 Wenger E. Communities of practice and social learning systems. Organization. 2000;7:225–246. 32 Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci. 1979;22:459–470. 33 McDonald PW, Viehbeck S. From evidencebased practice making to practice-based evidence making: Creating communities of (research) and practice. Health Promot Pract. 2007;8:140–144. 34 Graham ID, Tetroe J. How to translate health research knowledge into effective healthcare action. Healthc Q. 2007;10:20–22.
523
Research Report
Appendix 1 Interview Guide Used in a Qualitative Study of 15 Medical Educators, Relating to the Affordances of Empirical Knowledge Use in Medical Education, 2012 1.
What is your role with regard to teaching and curriculum development?
2.
What resources, if any, do you draw on for support as you develop the curriculum for your course?
Probes: Have you felt the need to use any research sources on medical education?
Have you felt the need to use any guides (electronic/books) on medical education?
Have you found the research resources to be useful? How so? Can you provide examples?
Have you experienced any challenges with regard to accessing resources on medical education? Can you provide examples?
Have you experienced any challenges with regard to utilizing resources on medical education? Can you provide examples?
Are there any other resources that you use to inform the development of your curriculum? Can you provide examples?
3.
Are there specific teaching/educational strategies that you employ on a consistent basis with regard to teaching (a) clinical skills, (b) technical skills, knowledge, etc?
Probes: Tell me more about these strategies.
How did you identify these strategies?
Why do you think these strategies are effective?
Do you use empirical sources to identify these strategies?
4.
What methods do you use to assess student progress (evaluate students) in your courses: (a) clinical skills, (b) technical skills, knowledge, etc?
Probes: Why do you choose to use these methods? 5.
Have you found these methods to be effective?
In your experience, have you found that there are certain “best practices” with regard to teaching medical students (a) clinical skills, (b) nonclinical skills?
Probes: Why or why not?
(If so) how did you identify these “best practices”?
Can you integrate these practices within your curriculum? How so? Can you provide examples?
Are there challenges to integrating these practices within the curriculum?
How do you integrate these practices within the curriculum for the courses you direct?
What do you think are some of the barriers to identifying “best practices” for medical education?
6.
Has your approach to instructing/directing your course evolved over time?
Probes: How so?
What led you to make that change?
What do you think has informed the changes you made? What has been your experience with regard to introducing changes into the curriculum?
Why did you adopt that strategy? Did you experience any challenges as you worked modify your curriculum/ teaching approach?
Do you adjust your instructional approach depending on the needs of the students?
Do you think medical education research has influenced the changes in your approach? How so?
7.
What do you think about the research evidence in medical education?
8.
Is there any other information about your teaching practices and/or medical education research/practice that you would like to share?
524
Academic Medicine, Vol. 90, No. 4 / April 2015
Affordances of Knowledge Translation in Medical Education: A Qualitative Exploration of Empirical Knowledge Use Among Medical Educators Betty Onyura, PhD, France Légaré, MD, PhD, Lindsay Baker, MEd, Scott Reeves, PhD, Jay Rosenfield, MD, MEd, Simon Kitto, PhD, Brian Hodges, MD, PhD, Ivan Silver, MD, MEd, Vernon Curran, MEd, PhD, Heather Armson, MD, MCE, and Karen Leslie, MD, MEd
Abstract Purpose Little is known about knowledge translation processes within medical education. Specifically, there is scant research on how and whether faculty incorporate empirical medical education knowledge into their educational practices. The authors use the conceptual framework of affordances to examine factors within the medical education practice environment that influence faculty utilization of empirical knowledge. Method In 2012, the authors, using a purposive sampling strategy, recruited medical education leaders in undergraduate medical education from a Canadian university. Recruits all had direct teaching
Medical education research has
drawn on diverse disciplines to produce a large body of empirical knowledge. Educators and researchers alike need to take stock of how this research interacts with practice and review the status of knowledge translation processes within medical education. Whereas our understanding of knowledge translation—the dynamic process by which research is synthesized, exchanged, and disseminated in order to inform decision making and implementation1— has grown steadily in the clinical realm,2,3 Please see the end of this article for information about the authors. Correspondence should be addressed to Dr. Onyura, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, 209 Victoria St., Toronto, ON, M5B 1T8T; telephone: (416) 864-6060 ext. 77420; fax: (416) 864-5929; e-mail: [email protected]. Acad Med. 2015;90:518–524. First published online December 2, 2014 doi: 10.1097/ACM.0000000000000590
518
and curricular development roles in either preclinical or clinical courses across the four years of the undergraduate curriculum. Data were collected through individual semistructured interviews on participants’ use of empirical evidence, as well as the factors that influence integration of empirical knowledge into practice. Data were analyzed using thematic analysis. Results Fifteen medical educators participated. The authors identified both constraining and facilitating affordances of empirical medical education knowledge use. Constraining affordances included poor quality and availability of evidence, inadequate knowledge delivery approaches, work and role
we still know little about knowledge translation in medical education. Indeed, there is concern that there is a gap between research and practice and that widely used educational methods are not always those supported by empirical data.4–8 Education researchers, educators, and policy makers have speculated about the cause of this gap. Some researchers contend that educators are not good consumers of science or lack the skills to understand research implications,8 whereas educators maintain that empirical research is often inaccessible or inapplicable in real-world settings.6,8,9 In the knowledge translation discourse, factors that contribute to research– practice discordance are identified as barriers in the knowledge-to-action (KTA) cycle.10 The cyclical KTA model illustrates processes involved in moving knowledge from creation to application. It positions knowledge creation at
overload, faculty and student change resistance, and resource limitations. Facilitating affordances included faculty development, peer recommendations, and local involvement in medical education knowledge creation. Conclusions Affordances of the medical education practice environment influence empirical knowledge use. Developing strategies for effective knowledge translation thus requires careful assessment of contextual factors that can enable, constrain, or inhibit evidence use. Empirical knowledge use is most likely to occur among medical educators who are afforded rich, facilitative opportunities for participation in creating, seeking, and implementing knowledge.
the cycle’s center; created knowledge then necessitates transformation into knowledge products (e.g., clinical guidelines) before it is fed into an action cycle for uptake into practice.11 This model positions both barriers and facilitators to knowledge translation as somewhat external to the knowledge creation process—thus implying that these factors exist within the realm of the practitioner and intended “knowledge user.” Scant attention has been directed to understanding how contextual factors interact with empirical knowledge use at various points of the KTA cycle. In this study, we aimed to identify and examine features of medical educators’ environments that might facilitate, constrain, or inhibit their use of empirical knowledge. To do so, we adopted the unifying conceptual framework of “affordances.” This framework, borrowed from the psychology of perception and prevalent in the workplace
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
learning literature,12–14 emphasizes the interaction between individuals and their environments. It describes how features of the learning or practice environment, as perceived by individuals, affect specified behaviors (such as the use of empirical knowledge).13 Affordances can be physical, informational, or social features, and they range along an “enablement gradient,” from facilitating to constraining to inhibiting the execution of specific behaviors and goals (see Table 1).15 This framework is a powerful, psychologically relevant way to evaluate environmental influences on individual behavior, and, we contend, affordances can be situated at any and all phases of the KTA cycle—from knowledge creation through knowledge implementation. We aimed to identify and describe the affordances of medical educators’ use of empirical knowledge. Method
We conducted this research in 2012. The project received ethical approval from the University of Toronto research ethics board. Participants Using a purposive sampling strategy, we recruited faculty in undergraduate medical education at a Canadian medical school. First, we examined institutional records to identify faculty in education leadership roles across preclinical and clinical courses who were actively involved in undergraduate teaching and curricular development. We sent e-mail requests to the 43 faculty members who met these criteria. We intentionally sampled the 21 respondents (49%) for diversity in years of experience and
formal training in medical education, continuing until saturation was reached. The 15 faculty members (10 women and 5 men) who completed the study were drawn from preclerkship (8) and clerkship (7) courses in the undergraduate medical education program; 14 were clinical faculty, and 1 was nonclinical faculty. They had worked in medical education between 1 and 36 years (mean = 15.93 years). Their formal training in education was diverse: 5 had completed either a master of education or a formal education scholarship program, while the other 10 had no formal training in education. Data collection One author (B.O.) collected data using semistructured interviews with individual participants. For this purpose, we designed an interview guide (see Appendix 1) consisting of nonleading questions about the knowledge sources that participants relied on for their educational practices, including curricular development, teaching, and assessment practices. Follow-up probes ascertained whether faculty relied on empirical knowledge sources and generated discussion on the facilitating and constraining affordances of evidence seeking and utilization in their educational practices. B.O. conducted the interviews using the guide to generate semistructured discussion with participants. Interviews lasted 20 to 65 minutes (mean = 45 minutes). Analytic approach and data analysis Adopting a critical realist orientation, we used thematic analysis18,19 to identify, analyze, and report data patterns. While
16,17
Table 1 Conceptual Definitions of Affordances Along an Enablement Gradienta as Used in a Qualitative Study of 15 Medical Educators at One Canadian Medical School, 2012 Placement on enablement gradient Facilitating affordances Constraining affordances
Inhibiting affordances
Definition Engage the skills and/or volition of an actor within a given context to resource his/her optimal performance of a given activity. Limit an individual’s ability and/or volition to engage his/her skills within the given context, thereby resulting in suboptimal performance of a given activity. Bring an individual to a permanent standstill or stalemate within the given context and thereby prevent even minimal engagement of their skills and volitional effort toward performing a given activity.
The enablement gradient ranges from facilitating (resource intensive) to completely blocking or inhibiting (resource deficient) state of the workers’ environment. Source: Cronshaw SF, Ong PY, Chappell DB. Workers’ adaptation enables work functioning. Psychol Rep. 2007;100:1043–1064.
a
Academic Medicine, Vol. 90, No. 4 / April 2015
acknowledging an inherent subjectivity in the production of knowledge, critical realism views social actors’ cognitions and emotions as real phenomena.16,17 The analytic approach allowed us to engage with the data in a quasi-deductive fashion in order to add theoretical depth.19 It is noteworthy that this study does not aim to make generalizable statements; rather, it aims to identify key issues and develop hypotheses about knowledge translation processes in medical education that may be tested in research with larger sample sizes and/or alternate methodologies. The interviews were audio-recorded and transcribed verbatim. We used NVivo Version 9 (QSR International Pty Ltd., Melbourne, Australia) as our data management tool. Data analytic methods were similar to those outlined in Braun and Clarke.18 Data coding was informed by the affordances framework. Specifically, we coded the transcripts for physical, social, and informational features of participants’ environments that they perceived to facilitate, constrain, or inhibit the search for and use of empirical knowledge. We developed preliminary themes by collating and assembling coded data extracts into various combinations. The confirmability of the results was supported by the reemergence of the identified themes across several transcripts. We reached a saturation of themes after conducting and analyzing 15 interviews. Results
The participants in our study reported that they occasionally engaged with empirical medical education knowledge. Many did not actively seek research evidence to inform their educational practices, relying instead on integrated experiential, historical, and institutional knowledge. List 1 highlights those features of the participants’ environments that they perceived as either constraining or facilitating their engagement with empirical knowledge. In the following sections, we share extracts from the interviews that we thought to be vivid, exemplary representations of the themes. Constraining and inhibiting affordances The participating faculty members reported several factors that limited their capacity or willingness to use empirical knowledge. These constraining (and sometimes entirely inhibiting)
519
Research Report
List 1 Themes and Subthemes Relating to the Affordances of Empirical Knowledge Use in Medical Education, Identified in a Qualitative Study of 15 Medical Educators at One Canadian Medical School, 2012 Constraining or inhibiting affordances • Poor evidence quality or availability • Inadequate knowledge delivery approaches • Work and role overload • Faculty resistance to change • Financial and staffing resource limitations • Student resistance to change Facilitating affordances • Faculty development • Peer recommendations • Local involvement in knowledge creation
affordances operated at various phases of the KTA cycle, from knowledge creation (e.g., poor availability of evidence) to knowledge implementation (e.g., faculty resistance to change). Poor quality and availability of evidence. The participants perceived empirical medical education knowledge as unavailable, weak, or otherwise limited in the extent to which it could impact their practices. Some argued that the quality of available literature was poor, and thus did not provide definitive answers to practitioners on the efficacy of various teaching approaches. A lot of … research evidence in medical education, for the most part, has been descriptive, it’s been about evaluation, but there’s not a whole lot on actual teaching methods, like, which methods actually work better.
They also thought that, in contrast with new evidence pertaining to clinical practice, some research in medical education (e.g., on soft skills such as giving feedback) was commonsensical and thus limited in the extent to which it could impact their educational practices. I’ve got to be fairly selective in terms of where I spend my time.… In a lot of ways I find that the more clinically focused journals, it actually changes my clinical practice. With the educational ones, I kind of read them over and I realize I’m kind of doing a lot of that stuff already. It’s not telling me anything new or novel.
Perceptions of evidence quality were, for some, influenced by the difficulty
520
inherent in interpreting education research conducted under epistemological traditions different from conventional clinical research. Indeed, some faculty acknowledged experiencing challenges using qualitative, as opposed to quantitative, evidence in medical education. Most of my training in the understanding of evidence has been quantitative and has been in the context of clinical studies and clinical evidence. And so, that’s where I have a huge foundational history and that’s where, I think, a lot of my deep foundational beliefs have come from.… In a classic sense, we’ve been taught that qualitative evidence is simply exploratory, it’s hypothesis generating and it’s not the final answer. And I must say, I think I’m still sighted in that paradigm and I need to learn more and see more and wait to see whether qualitative evidence can go beyond that, beyond hypothesis generation.
Several participants also lamented the lack of available empirical knowledge in their specific areas of medical education. Inadequate knowledge delivery approaches. A consistent theme across the interviews was that existing approaches to delivering empirical knowledge are inadequate. The participants wanted empirical evidence to be disseminated using approaches that are highly accessible to educators, in part through simple synthesis and brief presentation. I generally try to find [research sources] where [evidence] is fairly digested and fairly synthesized already.… They wrote the teachers’ handbook, which I think was a great idea and it’s a wonderful resource, but like all big fat books, it’s daunting and not that useable.
Some unfavorably compared the approaches to delivering knowledge in medical education with the much more effective approaches used in clinical research. There isn’t this tradition in the medical education literature that there is in the clinical literature of regular updates of evidence. So, for example, in the treatment of a disease, you can look for reviews of the current evidence for the treatment of that disease and they come out every year so you get updates. There’s no update in the medical education literature with review articles on the evidence for the effectiveness of a lecture every year.
Work and role overload. A significant part of the participants’ reflections
during the interviews had to do with their work environment and the organizational context within which they performed their educational roles. They described medical education faculty roles as having exceedingly high job demands. Many medical education leads perform multiple roles including teaching, advising students, managing curricula, and executing administrative functions. Clinical faculty must perform their educational and scholarly duties in addition to maintaining their clinical practices. Most participants found that they could not satisfy even the basic demands of the job within their allotted work hours, even when provided protected time. Consequentially, they simply could not prioritize time for seeking empirical knowledge in education. I do think that I don’t have enough time to really think critically about evidence, because I’m busy holding meetings and writing curricula and dealing with students, and so the priority isn’t as high as I’d like it to be.
Effects of this overload are compounded by the fact that work frequently spills over into other areas of the participants’ lives. Reports of suboptimal work–life balance were common as participants shared how work-related activities consume what should be personal time. People forget that we’re clinicians and we have a limited amount of time to put into this … it kind of envelops into our own personal time because it has to.… The things that they expect [us] to do fill up so much of your time that to actually go and do reading on education research? Like I said, there’s not enough time during the day.
The absence of discretionary time at work due to work and role overload completely inhibits many educators’ engagement with empirical knowledge. Empirical evidence seeking was thus relegated to an extra-role behavior that there was never time to perform. Extra-role behaviors are actions that promote organizational goals, but are not specified or formally prescribed role requirements.20 Faculty resistance to change. The participants described being unable to use empirical knowledge due to reluctance by fellow faculty to modify their existing educational practices. This constrained the extent to which new ideas
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
could be introduced or integrated into curricula. I feel there’s a lot of resistance to any kind of new things. I did a course recently … and I looked at the utility of [a certain instructional approach]. I reviewed all the literature and found out all this stuff. And then I thought well, we should maybe change how we’re [doing this]. And I went to the committee and basically got my head chopped off because they didn’t want to change anything. They just sat back and just said, no. We’ve been doing this, this way, for 20 years and we don’t want to change it.… I think that all they see is a person who’s coming with all these ideas and all it can mean to them is more work and they don’t want it.
The participants attributed part of this resistance to the perception that changes add demands to already-demanding jobs. They also attributed it to the fact that, for courses rated highly by students, faculty feel little motivation to modify existing practices. The challenge, really, is to get all the people that are involved in the course behind you in terms of changes you want to make.… It’s usually individuals who believe that, things aren’t broken, why fix them? Our course is rated very highly so that’s a real concern.
Financial and staff resource limitations. The participants experienced challenges in accessing the resources necessary to implement desired educational programming. These resource limitations are, in part, related to securing adequate staffing. I find it hard to find people who want to do stuff.… I have a sense of frustration trying to get help. I was trying to get [an educational program] off the ground for years.… You are handing someone an opportunity for scholarship and I could not find someone.
Sometimes, ideas for advancing innovative educational practices cannot be implemented due to a lack of funds. There are lots of things I could fix with more money and more space.… I could be more creative and inventive.… I can get more [equipment] that students find useful to learn from.
Students’ resistance to change. Some participants described students’ engagement as essential to successfully implementing empirically supported instructional approaches. If students
Academic Medicine, Vol. 90, No. 4 / April 2015
are not receptive, educators have a hard time introducing new teaching methods. This is particularly true for methods that require significant self-direction. Whenever you try to get into more selfdirection, usually the students balk. They want a syllabus, they want it in black and white, and they want to know what’s on the exam. When we started with PBL it was sort of the flavor of the month at the time and there was a lot of good rationale around why PBL was useful.… But it turned out the students just went through the case and they just wanted the answers and they were off as quickly as possible.
Facilitating affordances The participants also identified several factors that help make them able and willing to seek and implement empirical evidence in medical education. These facilitating affordances also operate at various phases of the KTA cycle, from creating knowledge (e.g., local involvement in knowledge creation) to knowledge access and implementation (e.g., peer recommendations). Faculty development. Some participants reported relying on faculty development resources for direction on emerging research that should be integrated into their curricula. We tend to [use empirical education research sources] if there is faculty development. I wouldn’t use it without having a certain amount of instruction coming for me in one particular venue.
In addition, several participants availed themselves of empirical knowledge on educational practices through faculty development programming. Such programs enabled them to more competently participate in discourse about education and comprehend relevant research. I realized afterwards, after I came on staff, that everybody really was getting real training in education, which I never had and I felt, I can’t possibly keep up. I can’t participate in conversations about these things or understand research or do things the right way without some more training. So, that was why I did the [faculty development program].
Peer recommendations. Recommendations on empirical resources from peers emerged as a facilitating affordance. A number of participants reported that they were encouraged to review empirical research written or referred to by colleagues.
Sometimes there’s some very good work in other journals that I don’t know.… I got an article sent to me by a colleague from a journal I’ve never heard of, but it was a phenomenal article on validating new assessment methods. I read [medical education research articles] from time to time, and to be honest with you, it’s if I have colleagues that have written something and said, hey, check this out.
Local involvement in knowledge creation. Those participants who were actively involved in conducting medical education research within their own work units reported using their findings to facilitate their educational practices. Well, we do medical education research and we use some of those findings to develop our curriculum. We’re doing that right now, so we do use some research that we are involved in to help us with that.
Discussion
Our research is a novel examination of the processes of knowledge translation within medical education. Using the affordances model, we identified features of medical educators’ practice environments that were perceived as constraining or facilitating the use of empirical knowledge in their educational practices. The findings empirically support the belief that environmental and contextual factors have an impact on evidence-based practice.21,22 Indeed, to successfully promote the use of empirical knowledge in medical education, attention must be paid to these affordances. Our findings have implications for different phases of the knowledge translation process, from creation through to implementation. Regarding knowledge creation, medical educators expressed concerns about the rigor and value of empirical knowledge in medical education. Some of these concerns relate to deficiencies in study design and execution; others to the poor availability of sound empirical knowledge in areas of direct relevance to educators’ educational practices. Indeed, it has been reported that few studies employ robust empirical designs to assess educational activities23; this view may partially explain why some medical educators do not believe existing research provides incisive answers to their questions. It is critical to note that differences in epistemological
521
Research Report
traditions between clinical and education research may underlie some of the tensions relating to evidence quality. The experimental control that is highly sought in clinical research realms is often unattainable or undesirable in education research. Educational interventions are heavily influenced by context, culture, and the emergent characteristics of students. These differences may call for means of interpreting and integrating research and practice that may be unfamiliar to faculty. Epistemological tensions need not paralyze the sharing and exchange of knowledge. As in clinical research, efforts should be extended toward synthesizing best available empirical knowledge into accessible, contextually appropriate knowledge tools (e.g., guidelines)24 so that knowledge can be effectively delivered to broad audiences. Even where knowledge creation is successful, institutional and organizational constraints can impede knowledge use and implementation. High job demands leave faculty little time for seeking and integrating empirical knowledge. Difficulties with securing institutional support can relegate the use of empirical knowledge to a low-priority, extra-role status as available resources are focused on activities critical to day-to-day operations. To counter these constraints, academic health systems should provide financial resources and related supports, including protected time and administrative aid for clinical faculty. This type of institutional support may moderate faculty resistance to new initiatives. It is worthwhile to note that resistance to change may be intensified by the collective decision-making processes required for integrating evidence into medical education. In contrast to the clinical environment, where empirical knowledge often informs individual decisions, medical educators often make these decisions in group settings, such as educational committees, where complex group dynamics, differences of opinion, and conflicts of interest must be negotiated before curricular changes can be enacted. Failure to engage decision makers and overcome resistance may constrain or inhibit the use of empirical knowledge. The facilitating affordances identified in our study shed light on factors that can be leveraged to enhance the success of
522
knowledge translation strategies. Faculty development, for example, can both promote empirical knowledge use and function as a key point of access. Indeed, faculty development programs are well situated as knowledge brokers, bridging the gap between those who research medical education and those who practice it. Effective knowledge brokers understand the complexities of both research and end-user cultures, and effectively exert the influence required to facilitate knowledge transfer between the groups.25–27 Knowledge brokers can also enable the use of researchin-practice and capacity building through skills training.28 Faculty developers are uniquely positioned to identify the empirical knowledge needs of the medical educators who use their services, and then develop strategies to address those needs.29 Faculty development programs, particularly those in leadership development, might also enable medical educators to promote the use of empirical knowledge by enhancing their ability to engage others, including students and colleagues. Our finding that peers are often conduits for disseminating empirical knowledge highlights the relevance of supportive workplace relationships to knowledge translation. Indeed, knowledge sharing through relationships has been described as a powerful tool in the knowledge transfer arsenal.26,27 Health professionals are more likely to consult a colleague they perceive as an expert than they are to review published research.30 Strategies that capitalize on workplace relationships, such as communities of practice,31 can leverage those relationships to promote the use of empirical knowledge. Local involvement in empirical research is a facilitating affordance with interesting implications for knowledge inquiry processes in medical education. Indeed, poor translation of research knowledge has been attributed, in part, to a separation between the research and practitioner communities.28,32 Conversely, reciprocal relationships between researchers and practitioners can generate empirical knowledge that is more effectively translated into practice.33 Involving medical educators in the creation of empirical knowledge relevant to their areas of expertise may be a critical strategy for successful knowledge translation. It is worthwhile to note that the Canadian Institutes of Health Research has defined an approach
to research, called integrated knowledge translation, in which researchers and practitioners share control, working together to shape research questions, methodologies, and applications.11,34 Integrated knowledge translation includes activities such as developing collaborative learning groups of researchers and educators/practitioners, a useful approach for encouraging the bidirectional exchange of knowledge, which, in turn, informs both the seeking and use of empirical knowledge. Future directions Future research might examine the success of various strategies in moderating the negative impact of constraining affordances and leveraging facilitating affordances to enhance the use of empirical knowledge translation. Additional research could use alternative methodologies to explore the contextual factors that influence educators’ integration of empirical knowledge, as well as draw on this study’s findings to develop a large-scale survey to gain a broader understanding of these issues regionally and internationally. Strengths and limitations To our knowledge, this is the first study to explore how medical educators interact with empirical evidence in their educational practices. Although we do not aim to generalize these findings, future research involving other regions and institutions may increase the transferability of our findings. It is important to note that individual faculty members’ accounts of the features of their practice environments may not accurately reflect the physical and social realities of their workplace. However, the conceptual framework of affordances highlights the power of individuals’ perceptions of their work environment in influencing their behavior. Conclusion
The findings of our preliminary investigation shed light on the challenges inherent in promoting empirical knowledge use within the demanding environment of medical education practice. Developing effective knowledge translation strategies necessitates careful assessment of environmental factors that can enable, constrain, or inhibit evidence use. Those medical educators who are afforded rich, facilitative
Academic Medicine, Vol. 90, No. 4 / April 2015
Research Report
opportunities are the ones most likely to seek, implement, and value empirical knowledge. Ultimately, reduction of the research–practice gap in medical education is only tenable through collaborative efforts of research and practice communities within supportive institutional environments. Funding/Support: Funded by a Knowledge Translation Canada (Canadian Institutes of Health Research) Seed Funding Grant. Other disclosures: Dr. Onyura, Ms. Baker, and Dr. Leslie work in faculty development research and programming within the Centre for Faculty Development in Toronto. Ethical approval: This project received ethical approval from the University of Toronto research ethics board (protocol reference number: 26822). Previous presentations: Parts of this work were presented at the Qualitative Health Research Conference of the International Institute of Qualitative Methodology, in Montreal, Quebec, Canada, October 2012; at the Canadian Conference on Medical Education, in Quebec City, Canada, April 2013; and at the International Conference on Faculty Development in the Health Professions, in Prague, Czech Republic, August 2013. Dr. Onyura is research and evaluation consultant, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, Toronto, Ontario, Canada. Dr. Le´gare´ is Canada Research Chair in Implementation of Shared Decision Making in Primary Care and professor, Department of Family Medicine and Emergency Medicine, Laval University, Québec, Québec, Canada. Ms. Baker is research and education consultant, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, Toronto, Ontario, Canada. Dr. Reeves is professor of interprofessional research, Faculty of Health, Social Care and Education, Kingston University and St. Georges, University of London, London, United Kingdom. Dr. Rosenfield is professor, Faculty of Medicine, and vice dean of undergraduate medical education, University of Toronto, Toronto, Ontario, Canada. Dr. Kitto is a medical sociologist and assistant professor, Department of Surgery, University of Toronto, Toronto, Ontario, Canada. Dr. Hodges is professor, Faculty of Medicine and Faculty of Education, University of Toronto, and vice president of education, University Health Network, Toronto, Ontario, Canada. Dr. Silver is professor, Faculty of Medicine, University of Toronto, and vice president of education, Centre for Addiction and Mental Health, Toronto, Ontario, Canada. Dr. Curran is professor, Faculty of Medicine, Memorial University of Newfoundland, St. John’s, Newfoundland, Canada.
Academic Medicine, Vol. 90, No. 4 / April 2015
Dr. Armson is director of research, Foundation of Medical Practice Education, and professor, University of Calgary, Calgary, Alberta, Canada. Dr. Leslie is director, Centre for Faculty Development, Li Ka Shing International Healthcare Education Centre, St. Michael’s Hospital, and associate professor, Department of Pediatrics, University of Toronto, Faculty of Medicine, Toronto, Ontario, Canada
References 1 Straus SE, Tetroe J, Graham I. Defining knowledge translation. CMAJ. 2009;181:165–168. 2 Baumbusch JL, Kirkham SR, Khan KB, et al. Pursuing common agendas: A collaborative model for knowledge translation between research and practice in clinical settings. Res Nurs Health. 2008;31:130–140. 3 Doran DM, Sidani S. Outcomes-focused knowledge translation: A framework for knowledge translation and patient outcomes improvement. Worldviews Evid Based Nurs. 2007;4:3–13. 4 Harden M, Grant J, Buckley G, Hart IR. BEME guide no. 1: Best evidence medical education. Med Teach. 1999;21:553–562. 5 Kaufman DM. ABC of learning and teaching in medicine: Applying educational theory in practice. BMJ. 2003;326:213–216. 6 Carnine D. Bridging the research-to-practice gap. J Except Child. 1997;63:513–521. 7 Stoiber KC, Kratochwill TR. Empirically supported interventions and school psychology: Rationale and methodological issues—part I. Sch Psychol Q. 2000;15:75–105. 8 Spencer TD, Detrich R, Slocum TA. Evidence-based practice: A framework for making effective decisions. Educ Treat Child. 2012;35:127–151. 9 Greenwood CR, Abbott M. The research to practice gap in special education. Teach Educ Spec Educ. 2001;24:276–289. 10 Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: Time for a map? J Contin Educ Health Prof. 2006;26:13–24. 11 Menear M, Grindrod K, Clouston K, Norton P, Légaré F. Advancing knowledge translation in primary care. Can Fam Physician. 2012;58:623–627, e302. 12 Billet S. Learning through work: Workplace affordances and individual engagement. J Workplace Learn. 2001;13:209–214. 13 Laurillard D, Stratfold M, Luckin R, Plowman L, Taylor J. Affordances for learning in a non-linear narrative medium. J Interact Media Educ. 2000;2:1–19. 14 Gibson JJ. The Ecological Approach to Visual Perception. Boston, Mass: Houghton Mifflin; 1979. 15 Cronshaw SF, Ong PY, Chappell DB. Workers’ adaptation enables work functioning. Psychol Rep. 2007;100(3 pt 2):1043–1064. 16 Maxwell JA, Mittappalli K. Realism as a stance for mixed method research. In: Tashakkori A, Teddlie C, eds. Sage Handbook of Mixed Methods in Social and Behavioural Research. 2nd ed. Thousand Oaks, Calif: Sage; 2010:241–272. 17 Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. Thousand Oaks, Calif: Sage; 1994.
18 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101. 19 Hayes N. Theory-led thematic analysis: Social identification in small companies. In: Hayes N, ed. Doing Qualitative Analysis in Psychology. Hove, UK: Psychology Press; 1997:93–114. 20 Organ DW, Konovsky M. Cognitive versus affective determinants of organizational citizenship behavior. J Appl Psychol. 1989;74:157–164. 21 Kitson AL. The need for systems change: Reflections on knowledge translation and organizational change. J Adv Nurs. 2009;65:217–228. 22 McCormack B, Kitson A, Harvey G, RycroftMalone J, Titchen A, Seers K. Getting evidence into practice: The meaning of “context.” J Adv Nurs. 2002;38:94–104. 23 Mazmanian PE. Advancing the body of knowledge: Evidence and study design for quality improvement. In: Davis DA, Barnes B, Fox R, eds. The Continuing Professional Development of Physicians: From Research to Practice. Chicago, Ill: American Medical Association Press; 2003:317–339. 24 Veldhuijzen W, Ram PM, van der Weijden T, van der Vleuten CP. Communication guidelines as a learning tool: An exploration of user preferences in general practice. Patient Educ Couns. 2013;90:213–219. 25 Dobbins M, Robeson P, Ciliska D, et al. A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci. 2009;4:23. 26 Metzler MJ, Metz GA. Analyzing the barriers and supports of knowledge translation using the PEO model. Can J Occup Ther. 2010;77:151–158. 27 Metzler MJ, Metz GA. Translating knowledge to practice: An occupational therapy perspective. Aust Occup Ther J. 2010;57:373–379. 28 Ward VL, House AO, Hamer S. Knowledge brokering: Exploring the process of transferring knowledge into action. BMC Health Serv Res. 2009;9:12. 29 Hodges B. Faculty development for research capacity building. In: Steinert Y, ed. Faculty Development in the Health Professions. Dordrecht, the Netherlands: Springer Verlag; 2014:79–96. 30 Rappolt S, Tassone M. How rehabilitation therapists gather, evaluate, and implement new knowledge. J Contin Educ Health Prof. 2002;22:170–180. 31 Wenger E. Communities of practice and social learning systems. Organization. 2000;7:225–246. 32 Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci. 1979;22:459–470. 33 McDonald PW, Viehbeck S. From evidencebased practice making to practice-based evidence making: Creating communities of (research) and practice. Health Promot Pract. 2007;8:140–144. 34 Graham ID, Tetroe J. How to translate health research knowledge into effective healthcare action. Healthc Q. 2007;10:20–22.
523
Research Report
Appendix 1 Interview Guide Used in a Qualitative Study of 15 Medical Educators, Relating to the Affordances of Empirical Knowledge Use in Medical Education, 2012 1.
What is your role with regard to teaching and curriculum development?
2.
What resources, if any, do you draw on for support as you develop the curriculum for your course?
Probes: Have you felt the need to use any research sources on medical education?
Have you felt the need to use any guides (electronic/books) on medical education?
Have you found the research resources to be useful? How so? Can you provide examples?
Have you experienced any challenges with regard to accessing resources on medical education? Can you provide examples?
Have you experienced any challenges with regard to utilizing resources on medical education? Can you provide examples?
Are there any other resources that you use to inform the development of your curriculum? Can you provide examples?
3.
Are there specific teaching/educational strategies that you employ on a consistent basis with regard to teaching (a) clinical skills, (b) technical skills, knowledge, etc?
Probes: Tell me more about these strategies.
How did you identify these strategies?
Why do you think these strategies are effective?
Do you use empirical sources to identify these strategies?
4.
What methods do you use to assess student progress (evaluate students) in your courses: (a) clinical skills, (b) technical skills, knowledge, etc?
Probes: Why do you choose to use these methods? 5.
Have you found these methods to be effective?
In your experience, have you found that there are certain “best practices” with regard to teaching medical students (a) clinical skills, (b) nonclinical skills?
Probes: Why or why not?
(If so) how did you identify these “best practices”?
Can you integrate these practices within your curriculum? How so? Can you provide examples?
Are there challenges to integrating these practices within the curriculum?
How do you integrate these practices within the curriculum for the courses you direct?
What do you think are some of the barriers to identifying “best practices” for medical education?
6.
Has your approach to instructing/directing your course evolved over time?
Probes: How so?
What led you to make that change?
What do you think has informed the changes you made? What has been your experience with regard to introducing changes into the curriculum?
Why did you adopt that strategy? Did you experience any challenges as you worked modify your curriculum/ teaching approach?
Do you adjust your instructional approach depending on the needs of the students?
Do you think medical education research has influenced the changes in your approach? How so?
7.
What do you think about the research evidence in medical education?
8.
Is there any other information about your teaching practices and/or medical education research/practice that you would like to share?
524
Academic Medicine, Vol. 90, No. 4 / April 2015
