Journal of Educational Evaluation for Health Professions Open Access

J Educ Eval Health Prof  2015, 12: 15  • http://dx.doi.org/10.3352/jeehp.2015.12.15

eISSN: 1975-5937

OPINION

Global differences in electronic portfolio utilization – a review of the literature and research implications Jason Chertoff* Department of Internal Medicine, University of Florida College of Medicine, Gainesville, Florida, USA

The philosophy and overall structure of medical education and knowledge assessment have undergone dramatic transformations over the last fifteen years, highlighted by the shift from summative-driven assessment methods to competencybased techniques that foster reflective learning and professional growth. Assessment of educational and clinical success in this new paradigm, although crucial, is challenging as historically assessments have been based on summative-driven methods like standardized tests and subjective instructor assessments [1,2]. The days of traditional numerical grades and pass-fail hierarchies which medical students were so accustomed are being replaced by self-assessment, introspection, and appraisal of achievements in core competencies [3-10]. To be successful, competency-based medical education (CBME) necessitates a robust and multifaceted assessment system with assessment processes that are more continuous, frequent, criterion-based, and developmental. We should use tools that meet minimum requirements for quality, and have both quantitative and qualitative measures [4]. Although this transition to competency-based medical education has the obvious potential of engendering more well-roundedness and professional maturity, the optimal methodology for assessing attainment of these competencies is less transparent [2-9]. One technique proven to be successful in reliably assessing student achievement in this new educational paradigm is the electronic portfolio (e-portfolio) [5,10-12]. The e-portfolio is a collection of electronic evidence assembled and managed by *Corresponding email: [email protected]

Received: January 26, 2015; Accepted: April 15, 2015; Published: April 19, 2015 This article is available from: http://jeehp.org/

an end user usually on the web that has the ability to longitudinally capture and visually display competency attainment data over time, has naturally gained popularity in the competency-based medical education arena. Many countries across the globe have already tapped into the inherent advantages of e-Portfolios, and implemented them into their medical education curriculum [13-15]. Sanchez Gomez et al. describes an eportfolio initiative for medical students at The University of Seville that provides quantitative feedback that can be chronologically followed over time to monitor student progress. When surveyed, all of the students answered affirmatively that the eportfolio tracked and guided their learning process, and indicated their outstanding competency deficiencies [13]. A similar report by Vernazza et al. describes a cutting-edge e-portfolio that was developed as a web-based system with access grant­ ed to students via laptops connected to a wireless network. The perceived benefits by students was that the e-portfolio allowed for additional quantity, quality, manipulation, and trending of data than previous assessment techniques [14]. Other countries around the world including the Netherlands, United Kingdom, Taiwan, and South Africa have reported implementing similar competency-based e-portfolios [14-17]. Typically, electronic portfolios are online programs with links to Web tools that allow documentation of students’ learning and assessment, while acting as a forum to showcase skills, progress, and reflections. In essence, it acts as a platform to encourage students to manage their own career planning and skill development, while offering instructors the ability to chronologically track progress and competency attainment over time [12,14]. When compared to paper-based portfolios, el-portfolios offer better customization, ease for data manipulation, and increased ability to share and transfer [12,14]. In fact, when

2015, National Health Personnel Licensing Examination Board of the Republic of Korea This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Page 1 of 3 (page number not for citation purposes)

J Educ Eval Health Prof  2015, 12: 15 • http://dx.doi.org/10.3352/jeehp.2015.12.15

compared head-to-head, users unanimously agree that e-portfolios are easier to use as they allow faster retrieval of evidence through hyperlinks, and enable access from a variety of sites at the instructor’s convenience [10-12]. Due to its unique ability to interactively engage students in professional development and the educational process, while simultaneously providing instructors with dynamic tools to track competency assessment, many countries worldwide have started implementing e-portfolios into their curriculum. Chen et al. describes an interactive and digitalized electronic portfolio system in Taiwan used to track emergency medical education competency attainment by allowing students to register, analyze, and reflectively comment on at least ten cases per day. The authors concluded that their e-portfolio enhanced competency analysis and curriculum improvement, due in large part to its innovative competency tracking functionality [15]. Sanchez Gomez et al. describes an e-portfolio, capable of automatically displaying competency feedback and providing numerical visualization of progress, that assesses medical students at The University of Seville. As described, this e-portfolio is web-based, available online via all standard browsers, and accessible via the institution’s web environment by individual username and password. In addition, this system collects and displays evidence of learning from numerous sources such as texts, presentations, images, photographs, and videos, which students can upload using laptops, tablets, smartphones, and other devices. The students that used this e-portfolio were overwhelmingly supportive, with all students reporting an improved understanding of their learning objectives due to the numerical visualization of progress. The authors concluded that their e-portfolio was highly successful in guiding students’ learning process by indicating competency gaps to themselves and teachers [13]. Scheele et al. describes major educational reforms in the Netherlands, featuring an electronic portfolio system used to assess competency attainment. In this system, medical trainees regularly gather evidence of their personal and professional development and store it in an electronic portfolio, which is shared and reviewed with educational supervisors every three months. The result is a clear two-dimensional picture of students’ progress over time that supervisors can use to assess proficiency and competency fulfillment [17]. Finally, Vernazza et al. describes the successful implementation of a web-based e-portfolio system introduced for undergraduate students in the United Kingdom, which is accessed via laptops connected to an institutional wireless network. This robust system generates large volumes of competency data by allowing regular student participation, often several times daily. The consensus among the students and supervisors that utilized this system was that it is user-friendly, provides large quantities of high-quality data, allows for better

http://jeehp.org

data manipulation as compared to paper-based systems, and gives accurate assessments of students’ progress toward competency attainment over time [14]. Despite the widespread use of e-portfolios around the globe in evaluating and longitudinally trending student aptitude, the literature review suggests that institutions in the United States may be lagging behind their global counterparts in the use of innovative competency assessment methodologies [2]. For example, Fishleder et al. described a hybrid electronic and paper portfolio system at the Cleveland Clinic Lerner College of Medicine used to assess undergraduate medical students in nine competency areas. The e-portfolio portion of the system, accessed via laptop computers, allows students to log patient encounters and receive competency-specific encounter-based feedback from faculty to refine their clinical skills. However, the electronic system that is described seems to evaluate students at isolated time periods throughout the five-year program, without the ability to trend, longitudinally track, and illustrate competency attainment, like the electronic systems of other countries [7,9]. Furthermore, a literature review by Carracio et al. investigating e-portfolios in the United States uncovered limited evidence regarding the use of e-portfolios in medical education, with only two descriptions of residency programs and no undergraduate medical schools, utilizing electronic portfolios for assessment of competency-based me­ dical education. Moreover, the e-portfolios that are discussed are not described as having the ability to longitudinally display competency attainment like the electronic portfolio systems of other countries [5]. Thus, it appears through review of the e-portfolio literature that e-portfolios are being used less frequently and are less innovative in the United States than in other countries. Across the globe, medical education has undergone dramatic transformations over the last fifteen years, highlighted by transitions in assessment techniques from traditional summative methods to competency-based approaches. One competency-based assessment technique that has gained popularity, with its ability to store large amounts of data and visually depict and track competency attainment over time, is the e-portfolio. Numerous reports show that many countries outside of the United States appear to have tapped into this robust assessment tool, detailing widespread success and satisfaction. On the contrary, similar reports from American medical schools are lacking. The purpose of this article is to inspire future research endeavors that investigate these international dissimilarities by exploring the current-state makeup of e-portfolios in American medical schools to determine whether the research, innovations, and successes of our global counterparts are being implemented.

Page 2 of 3 (page number not for citation purposes)

J Educ Eval Health Prof  2015, 12: 15 • http://dx.doi.org/10.3352/jeehp.2015.12.15

ORCID: Jason Chertoff : http://orcid.org/0000-0002-0572-5778 CONFLICT OF INTEREST No potential conflict of interest relevant to this article was reported.

REFERENCES 1. Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ. 2010;341:c5064. http://dx.doi.org/10.1136/bmj.c5064 2. Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009;84:301-309. http://dx.doi.org/ 10.1097/ACM.0b013e3181971f08 3. McNeill H, Brown JM, Shaw NJ. First year specialist trainees’ engagement with reflective practice in the e-portfolio. Adv Health Sci Educ Theory Pract. 2010;15:547-558. http://dx.doi.org/10.1007/ s10459-009-9217-8 4. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32:676-682. http://dx.doi.org/10.3109/0142159X. 2010.500704 5. Carraccio C, Englander R. Analyses/literature reviews: evaluating competence using a portfolio: a literature review and webbased application to the ACGME competencies. Teach Learn Med. 2004;16:381-387. http://dx.doi.org/10.1207/s15328015tlm 1604_13 6. Altahawi F, Sisk B, Poloskey S, Hicks C, Dannefer EF. Student perspectives on assessment: experience in a competency-based portfolio system. Med Teach. 2012;34:221-225. http://dx.doi.org /10.3109/0142159X.2012.652243 7. Fishleder AJ, Henson LC, Hull AL. Cleveland Clinic Lerner College of Medicine: an innovative approach to medical education and the training of physician investigators. Acad Med. 2007;82: 390-396. http://dx.doi.org/10.1097/ACM.0b013e318033364e 8. Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, Morley D, Pollard D, Ashcroft T, Popovic C, Sayers J. The educational effects of portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic review.

http://jeehp.org

BEME Guide No. 11. Med Teach. 2009;31:282-298. http://dx.doi. org/10.1080/01421590902889897 9. Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE Guide no. 45. Med Teach. 2009;31:790-801. http:// dx.doi.org/10.1080/01421590903139201 10. Driessen EW, Muijtjens AM, van Tartwijk J, van der Vleuten CP. Web‐ or paper‐based portfolios: is there a difference? Med Educ. 2007;41:1067-1073. http://dx.doi.org/10.1111/j.1365-2923.2007. 02859.x 11. Driessen EW, Overeem K, van Tartwijk, J, van der Vleuten CP, Muijtjens AM. Validity of portfolio assessment: which qualities determine ratings? Med Educ. 2006;40:862-866. http://dx.doi. org/10.1111/j.1365-2929.2006.02550.x 12. Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart I, Peacock H. The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12. Med Teach. 2009; 31:299-318. http://dx.doi.org/10.1080/01421590902883056 13. Sánchez Gómez S, Ostos EM, Solano JM, Salado TF. An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education. BMC Med Educ. 2013;13:65. http://dx.doi.org/10.1186/1472-6920-13-65 14. Vernazza C, Durham J, Ellis J, Teasdale D, Cotterill S, Scott L, Thomason M, Drummond P, Moss J. Introduction of an e‐portfolio in clinical dentistry: staff and student views. Eur J Dent Educ. 2011;15:36-41. http://dx.doi.org/10.1111/j.1600-0579.2010. 00631.x 15. Chen CK, Huan MN, Chang YC, Chen KF, Chen JC. 316 implementation of the e-portfolio to postgraduate year-1 residency training in emergency medicine: assessments of curriculum improvement and online feedback. Ann Emerg Med. 2011;58:S284. http://dx.doi.org/10.1016/j.annemergmed.2011.06.347 16. Jenkins L, Mash B, Derese A. The national portfolio for postgraduate family medicine training in South Africa: a descriptive study of acceptability, educational impact, and usefulness for assessment. BMC Med Educ. 2013;13:101. http://dx.doi.org/10.1186/ 1472-6920-13-101 17. Scheele F, Teunissen P, Luijk S V, Heineman E, Fluit L, Mulder H, Meininger A, Wijnen-Meijer M, Glas G, Sluiter H, Hummel T. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach. 2008;30:248-253. http://dx.doi. org/10.1080/01421590801993022

Page 3 of 3 (page number not for citation purposes)

Global differences in electronic portfolio utilization - a review of the literature and research implications.

Global differences in electronic portfolio utilization - a review of the literature and research implications. - PDF Download Free
169KB Sizes 1 Downloads 7 Views