BJD
British Journal of Dermatology
DE RM A TO P A T HO LO G Y
iSlide: a ‘big picture’ interactive teledermatopathology e-learning system P. Lee,1,2 C.-F. Chen,3,4,5,6,7 H.-T. Wan,8 W.-S. Jian,9,10 M.-H. Hsu,2,5 S. Syed-Abdul,2,5 C.-W. Huang,2,5 Y.-C. Huang,11 Y.-T. Lin,11 T.-J. Chen,11 Y.-H. Wu12,13 and Y.-C. Li2,5,11 1
Institute of Biomedical Informatics, National Yang-Ming University, Taipei, Taiwan College of Medical Science and Technology, 5Graduate Institute of Biomedical Informatics, 6Department of Public Health, 7Center for Evidence-Based Medicine, 8 Office of Biomedical Informatics and 9School of Health Care Administration, Taipei Medical University, Taipei, Taiwan 3 Evidence Based Medicine Center and 4Division of Plastic Surgery, Department of Surgery, 11Department of Dermatology, Taipei Medical University – Wan Fang Hospital, Taipei, Taiwan 10 Faculty of Health Sciences, Macau University of Science and Technology, Macau 12 Mackay Medical College, New Taipei City, Taiwan 13 Department of Dermatology, Mackay Memorial Hospital, Taipei, Taiwan 2
Summary Correspondence Yu-Chuan (Jack) Li. E-mail: [email protected] Yu-Hung Wu. E-mail: [email protected]
Accepted for publication 6 July 2014
Funding sources This research is sponsored in part by National Science Council (NSC) under grant NSC 992511-S-038-005-MY3, Ministry of Health and Welfare (MOHW), Taiwan, under grant MOHW103-TD-B-111-01, Taipei Medical University under grant 99TMU-WFH-10, 101TMU-SHH-21, TMU102-AE1-B31, Taipei Medical University and Taipei Medical University Hospital (101-TMU-TMUH-03) and Ministry of Education, Taiwan, under grant TMUTOP103006-6.
Conflicts of interest None declared. P.L. and W.-S.J contributed equally. DOI 10.1111/bjd.13274
Background Dermatopathology training is often limited by facilities and a dearth of specialists. Advancements in information and communication technologies have made possible the adoption of innovative learning techniques, especially in places where specialists are lacking. Objectives To implement and evaluate the performance of the iSlide system, which is an interactive dermatopathology training platform (http://scope.tmu.edu.tw/ islide2/). Methods Fifty-two cases representing a variety of dermatopathology conditions and complications were used to set up the iSlide system, and virtual slides of these cases were produced. Medical students from the Dermatology Department of Taipei Medical University were taught to use the system. Performance of the system was evaluated and validated using questionnaires, the first comprising 20 questions and the second a shorter, six-question telephone-based survey on 15 of the 96 interns. Twenty cases prepared by the iSlide system were also presented at an international dermatopathology conference and evaluated by conference participants. Results Ninety-six students and 72 experts participated in the study. Ninety-two per cent of the students and 98% of the experts found the iSlide system to be a useful tool for learning dermatopathology. Of these, 82% of the students and 63% of the experts felt that iSlide was easy to use. Conclusions iSlide is useful for dermatopathology. As only 82% of the student evaluators and 63% of the expert evaluators found the system easy to use, further work has to be done to improve the iSlide interface to make the system more user friendly.
What’s already known about this topic?
• • •
692
Teledermatopathology is the application of telemedicine to dermatopathology. Teledermatopathology is potentially similar to face-to-face interaction in dermatopathology. Through the effective use of health information technology, telemedicine makes it possible for health care to be universally and pervasively accessible to all.
British Journal of Dermatology (2014) 172, pp692–699
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 693
What does this study add?
• •
The iSlide system utilizes digital slides to improve training efficiency and reduces education and training costs in dermatopathology. The iSlide e-learning platform can integrate with social network applications such as Facebook to further enhance interactive discussions.
Dermatopathology is essential for the diagnosis of skin disorders. However, dermatopathology is not included in the core curriculum of dermatology training for medical students in Taiwan. Moreover, the shortage of multiheaded microscopes, which are typically required for dermatopathology training, further limits training. A few years ago, attempts were made to improve the learning environment by using digital ‘virtual slides’ in place of the traditional light microscope for both group and individual learning.1–4 Teledermatology was first discussed in 1995 by Perednia and Brown.5 Since then, teledermatology has grown into a medical subspecialty and is one of the most common applications of telemedicine and e-health systems.5–7 Telecommunication technologies can exchange medical information about skin conditions and tumours using audio, visual and data communication.8–11 Group learning and discussions in traditional clinicopathological conferences rely on prepared pathology photographs or glass slide demonstrations.12–14 The advances in internet technology and in the digitalization of glass slides using scanning systems to produce high-resolution digital images have made the online viewing of learning materials possible and convenient.1,15–18 With advances in information technology, medical education has relied extensively on computer- and internet-based training.19–22 Dermatopathology, with its visual characteristics, is particularly suitable for such applications. An integrated system of teledermatology and teledermatopathology has been proposed to provide user-friendly learning experiences in the diagnoses of skin diseases based on clinicopathological correlations. This educational web application approach was designed for use by medical students, interns, dermatologists and pathologists without the limitation of time or facilities. The system also creates a platform on which experts can discuss complicated and interesting cases through the use of social networks such as Facebook. The primary goal of this study was to evaluate the usefulness of an internetbased system for the training students in dermatopathology using skin diseases with different prevalence and complications. The second goal was to evaluate the use of this system by experts who are already familiar with reading conventional dermatopathological slides.
named iSlide because it is an internet-based training program. The concept, structure and e-learning platform were discussed and created by the iSlide committee, which comprised dermatologists, dermatopathologists and bioinformaticians. The system is managed and maintained remotely as a telemedicine system. The study was approved in 2010 by the university’s institutional review board (approval number: 99046). Research model The iSlide technology acceptance model is considered to be one of the best studied and tested theories of technology acceptance.23 This model was used to validate the usefulness of iSlide as a training tool for dermatopathology education. Our proposed research model, which is based on the technology acceptance model and the reviewed literature, is shown in Figure 1. According to this model, ‘actual use’ and ‘acceptance’ are functions of the concrete variables ‘perceived usefulness’ and ‘perceived ease of use’.24 Perceived usefulness is defined as ‘the degree to which a person believes that using a particular system would enhance his or her job performance’. Perceived ease of use refers to ‘the degree to which a person believes that using a particular system would be free of effort’.25 A subjective norm is defined as ‘a person’s perception that most people who are important to him think he should or should not perform the behavior in question’.26 The results of two studies by Bhattacherjee suggest that perceived usefulness is a significant determinant of user satisfaction.27,28 Some studies have indicated that perceived usefulness was positively correlated with satisfaction in three different online training environments with different degrees of social presence.29 Past research has indicated that satisfaction has a positive effect on future repurchase intentions. Therefore, user satisfaction has a positive effect on the intention to continue e-learning.29,30
Materials and methods The iSlide system was developed by Taipei Medical University in 2010 (http://scope.tmu.edu.tw/islide2/). The project was © 2014 British Association of Dermatologists
Fig 1. Research model based on the technology acceptance model. British Journal of Dermatology (2014) 172, pp692–699
694 iSlide teledermatopathology e-learning system, P. Lee et al.
Design of the evaluation tools To analyse the intentions and adoption behaviours of the end users, a 20-item questionnaire about participants’ perceptions was designed according to the technology acceptance model. The survey adopted a five-point Likert scale to measure each participant’s attitude. Participants were asked to indicate how they felt about the survey statements: 5 = strongly agreed, 4 = agreed, 3 = were neutral, 2 = disagreed, 1 = strongly disagreed. Design of the web-based interface The educational system was designed to contain clinical histories, digitalized skin images and interactive dermatopathology. An AperioTM digital slice scanning system (Leica Biosystems, Richmond, VA, U.S.A.) was used to scan pathology glass slides. This portal provides a platform service based on the transmission of digital images following the store-and-forward method. End users can view the materials through internet browsers without the installation of any specific software. Users can also mark and annotate each slide. Moreover, the iSlide system was integrated with a Facebook application. Annotations can be posted in the Facebook group entitled ‘iSlide Community’ and read by other members. The instructors can provide comments about complicated cases and participate in discussions. Preparation of teaching materials A number of cases were collected and imported into iSlide. These cases were selected according to variation in three domains: teaching value, challenge and rarity. Every case was graded according to its appropriate educational value, difficulty of diagnosis and frequency with which that type of case is observed in the clinic. In a consensus meeting, four senior dermatologists were involved in grading and selection of cases. These cases were further classified into four categories: inflammatory diseases, noninflammatory diseases, infections and tumours. The cases were selected and distributed into different categories and domains. All cases are associated with a linked PowerPoint (Microsoft, Redmond, WA, U.S.A.) file, which contains the chief complaint, objective findings, past history, clinical diagnosis, clinical pictures, histopathological findings, pathological diagnosis and clinical course of the case. These PowerPoint files can be used to augment the virtual slides to further enhance learning.
registered for the self-assessment session. These participants were board-certified dermatologists, pathologists, dermatopathologists, and residents and fellows in related fields. Design of the study Figure 2 summarizes the study design. Final-year medical students received 3 weeks of internship training in the dermatology department. On their first day of orientation, the students were taught the learning objectives of the iSlide system and how to operate it. The students could access the system at anytime from anywhere so long as they had internet access. They were encouraged to provide feedback within the iSlide system by submitting comments and views. The students were asked to review at least 20 cases during the training period, and they were informed that a general open discussion would be arranged at the end of their course. They were also asked to complete the questionnaire to evaluate the iSlide system. The learning experience was re-evaluated with a second questionnaire 6 months later. The second questionnaire, comprising only six questions, was implemented using a telephone survey. The first five questions were taken from the previous questionnaire. The sixth question asked whether the participants had completed the questionnaire without any pressure. The iSlide system was then presented at an international dermatopathology symposium. Ten cases selected from the database and 10 new cases were provided for the registered participants for a self-assessment session. The participants were able to preview the 20 slides
Participant enrolment The first phase of the study enrolled final-year interns from the dermatology department of Taipei Medical University. These interns had basic knowledge of dermatology and pathology but no prior training in dermatopathology. The second part of this study recruited participants who attended an international dermatopathology conference and British Journal of Dermatology (2014) 172, pp692–699
Fig 2. iSlide study design. © 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 695
1 month prior to the conference. The answers for 10 of the self-assessment cases were provided in the discussion during the meeting. The diagnoses and short discussions of the other 10 cases in the database were provided online 1 week after the conference. The participants were able to continue to use the system for 1 month after the conference and to provide their user experience and comments in the questionnaire.
Results Teaching materials Fifty-four cases were selected as the teaching materials for the first phase of the study. The cases were appropriately distributed in terms of the teaching domains described earlier. Regarding the teaching value domain, 43% of the 54 cases were appropriate for teaching in basic dermatopathology, 44% of the cases were suitable for departmental clinicopathological conference, and 13% of the cases were appropriate for submission to regional meetings. Regarding challenge value, 11% of the cases were suitable for medical students, 65% of the cases were appropriate for junior and senior residents, and 24% of the cases required the experience of general pathologists and dermatopathologists to answer. Regarding the rarity domain, 13% of the cases were observed weekly in clinics, 24% of the cases were observed monthly, 41% of the cases were observed yearly and 22% of the cases were rare (Fig. 3). Results of student evaluation Ninety-six students who trained in the dermatology department from October 2010 to June 2013 joined the study and
completed the questionnaire (Table 1). Of these students, 93% agreed that using iSlide had enabled them to learn dermatopathology more efficiently, and 89% stated that they would recommend iSlide to their classmates and would continue to use it during their professional careers. Ninety-seven per cent of the students agreed that iSlide had improved the quality of their learning in dermatopathology, and 80% agreed that learning to use iSlide was easy. The questionnaire contained reverse-coded questions to provide a validity check of the answers. Among the students, 62% disagreed with the statement that iSlide was inconvenient to use. Overall, 92% of the students agreed that iSlide was a useful tool for learning dermatopathology and 85% agreed that iSlide was easy to use (Fig. 4). Six months after training, 15 of the 96 students were randomly selected for a second telephone survey. The telephone survey revealed that 80% of the students agreed that iSlide was a useful tool for learning dermatopathology. The symmetrical measures showed that the major difference between the answers provided on the two surveys occurred for the item ‘Overall, iSlide is easy to use’ (asymptotic standard error was 016). Result for the experts iSlide was integrated with a self-assessment session at the 2013 Taiwan Dermatopathology Conference. There were 139 attendees, including 99 dermatologists and 40 pathologists. Among the 139 attendees, 72 registered for the self-assessment session and were able to log into the iSlide system. Fifty-one questionnaires were collected. Ninety-eight per cent of the respondents agreed that iSlide was a useful tool for learning dermatopathology. In contrast, 63% of the experts
Fig 3. Materials used in the iSlide system. Fifty-two cases were selected for this study. These cases were assessed with regard to the domain’s ‘teaching value’, ‘challenge’ and ‘rarity’. © 2014 British Association of Dermatologists
British Journal of Dermatology (2014) 172, pp692–699
British Journal of Dermatology (2014) 172, pp692–699
These questions were reverse-scored to avoid response bias.
a
Part 1: Perceived usefulness in learning According to your understanding of iSlide regarding its perceived usefulness in learning, please tick the appropriate box for each question 11 Using iSlide in my training can enable me to learn dermatopathology more efficiently 12 Using iSlide in my training can improve my dermatopathology 13 Using iSlide can make it easier to complete the dermatopathology training 14 I will recommend iSlide to my classmates 15 I will keep using it during my professional career 16 iSlide is an efficient and time-saving way to learn dermatopathology 17 iSlide is a valuable tool for studying dermatopathology 18 iSlide can improve the quality of my learning in dermatopathology 19 iSlide allows me to learn more about dermatopathology 110 Overall, iSlide is a useful tool for learning dermatopathology Part 2: Perceived ease of use According to your understanding of iSlide on perceived ease of use, please tick the appropriate box for each question 21 It is easy for me to learn how to use iSlide 22 I find it is easy to get iSlide to do what I want it to do 23 It is easy to learn how to use iSlide 24 iSlide can be confusinga 25 iSlide can be misleadinga 26 iSlide can be frustratinga 27 iSlide is a useful program for revision 28 iSlide is inconvenienta 29 I find the application guidance tool of iSlide very useful 210 Overall, iSlide is easy to use
Table 1 The questionnaire and the questionnaire results for interns
1 1 1 1 1 2 2 1 1 1
2 2 1 52 53 48 2 54 2 1
0 0 0 0 0 0 0 0 0
0 0 0 6 10 10 0 7 0 0
Disagree (%)
0
Strongly disagree (%)
18 18 24 24 24 26 17 23 16 14
10 13 10 23 2 9 7
6 9
6
Neutral (%)
65 62 57 13 7 13 71 12 69 70
54 53 58 51 66 50 56
59 59
54
Agree (%)
15 17 18 5 5 3 10 4 13 14
34 33 29 24 31 40 35
33 30
39
Strongly agree (%)
696 iSlide teledermatopathology e-learning system, P. Lee et al.
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 697
(a)
(b)
Fig 4. Questionnaire results reported by (a) the 96 interns (medical students in their final year of school) and (b) the 72 experts (who registered for the self-assessment session at the 2013 Taiwan Dermatopathology Conference).
agreed that iSlide was easy to use, and 85% of interns agreed with this statement. Some (4%) of the experts disagreed with this item (Fig. 4).
Discussion Several studies have shown the value of virtual microscopy with regard to feasibility, efficiency and diagnostic accuracy.31–35 Our study is the first to evaluate the use of teledermatopathology for training purposes for both interns and experts. This study demonstrated the effects of and improvement to the implementation of an online e-learning system for dermatology and dermatopathology training. The system provided an effective learning tool and may change the core curriculum for the study of skin disorders. The system reduces the barriers and limitations of facility-based microscopic learning and provides an alternative method for group education. The diagnosis of skin diseases often relies on the correlations between clinical images and pathological findings.36,37 This system is a good method for providing easily accessed pathology-based learning to beginners. One of the advantages of the iSlide is that many users, including teachers and students, can browse the same slide at © 2014 British Association of Dermatologists
the same time. The iSlide serves as a ubiquitous e-learning platform. Students can access the iSlide system anytime, anywhere. The second benefit is that instructors can design or demonstrate their teaching materials in advance. It is not necessary to teach students individually on a microscope, which is time consuming. The iSlide system is available 24 h a day, 7 days a week, which students can make use of as long as they have internet access, including on mobile devices. The system also resolves the problems of obtaining teaching slides, which are not easy to collect, and stains fade with time. The digitalized slides can preserve tissue colour and details. When managed properly, these slides can easily be retrieved from a database. Moreover, this system is a continually self-developing one that provides an excellent and inexpensive platform for people working in this field on which to share their cases An interesting phenomenon that we observed in this study is that although we assessed user satisfaction with iSlide for learning dermatopathology, the experts reported a lower satisfaction with the user interface. Some of the reasons for these lower ratings included the time taken to download the images and that the quality of the images was sometimes poorer than those on microscopes, which can be adjusted to achieve the desired focus. However, for iSlide, the focus was decided British Journal of Dermatology (2014) 172, pp692–699
698 iSlide teledermatopathology e-learning system, P. Lee et al.
during the scanning process. This is a technical drawback that needs to be addressed in the future. There are two limitations to this study. Firstly, this study was based on the self-reports of participants. Questionnaire results are often subjective. We were only able to analyse the behaviour and attitudes of the users. No valid measure of changes in knowledge was collected. A valid test or measurement should be designed to evaluate objectively changes in user ability. Secondly, the telephone survey may have been subject to recall bias, and the symmetrical measures revealed a difference. In conclusion, iSlide is a useful tool for the learning of either dermatology or dermatopathology for both interns and experts. Most of the participants in this study agreed that learning was more efficient with iSlide and would recommend it to their colleagues. The platform created an excellent independent learning environment. Future enhancement of iSlide will be focused on improving its user interface, especially in the light of comments from the expert users.
Acknowledgements We thank Professor Kwok Chan Lun, Professorial Fellow (Health Informatics) Department of Information Systems, School of Computing National University of Singapore, for providing very helpful feedback on a draft of the manuscript.
References 1 Massone C, Brunasso AMG, Biscak TM et al. Teledermatopathology. In: Telemedicine in Dermatology (Soyer PH, Binder M, Smith AC, Wurm EMT, eds). Berlin, Heidelberg: Springer, 2012; 57–65. 2 Krupinski EA. Virtual slide telepathology workstation-of-thefuture: lessons learned from teleradiology. Semin Diagn Pathol 2009; 26:194–205. 3 Weinstein RS, Graham AR, Richter LC et al. Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. Hum Pathol 2009; 40:1057–69. 4 Oikonomou S, Gwynedd Y, Trust NWN. Teledermatology: digital revolution in the management of skin disease. In Proceedings of the 2nd WSEAS International Conference on Biomedical Electronics and Biomedical Informatics. Stevens Point, WI: World Scientific and Engineering Academy and Society, 2009; 117–21. 5 Perednia DA, Brown NA. Teledermatology: one application of telemedicine. Bull Med Libr Assoc 1995; 83:42–7. 6 Massone C, Wurm EM, Hofmann-Wellenhof R et al. Teledermatology: an update. Semin Cutan Med Surg 2008; 27:101–5. 7 Guler NF, Ubeyli ED. Theory and applications of telemedicine. J Med Syst 2002; 26:199–220. 8 Vollmar HC, Schurer-Maly CC, Frahne J et al. An e-learning platform for guideline implementation–evidence- and case-based knowledge translation via the Internet. Methods Inf Med 2006; 45:389–96. 9 Hugenholtz NI, de Croon EM, Smits PB et al. Effectiveness of elearning in continuing medical education for occupational physicians. Occup Med (Lond) 2008; 58:370–2. 10 Mazzoleni MC, Rognoni C, Finozzi E et al. Usage and effectiveness of e-learning courses for continuous medical education. Stud Health Technol Inform 2009; 150:921–5.
British Journal of Dermatology (2014) 172, pp692–699
11 Carriero A, Zobel BB, Bonomo L et al. E-learning in radiology: Italian multicentre experience. Radiol Med 2011; 116:989–99. 12 Harden RM. A new vision for distance learning and continuing medical education. J Contin Educ Health Prof 2005; 25:43–51. 13 Taylor L, Abbott P, Hudson K. E-learning for health-care workforce development. Yearb Med Inform 2008: 3:83–7. 14 Hu SW, Foong HB, Elpern DJ. Virtual grand rounds in dermatology: an 8-year experience in web-based teledermatology. Int J Dermatol 2009; 48:1313–19. 15 Letinweber B, Massone C, Kodama K et al. Teledermatopathology: a controlled study about diagnostic validity and technical requirements for digital transmission. Am J Dermatopathol 2006; 28:413– 16. 16 Ward R, Moule P. A Study of ICT use and developments in supporting pre-registration students in practice. Stud Health Technol Inform 2006; 122:158. 17 Reime MH, Harris A, Aksnes J et al. The most successful method in teaching nursing students infection control–E-learning or lecture? Nurse Educ Today 2008; 28:798–806. 18 Kanthraj GR. Newer insights in teledermatology practice. Indian J Dermatol Venereol Leprol 2011; 77:276–87. 19 Nast A, Sch€afer-Hesterberg G, Zielke H et al. Online lectures for students in dermatology: a replacement for traditional teaching or a valuable addition? J Eur Acad Dermatol Venereol 2009; 23:1039–43. 20 Desai B, McKoy K, Kovarik C. Overview of international teledermatology. Pan Afr Med J 2010; 20:3. 21 Tsang MW, Kovarik CL. The role of dermatopathology in conjunction with teledermatology in resource-limited settings: lessons from the African Teledermatology Project. Int J Dermatol 2011; 50:150–6. 22 Pak HS. Teledermatology and teledermatopathology. Semin Cutan Med Surg 2002; 21:179–89. 23 Jian W-S, Syed-Abdul S, Sood S et al. Factors influencing consumer adoption of USB-based Personal Health Records in Taiwan. BMC Health Services Res 2012; 12:277. 24 Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inform 2010; 43:159–72. 25 Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989; 35:982–1003. 26 Fishbein M, Ajzen I. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley, 1975. 27 Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS Q 2001a; 25:351–70. 28 Bhattacherjee A. An empirical analysis of the antecedents of electronic commerce service continuance. Decision Support Syst 2001b; 32:201–14. 29 Roca JC, Chiu C-M, Martınez FJ. Understanding e-learning continuance intention: an extension of the Technology Acceptance Model. Int J Hum Comput Stud 2006; 64:683–96. 30 Chen M-L, Su Z-Y, Wu T-Y et al. Influence of dentistry students’elearning satisfaction: a questionnaire survey. J Med Syst 2011; 35:1595–603. 31 Massone C, Peter Soyer H, Lozzi GP et al. Feasibility and diagnostic agreement in teledermatopathology using a virtual slide system. Hum Pathol 2007; 38:546–54. 32 Koch LH, Lampros JN, Delong LK et al. Randomized comparison of virtual microscopy and traditional glass microscopy in diagnostic accuracy among dermatology and pathology residents. Hum Pathol 2009; 40:662–7. 33 Nielsen PS, Lindebjerg J, Rasmussen J et al. Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. Hum Pathol 2010; 41:1770–6.
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 699 34 Van der Heijden J, De Keizer N, Bos J et al. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br J Dermatol 2011; 165:1058–65. 35 Riedl E, Asgari M, Alvarez D et al. A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology. Dermatol Pract Concept 2012; 2:202a02.
© 2014 British Association of Dermatologists
36 Ferrara G, Argenyi Z, Argenziano G et al. The influence of clinical information in the histopathologic diagnosis of melanocytic skin neoplasms. PLoS ONE 2009; 4:e5375. 37 Cerroni L, Argenyi Z, Cerio R et al. Influence of evaluation of clinical pictures on the histopathologic diagnosis of inflammatory skin disorders. J Am Acad Dermatol 2010; 63:647–52.
British Journal of Dermatology (2014) 172, pp692–699
This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.
British Journal of Dermatology
DE RM A TO P A T HO LO G Y
iSlide: a ‘big picture’ interactive teledermatopathology e-learning system P. Lee,1,2 C.-F. Chen,3,4,5,6,7 H.-T. Wan,8 W.-S. Jian,9,10 M.-H. Hsu,2,5 S. Syed-Abdul,2,5 C.-W. Huang,2,5 Y.-C. Huang,11 Y.-T. Lin,11 T.-J. Chen,11 Y.-H. Wu12,13 and Y.-C. Li2,5,11 1
Institute of Biomedical Informatics, National Yang-Ming University, Taipei, Taiwan College of Medical Science and Technology, 5Graduate Institute of Biomedical Informatics, 6Department of Public Health, 7Center for Evidence-Based Medicine, 8 Office of Biomedical Informatics and 9School of Health Care Administration, Taipei Medical University, Taipei, Taiwan 3 Evidence Based Medicine Center and 4Division of Plastic Surgery, Department of Surgery, 11Department of Dermatology, Taipei Medical University – Wan Fang Hospital, Taipei, Taiwan 10 Faculty of Health Sciences, Macau University of Science and Technology, Macau 12 Mackay Medical College, New Taipei City, Taiwan 13 Department of Dermatology, Mackay Memorial Hospital, Taipei, Taiwan 2
Summary Correspondence Yu-Chuan (Jack) Li. E-mail: [email protected] Yu-Hung Wu. E-mail: [email protected]
Accepted for publication 6 July 2014
Funding sources This research is sponsored in part by National Science Council (NSC) under grant NSC 992511-S-038-005-MY3, Ministry of Health and Welfare (MOHW), Taiwan, under grant MOHW103-TD-B-111-01, Taipei Medical University under grant 99TMU-WFH-10, 101TMU-SHH-21, TMU102-AE1-B31, Taipei Medical University and Taipei Medical University Hospital (101-TMU-TMUH-03) and Ministry of Education, Taiwan, under grant TMUTOP103006-6.
Conflicts of interest None declared. P.L. and W.-S.J contributed equally. DOI 10.1111/bjd.13274
Background Dermatopathology training is often limited by facilities and a dearth of specialists. Advancements in information and communication technologies have made possible the adoption of innovative learning techniques, especially in places where specialists are lacking. Objectives To implement and evaluate the performance of the iSlide system, which is an interactive dermatopathology training platform (http://scope.tmu.edu.tw/ islide2/). Methods Fifty-two cases representing a variety of dermatopathology conditions and complications were used to set up the iSlide system, and virtual slides of these cases were produced. Medical students from the Dermatology Department of Taipei Medical University were taught to use the system. Performance of the system was evaluated and validated using questionnaires, the first comprising 20 questions and the second a shorter, six-question telephone-based survey on 15 of the 96 interns. Twenty cases prepared by the iSlide system were also presented at an international dermatopathology conference and evaluated by conference participants. Results Ninety-six students and 72 experts participated in the study. Ninety-two per cent of the students and 98% of the experts found the iSlide system to be a useful tool for learning dermatopathology. Of these, 82% of the students and 63% of the experts felt that iSlide was easy to use. Conclusions iSlide is useful for dermatopathology. As only 82% of the student evaluators and 63% of the expert evaluators found the system easy to use, further work has to be done to improve the iSlide interface to make the system more user friendly.
What’s already known about this topic?
• • •
692
Teledermatopathology is the application of telemedicine to dermatopathology. Teledermatopathology is potentially similar to face-to-face interaction in dermatopathology. Through the effective use of health information technology, telemedicine makes it possible for health care to be universally and pervasively accessible to all.
British Journal of Dermatology (2014) 172, pp692–699
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 693
What does this study add?
• •
The iSlide system utilizes digital slides to improve training efficiency and reduces education and training costs in dermatopathology. The iSlide e-learning platform can integrate with social network applications such as Facebook to further enhance interactive discussions.
Dermatopathology is essential for the diagnosis of skin disorders. However, dermatopathology is not included in the core curriculum of dermatology training for medical students in Taiwan. Moreover, the shortage of multiheaded microscopes, which are typically required for dermatopathology training, further limits training. A few years ago, attempts were made to improve the learning environment by using digital ‘virtual slides’ in place of the traditional light microscope for both group and individual learning.1–4 Teledermatology was first discussed in 1995 by Perednia and Brown.5 Since then, teledermatology has grown into a medical subspecialty and is one of the most common applications of telemedicine and e-health systems.5–7 Telecommunication technologies can exchange medical information about skin conditions and tumours using audio, visual and data communication.8–11 Group learning and discussions in traditional clinicopathological conferences rely on prepared pathology photographs or glass slide demonstrations.12–14 The advances in internet technology and in the digitalization of glass slides using scanning systems to produce high-resolution digital images have made the online viewing of learning materials possible and convenient.1,15–18 With advances in information technology, medical education has relied extensively on computer- and internet-based training.19–22 Dermatopathology, with its visual characteristics, is particularly suitable for such applications. An integrated system of teledermatology and teledermatopathology has been proposed to provide user-friendly learning experiences in the diagnoses of skin diseases based on clinicopathological correlations. This educational web application approach was designed for use by medical students, interns, dermatologists and pathologists without the limitation of time or facilities. The system also creates a platform on which experts can discuss complicated and interesting cases through the use of social networks such as Facebook. The primary goal of this study was to evaluate the usefulness of an internetbased system for the training students in dermatopathology using skin diseases with different prevalence and complications. The second goal was to evaluate the use of this system by experts who are already familiar with reading conventional dermatopathological slides.
named iSlide because it is an internet-based training program. The concept, structure and e-learning platform were discussed and created by the iSlide committee, which comprised dermatologists, dermatopathologists and bioinformaticians. The system is managed and maintained remotely as a telemedicine system. The study was approved in 2010 by the university’s institutional review board (approval number: 99046). Research model The iSlide technology acceptance model is considered to be one of the best studied and tested theories of technology acceptance.23 This model was used to validate the usefulness of iSlide as a training tool for dermatopathology education. Our proposed research model, which is based on the technology acceptance model and the reviewed literature, is shown in Figure 1. According to this model, ‘actual use’ and ‘acceptance’ are functions of the concrete variables ‘perceived usefulness’ and ‘perceived ease of use’.24 Perceived usefulness is defined as ‘the degree to which a person believes that using a particular system would enhance his or her job performance’. Perceived ease of use refers to ‘the degree to which a person believes that using a particular system would be free of effort’.25 A subjective norm is defined as ‘a person’s perception that most people who are important to him think he should or should not perform the behavior in question’.26 The results of two studies by Bhattacherjee suggest that perceived usefulness is a significant determinant of user satisfaction.27,28 Some studies have indicated that perceived usefulness was positively correlated with satisfaction in three different online training environments with different degrees of social presence.29 Past research has indicated that satisfaction has a positive effect on future repurchase intentions. Therefore, user satisfaction has a positive effect on the intention to continue e-learning.29,30
Materials and methods The iSlide system was developed by Taipei Medical University in 2010 (http://scope.tmu.edu.tw/islide2/). The project was © 2014 British Association of Dermatologists
Fig 1. Research model based on the technology acceptance model. British Journal of Dermatology (2014) 172, pp692–699
694 iSlide teledermatopathology e-learning system, P. Lee et al.
Design of the evaluation tools To analyse the intentions and adoption behaviours of the end users, a 20-item questionnaire about participants’ perceptions was designed according to the technology acceptance model. The survey adopted a five-point Likert scale to measure each participant’s attitude. Participants were asked to indicate how they felt about the survey statements: 5 = strongly agreed, 4 = agreed, 3 = were neutral, 2 = disagreed, 1 = strongly disagreed. Design of the web-based interface The educational system was designed to contain clinical histories, digitalized skin images and interactive dermatopathology. An AperioTM digital slice scanning system (Leica Biosystems, Richmond, VA, U.S.A.) was used to scan pathology glass slides. This portal provides a platform service based on the transmission of digital images following the store-and-forward method. End users can view the materials through internet browsers without the installation of any specific software. Users can also mark and annotate each slide. Moreover, the iSlide system was integrated with a Facebook application. Annotations can be posted in the Facebook group entitled ‘iSlide Community’ and read by other members. The instructors can provide comments about complicated cases and participate in discussions. Preparation of teaching materials A number of cases were collected and imported into iSlide. These cases were selected according to variation in three domains: teaching value, challenge and rarity. Every case was graded according to its appropriate educational value, difficulty of diagnosis and frequency with which that type of case is observed in the clinic. In a consensus meeting, four senior dermatologists were involved in grading and selection of cases. These cases were further classified into four categories: inflammatory diseases, noninflammatory diseases, infections and tumours. The cases were selected and distributed into different categories and domains. All cases are associated with a linked PowerPoint (Microsoft, Redmond, WA, U.S.A.) file, which contains the chief complaint, objective findings, past history, clinical diagnosis, clinical pictures, histopathological findings, pathological diagnosis and clinical course of the case. These PowerPoint files can be used to augment the virtual slides to further enhance learning.
registered for the self-assessment session. These participants were board-certified dermatologists, pathologists, dermatopathologists, and residents and fellows in related fields. Design of the study Figure 2 summarizes the study design. Final-year medical students received 3 weeks of internship training in the dermatology department. On their first day of orientation, the students were taught the learning objectives of the iSlide system and how to operate it. The students could access the system at anytime from anywhere so long as they had internet access. They were encouraged to provide feedback within the iSlide system by submitting comments and views. The students were asked to review at least 20 cases during the training period, and they were informed that a general open discussion would be arranged at the end of their course. They were also asked to complete the questionnaire to evaluate the iSlide system. The learning experience was re-evaluated with a second questionnaire 6 months later. The second questionnaire, comprising only six questions, was implemented using a telephone survey. The first five questions were taken from the previous questionnaire. The sixth question asked whether the participants had completed the questionnaire without any pressure. The iSlide system was then presented at an international dermatopathology symposium. Ten cases selected from the database and 10 new cases were provided for the registered participants for a self-assessment session. The participants were able to preview the 20 slides
Participant enrolment The first phase of the study enrolled final-year interns from the dermatology department of Taipei Medical University. These interns had basic knowledge of dermatology and pathology but no prior training in dermatopathology. The second part of this study recruited participants who attended an international dermatopathology conference and British Journal of Dermatology (2014) 172, pp692–699
Fig 2. iSlide study design. © 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 695
1 month prior to the conference. The answers for 10 of the self-assessment cases were provided in the discussion during the meeting. The diagnoses and short discussions of the other 10 cases in the database were provided online 1 week after the conference. The participants were able to continue to use the system for 1 month after the conference and to provide their user experience and comments in the questionnaire.
Results Teaching materials Fifty-four cases were selected as the teaching materials for the first phase of the study. The cases were appropriately distributed in terms of the teaching domains described earlier. Regarding the teaching value domain, 43% of the 54 cases were appropriate for teaching in basic dermatopathology, 44% of the cases were suitable for departmental clinicopathological conference, and 13% of the cases were appropriate for submission to regional meetings. Regarding challenge value, 11% of the cases were suitable for medical students, 65% of the cases were appropriate for junior and senior residents, and 24% of the cases required the experience of general pathologists and dermatopathologists to answer. Regarding the rarity domain, 13% of the cases were observed weekly in clinics, 24% of the cases were observed monthly, 41% of the cases were observed yearly and 22% of the cases were rare (Fig. 3). Results of student evaluation Ninety-six students who trained in the dermatology department from October 2010 to June 2013 joined the study and
completed the questionnaire (Table 1). Of these students, 93% agreed that using iSlide had enabled them to learn dermatopathology more efficiently, and 89% stated that they would recommend iSlide to their classmates and would continue to use it during their professional careers. Ninety-seven per cent of the students agreed that iSlide had improved the quality of their learning in dermatopathology, and 80% agreed that learning to use iSlide was easy. The questionnaire contained reverse-coded questions to provide a validity check of the answers. Among the students, 62% disagreed with the statement that iSlide was inconvenient to use. Overall, 92% of the students agreed that iSlide was a useful tool for learning dermatopathology and 85% agreed that iSlide was easy to use (Fig. 4). Six months after training, 15 of the 96 students were randomly selected for a second telephone survey. The telephone survey revealed that 80% of the students agreed that iSlide was a useful tool for learning dermatopathology. The symmetrical measures showed that the major difference between the answers provided on the two surveys occurred for the item ‘Overall, iSlide is easy to use’ (asymptotic standard error was 016). Result for the experts iSlide was integrated with a self-assessment session at the 2013 Taiwan Dermatopathology Conference. There were 139 attendees, including 99 dermatologists and 40 pathologists. Among the 139 attendees, 72 registered for the self-assessment session and were able to log into the iSlide system. Fifty-one questionnaires were collected. Ninety-eight per cent of the respondents agreed that iSlide was a useful tool for learning dermatopathology. In contrast, 63% of the experts
Fig 3. Materials used in the iSlide system. Fifty-two cases were selected for this study. These cases were assessed with regard to the domain’s ‘teaching value’, ‘challenge’ and ‘rarity’. © 2014 British Association of Dermatologists
British Journal of Dermatology (2014) 172, pp692–699
British Journal of Dermatology (2014) 172, pp692–699
These questions were reverse-scored to avoid response bias.
a
Part 1: Perceived usefulness in learning According to your understanding of iSlide regarding its perceived usefulness in learning, please tick the appropriate box for each question 11 Using iSlide in my training can enable me to learn dermatopathology more efficiently 12 Using iSlide in my training can improve my dermatopathology 13 Using iSlide can make it easier to complete the dermatopathology training 14 I will recommend iSlide to my classmates 15 I will keep using it during my professional career 16 iSlide is an efficient and time-saving way to learn dermatopathology 17 iSlide is a valuable tool for studying dermatopathology 18 iSlide can improve the quality of my learning in dermatopathology 19 iSlide allows me to learn more about dermatopathology 110 Overall, iSlide is a useful tool for learning dermatopathology Part 2: Perceived ease of use According to your understanding of iSlide on perceived ease of use, please tick the appropriate box for each question 21 It is easy for me to learn how to use iSlide 22 I find it is easy to get iSlide to do what I want it to do 23 It is easy to learn how to use iSlide 24 iSlide can be confusinga 25 iSlide can be misleadinga 26 iSlide can be frustratinga 27 iSlide is a useful program for revision 28 iSlide is inconvenienta 29 I find the application guidance tool of iSlide very useful 210 Overall, iSlide is easy to use
Table 1 The questionnaire and the questionnaire results for interns
1 1 1 1 1 2 2 1 1 1
2 2 1 52 53 48 2 54 2 1
0 0 0 0 0 0 0 0 0
0 0 0 6 10 10 0 7 0 0
Disagree (%)
0
Strongly disagree (%)
18 18 24 24 24 26 17 23 16 14
10 13 10 23 2 9 7
6 9
6
Neutral (%)
65 62 57 13 7 13 71 12 69 70
54 53 58 51 66 50 56
59 59
54
Agree (%)
15 17 18 5 5 3 10 4 13 14
34 33 29 24 31 40 35
33 30
39
Strongly agree (%)
696 iSlide teledermatopathology e-learning system, P. Lee et al.
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 697
(a)
(b)
Fig 4. Questionnaire results reported by (a) the 96 interns (medical students in their final year of school) and (b) the 72 experts (who registered for the self-assessment session at the 2013 Taiwan Dermatopathology Conference).
agreed that iSlide was easy to use, and 85% of interns agreed with this statement. Some (4%) of the experts disagreed with this item (Fig. 4).
Discussion Several studies have shown the value of virtual microscopy with regard to feasibility, efficiency and diagnostic accuracy.31–35 Our study is the first to evaluate the use of teledermatopathology for training purposes for both interns and experts. This study demonstrated the effects of and improvement to the implementation of an online e-learning system for dermatology and dermatopathology training. The system provided an effective learning tool and may change the core curriculum for the study of skin disorders. The system reduces the barriers and limitations of facility-based microscopic learning and provides an alternative method for group education. The diagnosis of skin diseases often relies on the correlations between clinical images and pathological findings.36,37 This system is a good method for providing easily accessed pathology-based learning to beginners. One of the advantages of the iSlide is that many users, including teachers and students, can browse the same slide at © 2014 British Association of Dermatologists
the same time. The iSlide serves as a ubiquitous e-learning platform. Students can access the iSlide system anytime, anywhere. The second benefit is that instructors can design or demonstrate their teaching materials in advance. It is not necessary to teach students individually on a microscope, which is time consuming. The iSlide system is available 24 h a day, 7 days a week, which students can make use of as long as they have internet access, including on mobile devices. The system also resolves the problems of obtaining teaching slides, which are not easy to collect, and stains fade with time. The digitalized slides can preserve tissue colour and details. When managed properly, these slides can easily be retrieved from a database. Moreover, this system is a continually self-developing one that provides an excellent and inexpensive platform for people working in this field on which to share their cases An interesting phenomenon that we observed in this study is that although we assessed user satisfaction with iSlide for learning dermatopathology, the experts reported a lower satisfaction with the user interface. Some of the reasons for these lower ratings included the time taken to download the images and that the quality of the images was sometimes poorer than those on microscopes, which can be adjusted to achieve the desired focus. However, for iSlide, the focus was decided British Journal of Dermatology (2014) 172, pp692–699
698 iSlide teledermatopathology e-learning system, P. Lee et al.
during the scanning process. This is a technical drawback that needs to be addressed in the future. There are two limitations to this study. Firstly, this study was based on the self-reports of participants. Questionnaire results are often subjective. We were only able to analyse the behaviour and attitudes of the users. No valid measure of changes in knowledge was collected. A valid test or measurement should be designed to evaluate objectively changes in user ability. Secondly, the telephone survey may have been subject to recall bias, and the symmetrical measures revealed a difference. In conclusion, iSlide is a useful tool for the learning of either dermatology or dermatopathology for both interns and experts. Most of the participants in this study agreed that learning was more efficient with iSlide and would recommend it to their colleagues. The platform created an excellent independent learning environment. Future enhancement of iSlide will be focused on improving its user interface, especially in the light of comments from the expert users.
Acknowledgements We thank Professor Kwok Chan Lun, Professorial Fellow (Health Informatics) Department of Information Systems, School of Computing National University of Singapore, for providing very helpful feedback on a draft of the manuscript.
References 1 Massone C, Brunasso AMG, Biscak TM et al. Teledermatopathology. In: Telemedicine in Dermatology (Soyer PH, Binder M, Smith AC, Wurm EMT, eds). Berlin, Heidelberg: Springer, 2012; 57–65. 2 Krupinski EA. Virtual slide telepathology workstation-of-thefuture: lessons learned from teleradiology. Semin Diagn Pathol 2009; 26:194–205. 3 Weinstein RS, Graham AR, Richter LC et al. Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. Hum Pathol 2009; 40:1057–69. 4 Oikonomou S, Gwynedd Y, Trust NWN. Teledermatology: digital revolution in the management of skin disease. In Proceedings of the 2nd WSEAS International Conference on Biomedical Electronics and Biomedical Informatics. Stevens Point, WI: World Scientific and Engineering Academy and Society, 2009; 117–21. 5 Perednia DA, Brown NA. Teledermatology: one application of telemedicine. Bull Med Libr Assoc 1995; 83:42–7. 6 Massone C, Wurm EM, Hofmann-Wellenhof R et al. Teledermatology: an update. Semin Cutan Med Surg 2008; 27:101–5. 7 Guler NF, Ubeyli ED. Theory and applications of telemedicine. J Med Syst 2002; 26:199–220. 8 Vollmar HC, Schurer-Maly CC, Frahne J et al. An e-learning platform for guideline implementation–evidence- and case-based knowledge translation via the Internet. Methods Inf Med 2006; 45:389–96. 9 Hugenholtz NI, de Croon EM, Smits PB et al. Effectiveness of elearning in continuing medical education for occupational physicians. Occup Med (Lond) 2008; 58:370–2. 10 Mazzoleni MC, Rognoni C, Finozzi E et al. Usage and effectiveness of e-learning courses for continuous medical education. Stud Health Technol Inform 2009; 150:921–5.
British Journal of Dermatology (2014) 172, pp692–699
11 Carriero A, Zobel BB, Bonomo L et al. E-learning in radiology: Italian multicentre experience. Radiol Med 2011; 116:989–99. 12 Harden RM. A new vision for distance learning and continuing medical education. J Contin Educ Health Prof 2005; 25:43–51. 13 Taylor L, Abbott P, Hudson K. E-learning for health-care workforce development. Yearb Med Inform 2008: 3:83–7. 14 Hu SW, Foong HB, Elpern DJ. Virtual grand rounds in dermatology: an 8-year experience in web-based teledermatology. Int J Dermatol 2009; 48:1313–19. 15 Letinweber B, Massone C, Kodama K et al. Teledermatopathology: a controlled study about diagnostic validity and technical requirements for digital transmission. Am J Dermatopathol 2006; 28:413– 16. 16 Ward R, Moule P. A Study of ICT use and developments in supporting pre-registration students in practice. Stud Health Technol Inform 2006; 122:158. 17 Reime MH, Harris A, Aksnes J et al. The most successful method in teaching nursing students infection control–E-learning or lecture? Nurse Educ Today 2008; 28:798–806. 18 Kanthraj GR. Newer insights in teledermatology practice. Indian J Dermatol Venereol Leprol 2011; 77:276–87. 19 Nast A, Sch€afer-Hesterberg G, Zielke H et al. Online lectures for students in dermatology: a replacement for traditional teaching or a valuable addition? J Eur Acad Dermatol Venereol 2009; 23:1039–43. 20 Desai B, McKoy K, Kovarik C. Overview of international teledermatology. Pan Afr Med J 2010; 20:3. 21 Tsang MW, Kovarik CL. The role of dermatopathology in conjunction with teledermatology in resource-limited settings: lessons from the African Teledermatology Project. Int J Dermatol 2011; 50:150–6. 22 Pak HS. Teledermatology and teledermatopathology. Semin Cutan Med Surg 2002; 21:179–89. 23 Jian W-S, Syed-Abdul S, Sood S et al. Factors influencing consumer adoption of USB-based Personal Health Records in Taiwan. BMC Health Services Res 2012; 12:277. 24 Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inform 2010; 43:159–72. 25 Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989; 35:982–1003. 26 Fishbein M, Ajzen I. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley, 1975. 27 Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS Q 2001a; 25:351–70. 28 Bhattacherjee A. An empirical analysis of the antecedents of electronic commerce service continuance. Decision Support Syst 2001b; 32:201–14. 29 Roca JC, Chiu C-M, Martınez FJ. Understanding e-learning continuance intention: an extension of the Technology Acceptance Model. Int J Hum Comput Stud 2006; 64:683–96. 30 Chen M-L, Su Z-Y, Wu T-Y et al. Influence of dentistry students’elearning satisfaction: a questionnaire survey. J Med Syst 2011; 35:1595–603. 31 Massone C, Peter Soyer H, Lozzi GP et al. Feasibility and diagnostic agreement in teledermatopathology using a virtual slide system. Hum Pathol 2007; 38:546–54. 32 Koch LH, Lampros JN, Delong LK et al. Randomized comparison of virtual microscopy and traditional glass microscopy in diagnostic accuracy among dermatology and pathology residents. Hum Pathol 2009; 40:662–7. 33 Nielsen PS, Lindebjerg J, Rasmussen J et al. Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. Hum Pathol 2010; 41:1770–6.
© 2014 British Association of Dermatologists
iSlide teledermatopathology e-learning system, P. Lee et al. 699 34 Van der Heijden J, De Keizer N, Bos J et al. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br J Dermatol 2011; 165:1058–65. 35 Riedl E, Asgari M, Alvarez D et al. A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology. Dermatol Pract Concept 2012; 2:202a02.
© 2014 British Association of Dermatologists
36 Ferrara G, Argenyi Z, Argenziano G et al. The influence of clinical information in the histopathologic diagnosis of melanocytic skin neoplasms. PLoS ONE 2009; 4:e5375. 37 Cerroni L, Argenyi Z, Cerio R et al. Influence of evaluation of clinical pictures on the histopathologic diagnosis of inflammatory skin disorders. J Am Acad Dermatol 2010; 63:647–52.
British Journal of Dermatology (2014) 172, pp692–699
This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.
