J Am Acad Audiol 24:859–866 (2013)
Optimizing Otoscopy Competency in Audiology Students through Supplementary Otoscopy Training DOI: 10.3766/jaaa.24.9.9 Wafaa A. Kaf* Caleb G. Masterson† Nancy Dion‡ Susan L. Berg§ Mohamed K. Abdelhakiem**
Abstract Background: Scope of practice in audiology encompasses proficiency in visual inspection of ear canal and tympanic membrane (TM) as well as otoscopy interpretation skills to determine normal versus abnormal conditions of outer and middle ear. Audiology students can develop skills in otoscopy through education and supervised training. Studies have shown that additional otoscopy training increased skills in medical students and general practitioners. However, educational and supervised practices targeting otoscopy competency during audiology graduate coursework are lacking. Also, no studies have attempted to determine otoscopy skills among audiology students. Purpose: To determine the effectiveness of the otoscopy training model on clinical competency and confidence level of audiology students in performing and interpreting otoscopy. Research Design: A combination of experimental treatment design with random assignment of treatment and control groups and delayed treatment for control group. Study Sample: Thirty-two first- and second-year audiology graduate students who were enrolled in a pediatric audiology class participated in this study. Students were randomly assigned to the control (n 5 16, 14 females) or experimental (n 5 16, 14 females) group. Intervention: Participants in the experimental group received supplementary otoscopy training including didactic otoscopy lectures as well as clinical training using manikin ears. The control group received the same pretest and posttest and then completed a third assessment (posttest 2) after receiving the same training. Data Collection and Analysis: An evaluation of knowledge and skills regarding otoscopy between groups and time was conducted at three times: (a) pretraining, (b) upon completion of training for the experimental group, (c) upon completion of training by the control group. The evaluation consisted of a written exam, a clinical exam, and a self-perception rating of confidence. Both written exam scores and clinical exam scores (otoscopy manikin) were analyzed via two-way analyses of variance (ANOVAs), whereas chi-square (x2) statistic was conducted to evaluate the effects of training on the confidence level of students of both groups. Results: Experimental and control groups demonstrated significant increased overall competency in otoscopy following the otoscopy training model with didactic and laboratory components. Posttest confidence ratings showed increases in all groups, and there were no significant differences between groups.
*Communication Sciences and Disorders Department, Missouri State University, Springfield, MO; †Osteopathic Medicine, Des Moines University, Des Moines, IA; ‡Physician Assistant Department, Missouri State University, Springfield, MO; §Nursing Department, Missouri State University, Springfield, MO; **Biomedical Sciences Department, Missouri State University, Springfield, MO Wafaa Kaf, Communication Sciences and Disorders Department, Missouri State University, 901 South National Avenue, Springfield, MO 65897; Phone: 417-836-4456; Fax: 417-836-4242; E-mail: [email protected] Presented at the American Speech-Language-Hearing Association (ASHA) Convention, November 17–21, 2010, Philadelphia, PA. This study was partially supported by the Citizenship and Service-Learning Office at Missouri State University.
859
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Conclusions: The need for supplementary otoscopy training was warranted by low knowledge and clinical competency in otoscopy skills of audiology students as measured by pretest mean scores. After completing the training, both experimental and control groups showed significant improvement in knowledge and competency. Results also suggest that perceived confidence ratings may be misleading in determining students’ clinical otoscopy skills. Key Words: Audiology students, competency, knowledge, otoscopy, skills, training Abbreviations: ASHA 5 American Speech-Language-Hearing Association; CSOM 5 chronic suppurative otitis media; OM 5 otitis media; SOM 5 serous otitis media; TM 5 tympanic membrane
O
toscopy is the standard for conducting an external and middle ear examination. Audiologists, pediatricians, ENT surgeons, and other healthcare providers perform otoscopy on a regular basis. Otoscopy helps an examiner to determine the presence of obstructing material in the ear canal, signs of congenital anomalies, or active infections of the outer or middle ear that may prohibit completion of pure-tone audiometry, tympanometry, or other audiological procedures (American Speech-Language-Hearing Association [ASHA], 1992). In addition, careful otoscopic examination is important to visualize pathologies that may not be detected with tympanometry or other audiological testing, thus allowing for appropriate referral. Furthermore, the use of video otoscopy provides additional valuable documentation for monitoring of medical conditions or referrals (Sullivan, 1993, 1997). The scope of otoscopy practice for audiologists includes ruling out contraindications for the use of earphone or probe and knowledge of anatomy and recognition of certain pathology of the outer ear and middle ear. These skills allow for appropriate referral for treatment of common outer and middle ear diseases (ASHA, 1992, 1997; American Academy of Audiology, 2004). According to standard IV-C4 of the Standards for the Certificate of Clinical Competence in Audiology, students applying for certification “must have the knowledge and skills necessary in performing otoscopy” (Council for Clinical Certification in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association [CFCC], 2012), indicating that audiologists should have the necessary training in otoscopy and knowledge of normal and abnormal conditions of the ear (ASHA, 1992). However, an online search for posted audiology curriculum from a variety of audiology programs including Missouri State University revealed that their curriculum and course descriptions do not specify any otoscopy training. Therefore, educational practices targeting otoscopy competency for audiology graduates may be inadequate due to lack of appropriate audiology course work and curricula. Scudder et al (2003) have reported high interrater reliability of otoscopy among graduate audiology and speech-language pathology students despite their variable level educational background and otoscopic training. However, this finding was related to their rating of whether earwax was excessive or impacted, not to
860
determine normal versus abnormal conditions of the ear. Previous studies have indicated that medical students’ and general practitioners’ otoscopic skills and confidence can be improved by providing extra education and training (Fisher and Pfleiderer, 1992a, 1992b). To date, there have been no studies regarding otoscopy skills among audiology students and the effectiveness of educational training activities in improving the otoscopy skills of audiology students. Kaf and colleagues (Kaf and Strong, 2011; Kaf et al, 2011) have studied the educational benefits of service learning for audiology students. They found that it increases their readiness to provide audiological evaluation to pediatric populations (Kaf and Strong, 2011) and to work with adults with dementia (Kaf et al, 2011). However, they did not formally assess students’ otoscopy skills while evaluating pediatric and dementia populations. The knowledge and competency of the examiner is known to correlate directly with the safety of the patient and the quality of care (British Society of Audiology, 2004). In order to address examiner competency, we developed and implemented an otoscopy training model designed to increase confidence, basic knowledge of ear anatomy, and diagnostic otoscopy skills. The current study was to determine the effectiveness of this otoscopy training model. METHODS Participants Thirty-two first-year audiology doctoral students from Missouri State University, enrolled in a pediatric audiology course, voluntarily signed an approved consent form before participation in this study. Students were randomly assigned to the control (n 5 16, 14 females) or experimental (n 5 16, 14 females) group. Supplementary Training The experimental group underwent training consisting of 90–120 min didactic sessions and three laboratory sessions including observation and clinical assignments. The training was administered over a period of 1 wk. The didactic sessions provided information about anatomy and physiology of the outer and middle ear and their common diseases, otoscopy and pneumatic
Otoscopy Competency and Training/Kaf et al
otoscopy, visual aids to supplement the anatomy of the external and middle ear and the mobility of the tympanic membrane (TM), and various still images of diagnostic signs of normal and abnormal ear diseases as seen via otoscopy. The three laboratory sessions (30 min each) focused on the use of handheld, pneumatic, and video otoscopy to develop competency in distinguishing between normal and abnormal TM mobility and identifying common ear problems. During laboratory sessions, students also were required to examine manikin ears with a handheld otoscope and to examine each other using both the handheld and video otoscope to practice the technique, determine normal versus abnormal otoscopy, and discuss signs of abnormality. The use of video otoscopy was primarily to engage the students in discussion of normal and abnormal ear canal and TM anatomy and not to compare handheld and video otoscopy. During the training that was provided separately to the experimental group, participants in the control group attended the same classes (Pediatric Audiology, Educational Audiology, and Amplification I) and had one clinical slot with their classmates in the experimental group; the control group did not receive the additional otoscopy training or practice time with the manikin. The control group did not receive the otoscopy and clinical training until after the first and second assessment was completed. After the otoscopy and clinical training was completed, the control group took the second posttest. The time span between the pretest and posttest for both groups varied from 7 to 10 days. Evaluation of Knowledge and Skills All students were assessed on three areas. These included a written evaluation (possible total score 5 54), otoscopy manikin clinical skills (possible total score 5 5), and self-perception of confidence and clinical skill (possible total rating 5 20).
Otoscopy Manikin Clinical Skill Evaluation Five manikin ears (model: Ear Examination Simulator, Kyoto Kagaku Co., Ltd.) were selected and presented to both groups in random order for the pretraining and 1 wk posttraining. The five ears included normal TM, serous otitis media (SOM), chronic suppurative otitis media (CSOM), cholesteatoma, and TM perforation. Each student was asked to examine the ears alone and to write a description of the otoscopic findings. Students were assessed on identification of the five ears. Responses were recorded and assessed as either correct (1 point/manikin ear) or incorrect (0 points/manikin ear); thus, each student’s score ranged between 0 and 5 points (0 5 incorrect identification of all five ears, 5 5 correct identification of all five ears). Scoring reliability was not run because the response has to match with the diagnostic labeling of each of the five manikin ears. Perceived Confidence and Clinical Skill Evaluation The report of the Audiology Education Summit (ASHA, 2005) stated that student acquisition of knowledge and skills in audiology can be measured via student selfevaluation opportunities. To assess their confidence and clinical skill level in otoscopy, students were asked to rate both their level of confidence and clinical otoscopy skill in four variables on a 1 to 5 scale (low to high, respectively). Student ratings were summed and analyzed for each of these four variables: (1) confidence level identifying the TM, (2) clinical skill in identifying the TM, (3) confidence level distinguishing normal from abnormal TM, and (4) clinical skill distinguishing normal from abnormal TM. The total possible score in each area ranged from 1 to 5, with a range of possible values from 4 to 20 across four areas.
Written Evaluation To assess the students’ knowledge and understanding of otologic anatomy and physiology and otoscopy technique, a 54-question short-answer examination was administered both before and after training, with a possible grade between 0 and 54 points. The main three categories of the 54-question examination included general knowledge of anatomy and physiology of the external ear, ear canal, and TM; otoscopy technique and use of pneumatic otoscopy to determine normal versus abnormal TM landmarks and their indications; and descriptions of abnormalities and diagnostic skills. Grading was conducted, using an answer key, by a graduate assistant, who was blind to condition and time. Test-retest scoring reliability by another grader was 98%. Results from these evaluations are presented as a ratio of total number correct to total number of questions in all groups.
RESULTS
A
ll scores were entered into a data file and analyzed via SPSS (PASW Version 18.0). Two separate 2 (groups as between-subjects variable: experimental, control) 3 2 (assessment time as within-subjects variable: pretraining, posttraining) analyses of variance (ANOVAs) were conducted for each of the dependent measures (written test and manikin ears) to evaluate the effects of the training on the students’ scores of the experimental and the control groups. To evaluate students’ self-perception of confidence, chi square (x2) analysis was conducted to compare training time by rating score for each group and between groups in each of the four areas. To illustrate changes in the control group across three times (pretest, posttest, posttest 2), a one-way ANOVA was conducted. Alpha level was set at .05.
861
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Written Evaluation Figure 1 illustrates the pre- and posttraining scores for the experimental group and the control group. The low pretest scores in both groups (,50%) suggest lack of knowledge mainly related to otoscopy technique and applications to determine normal versus abnormal TM landmarks and their indications, descriptions of abnormalities, diagnostic skills, and use of pneumatic otoscopy. Results of the 2 3 2 ANOVA indicated a significant main effect for group, F(1, 30) 5 7.485, p , .01, partial h2 5 .200, and a significant main effect for didactic training session, F(1,30) 5 57.344, p , .005, partial h2 5 .657, and a significant interaction between group and didactic training, F(1,30) 5 32.825, p , .005, partial h2 5 .522, as shown in Figure 1. A review of the group and training time means indicated that the experimental group had a significantly higher score (M 5 24.5) on the written exam than the control group (M 5 19.9), and the mean posttest score (M 5 27) was higher than the pretest score (M 5 17.5). Post hoc independent paired t-test indicated that there were no differences between the groups before training, t(30) 5 21.272, p 5 .213. After training, the score of the experimental group was significantly higher than the score of the control group, t(30) 5 5.575, p , .001. The scores of the experimental group increased significantly from pretraining (M 5 16.2) to posttraining (M 5 32.8), t(15) 5 28.976, p ,.005, whereas the scores of the control group did not, t(15) 5 21.317, p 5 .208. These findings and the specific means of the scores in each group in each condition are illustrated in Figure 1. To illustrate changes in the written scores of the control group across three times (pretest, posttest, posttest 2), a
one-way ANOVA was conducted. Results revealed a significant main effect of time, F(2,45) 5 8.971, p , .001. Tukey multiple comparisons showed that the significant difference is due to the posttest 2 score being higher than both the pretest score ( p , .001) and posttest score ( p , .01), but there was no significant difference between the pre- and posttest scores ( p 5 .609). Otoscopy Manikin Clinical Skill Evaluation Figure 2 displays the number of the total correct responses on otoscopic examination of the five manikin ears. As shown in the figure, both groups performed poorly prior to training in describing or diagnosing the five ears, with relatively better responses for the ear with normal finding and the ear with TM perforation. A 2 3 2 ANOVA was conducted to evaluate the effects of laboratory manikin training on clinical otoscopy skill of students of both groups. Results revealed a significant main effect for group, F(1, 30) 5 66.818, p , .005, partial h2 5 .690, and time of laboratory manikin ears, F(1,30) 5 120.714, p , .005, partial h2 5 .801, as well as a significant group 3 time interaction, F(1,30) 5 114.120, p , .005, partial h2 5 .781(Fig. 3) due to higher posttest scores (M 5 2.6) than pretest scores (M 5 .94). A paired samples t-test revealed significantly larger posttest scores than pretest scores for the experimental group, t(15) 5 215.181, p , .001. Results for the control group using one-way ANOVA (pretest, posttest, posttest 2) revealed a significant main effect of time, F(2,45) 5 93.373, p , .001. Tukey multiple comparisons showed that the significant difference is due to higher posttest 2 scores than both pretest scores and posttest scores ( p , .001), with no significant difference between pre- and posttest scores ( p 5 .808). Perceived Confidence and Clinical Skill Rating
Figure 1. Audiology students mean written scores pre- and posttraining in both groups. The figure shows that students in the experimental group had higher mean percentage score posttest than their pretest score. The figure also illustrates the significant interaction between group and didactic assessment time (p , .01). The experimental group that received the training had higher mean posttest written score than the control mean posttest score (p , .005). After receiving the training, the control group had higher posttest 2 score than their pretest score (p , .001).
862
As summarized in Table 1, not a single pre- versus posttraining comparison was statistically significant ( p . .05) for each group. When comparing posttraining rating of the experimental group to the posttraining rating of the control group, to see if at that point in time there was a difference, the only rating that was statistically significant, x2(12) 5 21.50, p , .05, was to question 3 (“Rate your confidence level distinguishing normal from abnormal TM”). These results are graphed in Figure 4, which shows that changes in perceived confidence of both control and experimental groups slightly increased posttraining in relation to identifying the TM and differentiating between abnormal and normal TM (questions 1 and 3). Similarly, students’ self-assessments of clinical skills (questions 2 and 4) showed small increases in both groups when the students were asked to identify the TM and to distinguish between normal and abnormal TM.
Otoscopy Competency and Training/Kaf et al
Figure 2. Practical otoscopic laboratory mean scores prior to and following otoscopy training using manikin ears in the experimental (gray bars) and control (black bars) groups, and following otoscopy training posttest 2 to the control group (white bars). Compared to pretraining scores, students in both the experimental (posttest) and control (posttest 2) correctly identified more ears with normal and abnormal status following training (p , .001). CSOM 5 chronic suppurative OM; SOM 5 serous OM.
Also, the sum of confidence rating and overall mean across questions for participants of both groups and training times showed that more students (63%) of the experimental group in the posttraining rated their confidence level at $70% than students in the pretraining (31%). Also, the percentage of students in the control group, whose confidence rating was $70%, was higher both for posttest (50%) and posttest 2 (50%) than pretraining (31%). The 70% rating was our cut-off indicator of an acceptable level of confidence level using otoscopy. Yet Figure 5 shows that students in the experimental group had slightly higher posttraining rating than their pretraining rating (p , .05). After receiving the training, the control group also had slight improvement in their posttest 2 score over their pretraining score (p , .05).
to anatomy, physiology, and otoscopy technique significantly improved, whereas posttest scores nearly doubled pretest scores. These scores demonstrate a correlation of basic anatomy and physiology and clinical knowledge for successful otoscopy in agreement with ASHA (1992) and the British Society of Audiology (2004) recommendations that audiologists should have knowledge and skills in knowing the structures and physiology of outer and middle ear to be able to perform otoscopy safely. In contrast to the enhanced performance of the experimental group posttraining, the control group did not improve in either written exam or otoscopy of manikin ears without the supplemental training. However, their performance on posttest 2 improved significantly after receiving the training. These findings indicate that any differences between the groups were dependent on which group received the didactic training. Approximately 52% (partial h2 5 .522) of the total variation of the written scores was attributed to the interaction of group and didactic session. Additionally, students of the experimental group showed significant improvement in clinical otoscopy skills once appropriately trained (posttest). Also, the control group demonstrated significant improvements on posttest 2 when given the same laboratory training as the experimental group. Additional studies have indicated that not only do students exposed to ear manikins notice increases in diagnostic ability but also increases in dexterity with regard to appropriate pressure placed on both patients and manikins (Morris et al, 2012). Furthermore, low pretest scores both on the written exam and of otoscopy skill demonstrated that otoscopy skill and training in audiology students may not be sufficient. In the present study, the overall comparison of the students’ self-evaluations of their clinical otoscopy rating of
DISCUSSION
O
toscopy is known to optimize assessment of the ear to determine the need for cerumen removal, improve detection of outer and middle ear disorders, provide appropriate audiological assessment, and provide a basis for medical referral (Jones and Kaleida, 2003). Otoscopy skill is essential for clinical audiologists and for health-care providers involved in the clinical diagnosis of middle and external ear pathology. Therefore, it is critical to improve otoscopy competency by training audiology graduate students to examine ears using a variety of techniques to allow them to recognize a wide range of ear conditions. Results of this study suggest that incorporating an otoscopy curriculum is necessary. Following otoscopy training, students’ performance on questions related
Figure 3. Mean laboratory manikin percent scores pre- and posttraining in both groups. Although both groups did not differ in their scores pretraining, the experimental group had higher posttraining (posttest) scores than the control group (p , .005), due to the effect of laboratory training. The control group did not differ in their scores pre- and posttraining (p 5 .808). However, they had higher posttest 2 scores than pretest scores after receiving the training. The figure also illustrates the significant interaction between group and assessment time (p , .001).
863
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Table 1. Summary of Confidence Rating Analysis Using Chi-Square (x2) Test for Each of the Four Questions within Each Group and between Groups
Q1 Q2 Q3 Q4
Control
Experimental Pre vs. Post
Pre vs. Post
Post vs. Post 2
Control vs. Experimental Post- vs. Posttraining
x2(4) 5 5.37, p 5 .251 x2(6) 5 6.96, p 5 .324 x2(9) 5 12.44, p 5 .189 x2(4) 5 4.59, p 5 .331
x2(4) 5 3.55, p 5 .469 x2(8) 5 6.66, p 5 .573 x2(12) 5 20.13, p 5 .065 x2(9) 5 7.23, p 5 .612
x2(4) 5 5.20, p 5 .267 x2(4) 5 8.44, p 5 .077 x2(6) 5 7.11, p 5 .311 x2(6) 5 1.60, p 5 .953
x2(4) 5 2.77, p 5 .596 x2(12) 5 9.63, p 5 .648 x2(12) 5 21.50, p 5 .044* x2(6) 5 2.36, p 5 .883
Note: p 5 alpha significant level. *p is significant at alpha level ,.05
$70% showed more students felt confident posttraining (63% experimental and 50% control group) compared to 31% pretraining. This result suggests that otoscopy training increases students’ confidence in otoscopy. However, this significant increase of confidence of audiology students was far from that reported by 99% of pediatricians, who felt comfortable using otoscopy to distinguish between acute otitis media (AOM) and otitis media with effusion (OME) (Pichichero and Poole, 2001), meeting the American Academy of Pediatrics (AAP) guideline (AAP, 2004). The higher confidence rating in pediatricians is mainly due to their extra years of experience and exposure to otoscopy as well as the expectation to diagnose and treat medical conditions of the ear compared to audiology students who participated in this study. Furthermore, given the scope of practice guidelines as outlined above, audiology students would not be expected to distinguish these pathologies in professional practice. However, these data do suggest that implantation of this training module is useful for audiology students to improve interpretation of tympanometric findings (Nozza et al, 1994; Helenius et al, 2012) and may be beneficial for other medical disciplines, including those required to identify specific pathological etiologies and treat. Although we used the 70% cutoff rating as an indicator of an acceptable level of confidence using otoscopy, there is no available guideline for what constitutes an acceptable level for accurately distinguishing between structures or middle ear diseases and, for example, between acute OM and OME using otoscopy. In contrast to the performance on written examination and an ear manikin, students’ confidence ratings were not significantly different between control and experimental groups, which may be explained by their high pretest rating (64% for experimental and 62% for control). This high level of confidence before the training may be due to the students’ perceptions that they knew how to perform otoscopy and interpret its finding. However, this high pretest confidence level was not comparable to their low pretest scores on the written
864
examination and the manikin ears. These findings suggest that the perceived confidence does not reflect competency and can be misleading, evidenced by those students in the control group who performed poorly on posttest scores on manikin ears yet still reported increases in confidence values. This data suggests that exposure may be the most significant influence when evaluating confidence in both identification and distinguishing between normal and abnormal TM. Other studies evaluating otoscopy training demonstrated increased confidence in students with repeated exposure
Figure 4. Student confidence percentage rating per question across participants in both the experimental (circle symbols) and control (triangle symbols) groups before and following otoscopy training, and after providing otoscopy training to the control group (posttest 2: crossing symbols). Both groups indicated relatively high confidence pretraining for all questions. The same pattern is also shown posttraining, but with slight improvement of the confidence scores for students following training (experimental posttest and control posttest 2) than students who did not have training (control posttest). Q1: Rate your confidence level identifying the tympanic membrane (TM). Q2: Rate your clinical skill in identifying the TM. Q3: Rate your confidence level distinguishing normal from abnormal TM. Q4: Rate your clinical skill distinguishing normal from abnormal TM.
Otoscopy Competency and Training/Kaf et al
Figure 5. Student confidence of clinical skill sum rating scores across questions (1–5 score, low to high; with a range of possible values from 4 to 20 across four areas) pre- and posttraining in both groups. The figure shows that students in the experimental group had slightly higher posttraining rating than their pretraining rating (p . .05). After receiving the training, the control group also had slight improvement in their posttest 2 score over their pretraining score (p . .05).
to otoscopy, regardless of additional training (Fisher and Pfleiderer, 1992a). According to the above assumption that repeated exposure should improve confidence level without training, we would expect that second-year audiology students should perform better than first-year audiology students after training simply because they had an extra year of school, despite the lack of additional otoscopy training. To study this assumption, a post hoc independent samples t-test analysis was conducted to compare pretest scores of the second-year students (n 5 8) and the posttest scores of the first-year students in the experimental group (n 5 8) who completed the training. This design is an indicator of the knowledge that the second-year students already have compared to first-year students posttraining. As shown in Figure 6, results interestingly revealed that mean performance of students in the first year posttraining was much better than the mean performance of the second-year students on both the written exam, t(14) 5 26.412, p , .001, and ear manikin, t(14) 5 28.881, p , .001, with no difference in their perception of confidence rating, t(14) 5 2.458, p 5 .654. These findings suggest that additional didactic and supervised clinical training should result in improved learning gain and otoscopy competency more than an extra year of unsupervised exposure does without close otoscopy supervision. Although Villasen˜or et al (1998) has demonstrated that didactic training in otoscopy without the use of manikins improved the otoscopy skill of students, Fisher and Pfleiderer (1992b) and results of this study evidenced by data above underline the importance of not only manikin otoscopy training but also baseline knowledge of anatomy and technique as it relates to otoscopy, both of which will likely improve clinician confidence in otoscopy. Regardless of the approach, direct supervision is considered the most important aspect to
improve students’ otoscopy skills. According to Cox (1982), direct supervision from specifically trained and experienced professionals should provide students with immediate feedback. Also, it is suggested that supervised otoscopy of at least 12 cases would improve students’ otoscopy competency to a satisfactory level (Cox, 1982; Fisher and Pfleiderer, 1992a). This has been supported by Silva and Hotaling (1997), who reported increased sensitivity (.80%) and specificity (.70%) of otolaryngology residents’ pneumatic otoscopy in diagnosing cases with OME following 4 mo of didactic and clinical training as well as examining 275 ears via pneumatic otoscopy. For those graduates and health-care professionals who are lacking otoscopy skills, Fisher and Pfleiderer (1992b) recommended taking a continuing education course or vocational training on otoscopy to improve their skills. In conclusion, results of this study demonstrated that additional training in otoscopy is a valuable tool in the education of future audiologists. Supplementing educational courses by focusing on anatomy and physiology of the outer and middle ear, otoscopy techniques, manikin training, and repetition is essential. Despite the limited clinical experience with review of still images and particular abnormal cases of ear diseases during training, the current study and others have suggested that improvements in clinical otoscopy lead to decreases in morbidity of otological pathologies (Silva and Hotaling, 1997). To gain knowledge and clinical skills for developing competency in otoscopy, communication sciences and disorders programs need to provide additional educational and clinical training to their students. Our recommendation is to implement otoscopy education and training early in the audiology curriculum starting in a course such as diagnostic audiology and advancing student otoscopy clinical skill in courses such as pediatric audiology and medical audiology to gain more
Figure 6. Subgroup comparison of pretest scores of second-year students in the control group to posttest scores of first-year students in the experimental group. Despite the extra school year of the second-year students, first-year students showed better knowledge on the written exam and better otoscopy skill using ear manikin posttraining (**p , .001). Both groups did not differ in their confidence rating.
865
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
experience in children and in cases with medical disorders of the ear. The present study suggests that training of those who will be trusted with the responsibility of assessing and managing otologic complaints must be provided with appropriate didactic and clinical training in otoscopy. In addition, a training module that consists of providing knowledge, skill, and techniques including familiarity with the proper use of the otoscope and interpretation of findings is required. It is also recommended that repeated supervised exposure and experience with patients with a wide variety of ear diseases should lead to a higher level of accurately identifying ear conditions that demand referral. Finally, why should competency in otoscopy be any different than assessing students’ competency in conducting puretone and speech audiometry with and without masking, performing and interpreting tympanometry and acoustic reflex testing, making ear mold impressions, and other audiological site-of-lesion testing? This study provides both a training module and an evaluation plan to assess competency in otoscopy of audiology students. Acknowledgments. We would like to thank all audiology students who participated in the study and Dr. Julie Masterson for providing valuable input on the statistical analysis and editorial comments on this manuscript. Also, we want to thank Lindsay Garrison and Uzma Wilson for their assistance with collecting and coding data.
Council for Clinical Certification in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association (CFCC). (2012) 2012 Standards and Implementation Procedures for the Certificate of Clinical Competence in Audiology. www. asha.org/Certification/2012-Audiology-Certification-Standards/ (accessed December 27, 2012). Cox KR. (1982) Teaching physical examination skills. In: Cox KR, Ewan CE, eds. Med Teach. Edinburgh: Churchill Livingstone, 110–113. Fisher EW, Pfleiderer AG. (1992a) Is undergraduate otoscopy teaching adequate? An audit of clinical teaching. J R Soc Med 85:23–25. Fisher EW, Pfleiderer AG. (1992b) Assessment of the otoscopic skills of general practitioners and medical students: is there room for improvement? Br J Gen Pract 42:65–67. Helenius KK, Laine MK, Ta¨ htinen PA, Elina Lahti E, Ruohola AM. (2012) Tympanometry in discrimination of otoscopic diagnoses in young ambulatory children. Pediatr Infect Dis J 31: 1003–1006. Jones WS, Kaleida PH. (2003) How helpful is pneumatic otoscopy in improving diagnostic accuracy? Pediatrics 112(3, Pt. 1): 510–513. Kaf WA, Barboa L, Fisher B, Snavely L. (2011) Effect of interdisciplinary service learning experience for audiology and speechlanguage pathology students working with adults with dementia. Am J Audiol 20:S241–S249. Kaf WA, Strong E. (2011) The promise of service learning in a pediatric audiology course on clinical training with the pediatric population. Am J Audiol 20:S220–S232.
REFERENCES
Morris E, Kesser BW, Peirce-Cottler S, Keeley M. (2012) Development and validation of a novel ear simulator to teach pneumatic otoscopy. Simul Healthc 7(1):22–26.
American Academy of Audiology. (2004) Scope of Practice. www. audiology.org/resources/documentlibrary/Pages/ScopeofPractice. aspx (accessed October 4, 2012).
Nozza RJ, Bluestone CD, Kardatzke D, Bachman R. (1994) Identification of middle ear effusion by aural acoustic admittance and otoscopy. Ear Hear 15(4):310–323.
American Academy of Pediatrics (AAP). (2004) Diagnosis and management of acute otitis media. Pediatrics 113(5):1451–1465.
Pichichero ME, Poole MD. (2001) Assessing diagnostic accuracy and tympanocentesis skills in the management of otitis media. Arch Pediatr Adolesc Med 155:1137–1142.
American Speech-Language-Hearing Association (ASHA). (1992) External Auditory Canal Examination and Cerumen Management [Guidelines, Knowledge and Skills, Position Statement]. www. asha.org/policy/GLKSPS1992-00034.htm (accessed August 20, 2012).
Scudder SG, Culbertson DS, Waldron CM, Stewart J. (2003) Predictive validity and reliability of adult hearing screening techniques. J Am Acad Audiol 14(1):9–19.
American Speech-Language-Hearing Association (ASHA). (1997) Guidelines for Audiologic Screening. Rockville, MD: American Speech-Language-Hearing Association.
Silva AB, Hotaling AJ. (1997) A protocol for otolaryngology-head and neck resident training in pneumatic otoscopy. Int J Pediatr Otorhinolaryngol 40(2–3):125–131. Sullivan R. (1993) How video otoscopy benefits hearing health care practice. Hear Instrum 44(4):15–18.
American Speech-Language-Hearing Association (ASHA). (2005) Audiology Education Summit: A Collaborative Approach [Conference report]. www.asha.org/uploadedFiles/events/audsummit2005. pdf#search5%22Audiology%22 (accessed July 13, 2012).
Sullivan R. (1997) Video otoscopy in audiologic practice. J Am Acad Audiol 8(6):447–467.
British Society of Audiology. (2004) Guidelines on Minimum Training Standards for Otoscopy and Impression Taking. www.thebsa.org.uk/ docs/Guidelines/otoscopytrainingstandardsfinalversionoct2004. pdf (accessed September 1, 2012).
Villasen˜or A, Arriaga MA, Eavey RD, Santos JI, Chissone E. (1998) Educational outcomes of an otitis media workshop for primary care providers in Latin America. Otolaryngol Head Neck Surg 118:394–396.
866
Copyright of Journal of the American Academy of Audiology is the property of American Academy of Audiology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Optimizing Otoscopy Competency in Audiology Students through Supplementary Otoscopy Training DOI: 10.3766/jaaa.24.9.9 Wafaa A. Kaf* Caleb G. Masterson† Nancy Dion‡ Susan L. Berg§ Mohamed K. Abdelhakiem**
Abstract Background: Scope of practice in audiology encompasses proficiency in visual inspection of ear canal and tympanic membrane (TM) as well as otoscopy interpretation skills to determine normal versus abnormal conditions of outer and middle ear. Audiology students can develop skills in otoscopy through education and supervised training. Studies have shown that additional otoscopy training increased skills in medical students and general practitioners. However, educational and supervised practices targeting otoscopy competency during audiology graduate coursework are lacking. Also, no studies have attempted to determine otoscopy skills among audiology students. Purpose: To determine the effectiveness of the otoscopy training model on clinical competency and confidence level of audiology students in performing and interpreting otoscopy. Research Design: A combination of experimental treatment design with random assignment of treatment and control groups and delayed treatment for control group. Study Sample: Thirty-two first- and second-year audiology graduate students who were enrolled in a pediatric audiology class participated in this study. Students were randomly assigned to the control (n 5 16, 14 females) or experimental (n 5 16, 14 females) group. Intervention: Participants in the experimental group received supplementary otoscopy training including didactic otoscopy lectures as well as clinical training using manikin ears. The control group received the same pretest and posttest and then completed a third assessment (posttest 2) after receiving the same training. Data Collection and Analysis: An evaluation of knowledge and skills regarding otoscopy between groups and time was conducted at three times: (a) pretraining, (b) upon completion of training for the experimental group, (c) upon completion of training by the control group. The evaluation consisted of a written exam, a clinical exam, and a self-perception rating of confidence. Both written exam scores and clinical exam scores (otoscopy manikin) were analyzed via two-way analyses of variance (ANOVAs), whereas chi-square (x2) statistic was conducted to evaluate the effects of training on the confidence level of students of both groups. Results: Experimental and control groups demonstrated significant increased overall competency in otoscopy following the otoscopy training model with didactic and laboratory components. Posttest confidence ratings showed increases in all groups, and there were no significant differences between groups.
*Communication Sciences and Disorders Department, Missouri State University, Springfield, MO; †Osteopathic Medicine, Des Moines University, Des Moines, IA; ‡Physician Assistant Department, Missouri State University, Springfield, MO; §Nursing Department, Missouri State University, Springfield, MO; **Biomedical Sciences Department, Missouri State University, Springfield, MO Wafaa Kaf, Communication Sciences and Disorders Department, Missouri State University, 901 South National Avenue, Springfield, MO 65897; Phone: 417-836-4456; Fax: 417-836-4242; E-mail: [email protected] Presented at the American Speech-Language-Hearing Association (ASHA) Convention, November 17–21, 2010, Philadelphia, PA. This study was partially supported by the Citizenship and Service-Learning Office at Missouri State University.
859
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Conclusions: The need for supplementary otoscopy training was warranted by low knowledge and clinical competency in otoscopy skills of audiology students as measured by pretest mean scores. After completing the training, both experimental and control groups showed significant improvement in knowledge and competency. Results also suggest that perceived confidence ratings may be misleading in determining students’ clinical otoscopy skills. Key Words: Audiology students, competency, knowledge, otoscopy, skills, training Abbreviations: ASHA 5 American Speech-Language-Hearing Association; CSOM 5 chronic suppurative otitis media; OM 5 otitis media; SOM 5 serous otitis media; TM 5 tympanic membrane
O
toscopy is the standard for conducting an external and middle ear examination. Audiologists, pediatricians, ENT surgeons, and other healthcare providers perform otoscopy on a regular basis. Otoscopy helps an examiner to determine the presence of obstructing material in the ear canal, signs of congenital anomalies, or active infections of the outer or middle ear that may prohibit completion of pure-tone audiometry, tympanometry, or other audiological procedures (American Speech-Language-Hearing Association [ASHA], 1992). In addition, careful otoscopic examination is important to visualize pathologies that may not be detected with tympanometry or other audiological testing, thus allowing for appropriate referral. Furthermore, the use of video otoscopy provides additional valuable documentation for monitoring of medical conditions or referrals (Sullivan, 1993, 1997). The scope of otoscopy practice for audiologists includes ruling out contraindications for the use of earphone or probe and knowledge of anatomy and recognition of certain pathology of the outer ear and middle ear. These skills allow for appropriate referral for treatment of common outer and middle ear diseases (ASHA, 1992, 1997; American Academy of Audiology, 2004). According to standard IV-C4 of the Standards for the Certificate of Clinical Competence in Audiology, students applying for certification “must have the knowledge and skills necessary in performing otoscopy” (Council for Clinical Certification in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association [CFCC], 2012), indicating that audiologists should have the necessary training in otoscopy and knowledge of normal and abnormal conditions of the ear (ASHA, 1992). However, an online search for posted audiology curriculum from a variety of audiology programs including Missouri State University revealed that their curriculum and course descriptions do not specify any otoscopy training. Therefore, educational practices targeting otoscopy competency for audiology graduates may be inadequate due to lack of appropriate audiology course work and curricula. Scudder et al (2003) have reported high interrater reliability of otoscopy among graduate audiology and speech-language pathology students despite their variable level educational background and otoscopic training. However, this finding was related to their rating of whether earwax was excessive or impacted, not to
860
determine normal versus abnormal conditions of the ear. Previous studies have indicated that medical students’ and general practitioners’ otoscopic skills and confidence can be improved by providing extra education and training (Fisher and Pfleiderer, 1992a, 1992b). To date, there have been no studies regarding otoscopy skills among audiology students and the effectiveness of educational training activities in improving the otoscopy skills of audiology students. Kaf and colleagues (Kaf and Strong, 2011; Kaf et al, 2011) have studied the educational benefits of service learning for audiology students. They found that it increases their readiness to provide audiological evaluation to pediatric populations (Kaf and Strong, 2011) and to work with adults with dementia (Kaf et al, 2011). However, they did not formally assess students’ otoscopy skills while evaluating pediatric and dementia populations. The knowledge and competency of the examiner is known to correlate directly with the safety of the patient and the quality of care (British Society of Audiology, 2004). In order to address examiner competency, we developed and implemented an otoscopy training model designed to increase confidence, basic knowledge of ear anatomy, and diagnostic otoscopy skills. The current study was to determine the effectiveness of this otoscopy training model. METHODS Participants Thirty-two first-year audiology doctoral students from Missouri State University, enrolled in a pediatric audiology course, voluntarily signed an approved consent form before participation in this study. Students were randomly assigned to the control (n 5 16, 14 females) or experimental (n 5 16, 14 females) group. Supplementary Training The experimental group underwent training consisting of 90–120 min didactic sessions and three laboratory sessions including observation and clinical assignments. The training was administered over a period of 1 wk. The didactic sessions provided information about anatomy and physiology of the outer and middle ear and their common diseases, otoscopy and pneumatic
Otoscopy Competency and Training/Kaf et al
otoscopy, visual aids to supplement the anatomy of the external and middle ear and the mobility of the tympanic membrane (TM), and various still images of diagnostic signs of normal and abnormal ear diseases as seen via otoscopy. The three laboratory sessions (30 min each) focused on the use of handheld, pneumatic, and video otoscopy to develop competency in distinguishing between normal and abnormal TM mobility and identifying common ear problems. During laboratory sessions, students also were required to examine manikin ears with a handheld otoscope and to examine each other using both the handheld and video otoscope to practice the technique, determine normal versus abnormal otoscopy, and discuss signs of abnormality. The use of video otoscopy was primarily to engage the students in discussion of normal and abnormal ear canal and TM anatomy and not to compare handheld and video otoscopy. During the training that was provided separately to the experimental group, participants in the control group attended the same classes (Pediatric Audiology, Educational Audiology, and Amplification I) and had one clinical slot with their classmates in the experimental group; the control group did not receive the additional otoscopy training or practice time with the manikin. The control group did not receive the otoscopy and clinical training until after the first and second assessment was completed. After the otoscopy and clinical training was completed, the control group took the second posttest. The time span between the pretest and posttest for both groups varied from 7 to 10 days. Evaluation of Knowledge and Skills All students were assessed on three areas. These included a written evaluation (possible total score 5 54), otoscopy manikin clinical skills (possible total score 5 5), and self-perception of confidence and clinical skill (possible total rating 5 20).
Otoscopy Manikin Clinical Skill Evaluation Five manikin ears (model: Ear Examination Simulator, Kyoto Kagaku Co., Ltd.) were selected and presented to both groups in random order for the pretraining and 1 wk posttraining. The five ears included normal TM, serous otitis media (SOM), chronic suppurative otitis media (CSOM), cholesteatoma, and TM perforation. Each student was asked to examine the ears alone and to write a description of the otoscopic findings. Students were assessed on identification of the five ears. Responses were recorded and assessed as either correct (1 point/manikin ear) or incorrect (0 points/manikin ear); thus, each student’s score ranged between 0 and 5 points (0 5 incorrect identification of all five ears, 5 5 correct identification of all five ears). Scoring reliability was not run because the response has to match with the diagnostic labeling of each of the five manikin ears. Perceived Confidence and Clinical Skill Evaluation The report of the Audiology Education Summit (ASHA, 2005) stated that student acquisition of knowledge and skills in audiology can be measured via student selfevaluation opportunities. To assess their confidence and clinical skill level in otoscopy, students were asked to rate both their level of confidence and clinical otoscopy skill in four variables on a 1 to 5 scale (low to high, respectively). Student ratings were summed and analyzed for each of these four variables: (1) confidence level identifying the TM, (2) clinical skill in identifying the TM, (3) confidence level distinguishing normal from abnormal TM, and (4) clinical skill distinguishing normal from abnormal TM. The total possible score in each area ranged from 1 to 5, with a range of possible values from 4 to 20 across four areas.
Written Evaluation To assess the students’ knowledge and understanding of otologic anatomy and physiology and otoscopy technique, a 54-question short-answer examination was administered both before and after training, with a possible grade between 0 and 54 points. The main three categories of the 54-question examination included general knowledge of anatomy and physiology of the external ear, ear canal, and TM; otoscopy technique and use of pneumatic otoscopy to determine normal versus abnormal TM landmarks and their indications; and descriptions of abnormalities and diagnostic skills. Grading was conducted, using an answer key, by a graduate assistant, who was blind to condition and time. Test-retest scoring reliability by another grader was 98%. Results from these evaluations are presented as a ratio of total number correct to total number of questions in all groups.
RESULTS
A
ll scores were entered into a data file and analyzed via SPSS (PASW Version 18.0). Two separate 2 (groups as between-subjects variable: experimental, control) 3 2 (assessment time as within-subjects variable: pretraining, posttraining) analyses of variance (ANOVAs) were conducted for each of the dependent measures (written test and manikin ears) to evaluate the effects of the training on the students’ scores of the experimental and the control groups. To evaluate students’ self-perception of confidence, chi square (x2) analysis was conducted to compare training time by rating score for each group and between groups in each of the four areas. To illustrate changes in the control group across three times (pretest, posttest, posttest 2), a one-way ANOVA was conducted. Alpha level was set at .05.
861
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Written Evaluation Figure 1 illustrates the pre- and posttraining scores for the experimental group and the control group. The low pretest scores in both groups (,50%) suggest lack of knowledge mainly related to otoscopy technique and applications to determine normal versus abnormal TM landmarks and their indications, descriptions of abnormalities, diagnostic skills, and use of pneumatic otoscopy. Results of the 2 3 2 ANOVA indicated a significant main effect for group, F(1, 30) 5 7.485, p , .01, partial h2 5 .200, and a significant main effect for didactic training session, F(1,30) 5 57.344, p , .005, partial h2 5 .657, and a significant interaction between group and didactic training, F(1,30) 5 32.825, p , .005, partial h2 5 .522, as shown in Figure 1. A review of the group and training time means indicated that the experimental group had a significantly higher score (M 5 24.5) on the written exam than the control group (M 5 19.9), and the mean posttest score (M 5 27) was higher than the pretest score (M 5 17.5). Post hoc independent paired t-test indicated that there were no differences between the groups before training, t(30) 5 21.272, p 5 .213. After training, the score of the experimental group was significantly higher than the score of the control group, t(30) 5 5.575, p , .001. The scores of the experimental group increased significantly from pretraining (M 5 16.2) to posttraining (M 5 32.8), t(15) 5 28.976, p ,.005, whereas the scores of the control group did not, t(15) 5 21.317, p 5 .208. These findings and the specific means of the scores in each group in each condition are illustrated in Figure 1. To illustrate changes in the written scores of the control group across three times (pretest, posttest, posttest 2), a
one-way ANOVA was conducted. Results revealed a significant main effect of time, F(2,45) 5 8.971, p , .001. Tukey multiple comparisons showed that the significant difference is due to the posttest 2 score being higher than both the pretest score ( p , .001) and posttest score ( p , .01), but there was no significant difference between the pre- and posttest scores ( p 5 .609). Otoscopy Manikin Clinical Skill Evaluation Figure 2 displays the number of the total correct responses on otoscopic examination of the five manikin ears. As shown in the figure, both groups performed poorly prior to training in describing or diagnosing the five ears, with relatively better responses for the ear with normal finding and the ear with TM perforation. A 2 3 2 ANOVA was conducted to evaluate the effects of laboratory manikin training on clinical otoscopy skill of students of both groups. Results revealed a significant main effect for group, F(1, 30) 5 66.818, p , .005, partial h2 5 .690, and time of laboratory manikin ears, F(1,30) 5 120.714, p , .005, partial h2 5 .801, as well as a significant group 3 time interaction, F(1,30) 5 114.120, p , .005, partial h2 5 .781(Fig. 3) due to higher posttest scores (M 5 2.6) than pretest scores (M 5 .94). A paired samples t-test revealed significantly larger posttest scores than pretest scores for the experimental group, t(15) 5 215.181, p , .001. Results for the control group using one-way ANOVA (pretest, posttest, posttest 2) revealed a significant main effect of time, F(2,45) 5 93.373, p , .001. Tukey multiple comparisons showed that the significant difference is due to higher posttest 2 scores than both pretest scores and posttest scores ( p , .001), with no significant difference between pre- and posttest scores ( p 5 .808). Perceived Confidence and Clinical Skill Rating
Figure 1. Audiology students mean written scores pre- and posttraining in both groups. The figure shows that students in the experimental group had higher mean percentage score posttest than their pretest score. The figure also illustrates the significant interaction between group and didactic assessment time (p , .01). The experimental group that received the training had higher mean posttest written score than the control mean posttest score (p , .005). After receiving the training, the control group had higher posttest 2 score than their pretest score (p , .001).
862
As summarized in Table 1, not a single pre- versus posttraining comparison was statistically significant ( p . .05) for each group. When comparing posttraining rating of the experimental group to the posttraining rating of the control group, to see if at that point in time there was a difference, the only rating that was statistically significant, x2(12) 5 21.50, p , .05, was to question 3 (“Rate your confidence level distinguishing normal from abnormal TM”). These results are graphed in Figure 4, which shows that changes in perceived confidence of both control and experimental groups slightly increased posttraining in relation to identifying the TM and differentiating between abnormal and normal TM (questions 1 and 3). Similarly, students’ self-assessments of clinical skills (questions 2 and 4) showed small increases in both groups when the students were asked to identify the TM and to distinguish between normal and abnormal TM.
Otoscopy Competency and Training/Kaf et al
Figure 2. Practical otoscopic laboratory mean scores prior to and following otoscopy training using manikin ears in the experimental (gray bars) and control (black bars) groups, and following otoscopy training posttest 2 to the control group (white bars). Compared to pretraining scores, students in both the experimental (posttest) and control (posttest 2) correctly identified more ears with normal and abnormal status following training (p , .001). CSOM 5 chronic suppurative OM; SOM 5 serous OM.
Also, the sum of confidence rating and overall mean across questions for participants of both groups and training times showed that more students (63%) of the experimental group in the posttraining rated their confidence level at $70% than students in the pretraining (31%). Also, the percentage of students in the control group, whose confidence rating was $70%, was higher both for posttest (50%) and posttest 2 (50%) than pretraining (31%). The 70% rating was our cut-off indicator of an acceptable level of confidence level using otoscopy. Yet Figure 5 shows that students in the experimental group had slightly higher posttraining rating than their pretraining rating (p , .05). After receiving the training, the control group also had slight improvement in their posttest 2 score over their pretraining score (p , .05).
to anatomy, physiology, and otoscopy technique significantly improved, whereas posttest scores nearly doubled pretest scores. These scores demonstrate a correlation of basic anatomy and physiology and clinical knowledge for successful otoscopy in agreement with ASHA (1992) and the British Society of Audiology (2004) recommendations that audiologists should have knowledge and skills in knowing the structures and physiology of outer and middle ear to be able to perform otoscopy safely. In contrast to the enhanced performance of the experimental group posttraining, the control group did not improve in either written exam or otoscopy of manikin ears without the supplemental training. However, their performance on posttest 2 improved significantly after receiving the training. These findings indicate that any differences between the groups were dependent on which group received the didactic training. Approximately 52% (partial h2 5 .522) of the total variation of the written scores was attributed to the interaction of group and didactic session. Additionally, students of the experimental group showed significant improvement in clinical otoscopy skills once appropriately trained (posttest). Also, the control group demonstrated significant improvements on posttest 2 when given the same laboratory training as the experimental group. Additional studies have indicated that not only do students exposed to ear manikins notice increases in diagnostic ability but also increases in dexterity with regard to appropriate pressure placed on both patients and manikins (Morris et al, 2012). Furthermore, low pretest scores both on the written exam and of otoscopy skill demonstrated that otoscopy skill and training in audiology students may not be sufficient. In the present study, the overall comparison of the students’ self-evaluations of their clinical otoscopy rating of
DISCUSSION
O
toscopy is known to optimize assessment of the ear to determine the need for cerumen removal, improve detection of outer and middle ear disorders, provide appropriate audiological assessment, and provide a basis for medical referral (Jones and Kaleida, 2003). Otoscopy skill is essential for clinical audiologists and for health-care providers involved in the clinical diagnosis of middle and external ear pathology. Therefore, it is critical to improve otoscopy competency by training audiology graduate students to examine ears using a variety of techniques to allow them to recognize a wide range of ear conditions. Results of this study suggest that incorporating an otoscopy curriculum is necessary. Following otoscopy training, students’ performance on questions related
Figure 3. Mean laboratory manikin percent scores pre- and posttraining in both groups. Although both groups did not differ in their scores pretraining, the experimental group had higher posttraining (posttest) scores than the control group (p , .005), due to the effect of laboratory training. The control group did not differ in their scores pre- and posttraining (p 5 .808). However, they had higher posttest 2 scores than pretest scores after receiving the training. The figure also illustrates the significant interaction between group and assessment time (p , .001).
863
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
Table 1. Summary of Confidence Rating Analysis Using Chi-Square (x2) Test for Each of the Four Questions within Each Group and between Groups
Q1 Q2 Q3 Q4
Control
Experimental Pre vs. Post
Pre vs. Post
Post vs. Post 2
Control vs. Experimental Post- vs. Posttraining
x2(4) 5 5.37, p 5 .251 x2(6) 5 6.96, p 5 .324 x2(9) 5 12.44, p 5 .189 x2(4) 5 4.59, p 5 .331
x2(4) 5 3.55, p 5 .469 x2(8) 5 6.66, p 5 .573 x2(12) 5 20.13, p 5 .065 x2(9) 5 7.23, p 5 .612
x2(4) 5 5.20, p 5 .267 x2(4) 5 8.44, p 5 .077 x2(6) 5 7.11, p 5 .311 x2(6) 5 1.60, p 5 .953
x2(4) 5 2.77, p 5 .596 x2(12) 5 9.63, p 5 .648 x2(12) 5 21.50, p 5 .044* x2(6) 5 2.36, p 5 .883
Note: p 5 alpha significant level. *p is significant at alpha level ,.05
$70% showed more students felt confident posttraining (63% experimental and 50% control group) compared to 31% pretraining. This result suggests that otoscopy training increases students’ confidence in otoscopy. However, this significant increase of confidence of audiology students was far from that reported by 99% of pediatricians, who felt comfortable using otoscopy to distinguish between acute otitis media (AOM) and otitis media with effusion (OME) (Pichichero and Poole, 2001), meeting the American Academy of Pediatrics (AAP) guideline (AAP, 2004). The higher confidence rating in pediatricians is mainly due to their extra years of experience and exposure to otoscopy as well as the expectation to diagnose and treat medical conditions of the ear compared to audiology students who participated in this study. Furthermore, given the scope of practice guidelines as outlined above, audiology students would not be expected to distinguish these pathologies in professional practice. However, these data do suggest that implantation of this training module is useful for audiology students to improve interpretation of tympanometric findings (Nozza et al, 1994; Helenius et al, 2012) and may be beneficial for other medical disciplines, including those required to identify specific pathological etiologies and treat. Although we used the 70% cutoff rating as an indicator of an acceptable level of confidence using otoscopy, there is no available guideline for what constitutes an acceptable level for accurately distinguishing between structures or middle ear diseases and, for example, between acute OM and OME using otoscopy. In contrast to the performance on written examination and an ear manikin, students’ confidence ratings were not significantly different between control and experimental groups, which may be explained by their high pretest rating (64% for experimental and 62% for control). This high level of confidence before the training may be due to the students’ perceptions that they knew how to perform otoscopy and interpret its finding. However, this high pretest confidence level was not comparable to their low pretest scores on the written
864
examination and the manikin ears. These findings suggest that the perceived confidence does not reflect competency and can be misleading, evidenced by those students in the control group who performed poorly on posttest scores on manikin ears yet still reported increases in confidence values. This data suggests that exposure may be the most significant influence when evaluating confidence in both identification and distinguishing between normal and abnormal TM. Other studies evaluating otoscopy training demonstrated increased confidence in students with repeated exposure
Figure 4. Student confidence percentage rating per question across participants in both the experimental (circle symbols) and control (triangle symbols) groups before and following otoscopy training, and after providing otoscopy training to the control group (posttest 2: crossing symbols). Both groups indicated relatively high confidence pretraining for all questions. The same pattern is also shown posttraining, but with slight improvement of the confidence scores for students following training (experimental posttest and control posttest 2) than students who did not have training (control posttest). Q1: Rate your confidence level identifying the tympanic membrane (TM). Q2: Rate your clinical skill in identifying the TM. Q3: Rate your confidence level distinguishing normal from abnormal TM. Q4: Rate your clinical skill distinguishing normal from abnormal TM.
Otoscopy Competency and Training/Kaf et al
Figure 5. Student confidence of clinical skill sum rating scores across questions (1–5 score, low to high; with a range of possible values from 4 to 20 across four areas) pre- and posttraining in both groups. The figure shows that students in the experimental group had slightly higher posttraining rating than their pretraining rating (p . .05). After receiving the training, the control group also had slight improvement in their posttest 2 score over their pretraining score (p . .05).
to otoscopy, regardless of additional training (Fisher and Pfleiderer, 1992a). According to the above assumption that repeated exposure should improve confidence level without training, we would expect that second-year audiology students should perform better than first-year audiology students after training simply because they had an extra year of school, despite the lack of additional otoscopy training. To study this assumption, a post hoc independent samples t-test analysis was conducted to compare pretest scores of the second-year students (n 5 8) and the posttest scores of the first-year students in the experimental group (n 5 8) who completed the training. This design is an indicator of the knowledge that the second-year students already have compared to first-year students posttraining. As shown in Figure 6, results interestingly revealed that mean performance of students in the first year posttraining was much better than the mean performance of the second-year students on both the written exam, t(14) 5 26.412, p , .001, and ear manikin, t(14) 5 28.881, p , .001, with no difference in their perception of confidence rating, t(14) 5 2.458, p 5 .654. These findings suggest that additional didactic and supervised clinical training should result in improved learning gain and otoscopy competency more than an extra year of unsupervised exposure does without close otoscopy supervision. Although Villasen˜or et al (1998) has demonstrated that didactic training in otoscopy without the use of manikins improved the otoscopy skill of students, Fisher and Pfleiderer (1992b) and results of this study evidenced by data above underline the importance of not only manikin otoscopy training but also baseline knowledge of anatomy and technique as it relates to otoscopy, both of which will likely improve clinician confidence in otoscopy. Regardless of the approach, direct supervision is considered the most important aspect to
improve students’ otoscopy skills. According to Cox (1982), direct supervision from specifically trained and experienced professionals should provide students with immediate feedback. Also, it is suggested that supervised otoscopy of at least 12 cases would improve students’ otoscopy competency to a satisfactory level (Cox, 1982; Fisher and Pfleiderer, 1992a). This has been supported by Silva and Hotaling (1997), who reported increased sensitivity (.80%) and specificity (.70%) of otolaryngology residents’ pneumatic otoscopy in diagnosing cases with OME following 4 mo of didactic and clinical training as well as examining 275 ears via pneumatic otoscopy. For those graduates and health-care professionals who are lacking otoscopy skills, Fisher and Pfleiderer (1992b) recommended taking a continuing education course or vocational training on otoscopy to improve their skills. In conclusion, results of this study demonstrated that additional training in otoscopy is a valuable tool in the education of future audiologists. Supplementing educational courses by focusing on anatomy and physiology of the outer and middle ear, otoscopy techniques, manikin training, and repetition is essential. Despite the limited clinical experience with review of still images and particular abnormal cases of ear diseases during training, the current study and others have suggested that improvements in clinical otoscopy lead to decreases in morbidity of otological pathologies (Silva and Hotaling, 1997). To gain knowledge and clinical skills for developing competency in otoscopy, communication sciences and disorders programs need to provide additional educational and clinical training to their students. Our recommendation is to implement otoscopy education and training early in the audiology curriculum starting in a course such as diagnostic audiology and advancing student otoscopy clinical skill in courses such as pediatric audiology and medical audiology to gain more
Figure 6. Subgroup comparison of pretest scores of second-year students in the control group to posttest scores of first-year students in the experimental group. Despite the extra school year of the second-year students, first-year students showed better knowledge on the written exam and better otoscopy skill using ear manikin posttraining (**p , .001). Both groups did not differ in their confidence rating.
865
Journal of the American Academy of Audiology/Volume 24, Number 9, 2013
experience in children and in cases with medical disorders of the ear. The present study suggests that training of those who will be trusted with the responsibility of assessing and managing otologic complaints must be provided with appropriate didactic and clinical training in otoscopy. In addition, a training module that consists of providing knowledge, skill, and techniques including familiarity with the proper use of the otoscope and interpretation of findings is required. It is also recommended that repeated supervised exposure and experience with patients with a wide variety of ear diseases should lead to a higher level of accurately identifying ear conditions that demand referral. Finally, why should competency in otoscopy be any different than assessing students’ competency in conducting puretone and speech audiometry with and without masking, performing and interpreting tympanometry and acoustic reflex testing, making ear mold impressions, and other audiological site-of-lesion testing? This study provides both a training module and an evaluation plan to assess competency in otoscopy of audiology students. Acknowledgments. We would like to thank all audiology students who participated in the study and Dr. Julie Masterson for providing valuable input on the statistical analysis and editorial comments on this manuscript. Also, we want to thank Lindsay Garrison and Uzma Wilson for their assistance with collecting and coding data.
Council for Clinical Certification in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association (CFCC). (2012) 2012 Standards and Implementation Procedures for the Certificate of Clinical Competence in Audiology. www. asha.org/Certification/2012-Audiology-Certification-Standards/ (accessed December 27, 2012). Cox KR. (1982) Teaching physical examination skills. In: Cox KR, Ewan CE, eds. Med Teach. Edinburgh: Churchill Livingstone, 110–113. Fisher EW, Pfleiderer AG. (1992a) Is undergraduate otoscopy teaching adequate? An audit of clinical teaching. J R Soc Med 85:23–25. Fisher EW, Pfleiderer AG. (1992b) Assessment of the otoscopic skills of general practitioners and medical students: is there room for improvement? Br J Gen Pract 42:65–67. Helenius KK, Laine MK, Ta¨ htinen PA, Elina Lahti E, Ruohola AM. (2012) Tympanometry in discrimination of otoscopic diagnoses in young ambulatory children. Pediatr Infect Dis J 31: 1003–1006. Jones WS, Kaleida PH. (2003) How helpful is pneumatic otoscopy in improving diagnostic accuracy? Pediatrics 112(3, Pt. 1): 510–513. Kaf WA, Barboa L, Fisher B, Snavely L. (2011) Effect of interdisciplinary service learning experience for audiology and speechlanguage pathology students working with adults with dementia. Am J Audiol 20:S241–S249. Kaf WA, Strong E. (2011) The promise of service learning in a pediatric audiology course on clinical training with the pediatric population. Am J Audiol 20:S220–S232.
REFERENCES
Morris E, Kesser BW, Peirce-Cottler S, Keeley M. (2012) Development and validation of a novel ear simulator to teach pneumatic otoscopy. Simul Healthc 7(1):22–26.
American Academy of Audiology. (2004) Scope of Practice. www. audiology.org/resources/documentlibrary/Pages/ScopeofPractice. aspx (accessed October 4, 2012).
Nozza RJ, Bluestone CD, Kardatzke D, Bachman R. (1994) Identification of middle ear effusion by aural acoustic admittance and otoscopy. Ear Hear 15(4):310–323.
American Academy of Pediatrics (AAP). (2004) Diagnosis and management of acute otitis media. Pediatrics 113(5):1451–1465.
Pichichero ME, Poole MD. (2001) Assessing diagnostic accuracy and tympanocentesis skills in the management of otitis media. Arch Pediatr Adolesc Med 155:1137–1142.
American Speech-Language-Hearing Association (ASHA). (1992) External Auditory Canal Examination and Cerumen Management [Guidelines, Knowledge and Skills, Position Statement]. www. asha.org/policy/GLKSPS1992-00034.htm (accessed August 20, 2012).
Scudder SG, Culbertson DS, Waldron CM, Stewart J. (2003) Predictive validity and reliability of adult hearing screening techniques. J Am Acad Audiol 14(1):9–19.
American Speech-Language-Hearing Association (ASHA). (1997) Guidelines for Audiologic Screening. Rockville, MD: American Speech-Language-Hearing Association.
Silva AB, Hotaling AJ. (1997) A protocol for otolaryngology-head and neck resident training in pneumatic otoscopy. Int J Pediatr Otorhinolaryngol 40(2–3):125–131. Sullivan R. (1993) How video otoscopy benefits hearing health care practice. Hear Instrum 44(4):15–18.
American Speech-Language-Hearing Association (ASHA). (2005) Audiology Education Summit: A Collaborative Approach [Conference report]. www.asha.org/uploadedFiles/events/audsummit2005. pdf#search5%22Audiology%22 (accessed July 13, 2012).
Sullivan R. (1997) Video otoscopy in audiologic practice. J Am Acad Audiol 8(6):447–467.
British Society of Audiology. (2004) Guidelines on Minimum Training Standards for Otoscopy and Impression Taking. www.thebsa.org.uk/ docs/Guidelines/otoscopytrainingstandardsfinalversionoct2004. pdf (accessed September 1, 2012).
Villasen˜or A, Arriaga MA, Eavey RD, Santos JI, Chissone E. (1998) Educational outcomes of an otitis media workshop for primary care providers in Latin America. Otolaryngol Head Neck Surg 118:394–396.
866
Copyright of Journal of the American Academy of Audiology is the property of American Academy of Audiology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
