http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
EDUCATIONAL INNOVATION IN ACTION
Design and Validation of a ComputerAided Learning Program to Enhance Students’ Ability to Recognize Lameness in the Horse Amy Barstow n Thilo Pfau n David M. Bolt n Roger K. Smith n Renate Weller ABSTRACT The ability to recognize lameness in the horse is an important skill for veterinary graduates; however, opportunities to develop this skill at the undergraduate level are limited. Computer-aided learning programs (CALs) have been successful in supplementing practical skills teaching. The aim of this study was to design and validate a CAL for the teaching of equine lameness recognition (CAL1). A control CAL was designed to simulate learning by experience (CAL2). Student volunteers were randomly assigned to either CAL and tested to establish their current ability to recognize lameness. Retesting occurred both immediately following exposure and 1 week later. At each test point, the number of correct responses for forelimb and hind limb cases was determined. Student confidence was assessed before and after CAL exposure, with previous opportunities to recognize lameness taken into account. Immediately following exposure, the number of correct responses was significantly higher for CAL1 than for CAL2, both overall and for forelimb cases but not for hind limb cases. After 1 week, the CAL1 group performed significantly better overall compared to the CAL2 group, with no significant difference between forelimb and hind limb cases. Student confidence and ability to recognize lameness were significantly improved following exposure to CAL1. When considered as one category, students in years 4 and 5 performed significantly better than year 3 students. Gender did not significantly affect performance. CAL1 could be used to supplement current lameness recognition opportunities. CAL1 is, however, limited in its ability to improve lameness recognition, especially in relation to hind limb lameness where it was unable to attain a significant difference from CAL2. Key words: equine, lameness recognition, horse, education, computer-aided learning
INTRODUCTION Lameness is one of the most common clinical problems seen in horses.1 Being able to recognize the lame limb is an essential skill for veterinary graduates as this is not only the first step in any lameness exam, but is also repeatedly used in the diagnosis and management of the lame horse.2 Visual characteristics of lameness are described in equine orthopedic textbooks2,3 and have been quantified by objective gait analysis.4–9 However, there is currently no evidence evaluating the efficacy of different methods to teach veterinary students lameness recognition.3 Current evidence suggests that more experienced observers are able to recognize equine lameness more consistently than those who are less experienced.10 For example, there is a significant difference between inexperienced and experienced observers when recognizing asymmetry based upon hind limb lame horses.11 Similarly, graduated veterinarians not working in the field of equine orthopedics are less consistent in grading lameness compared to experts in equine orthopedics and final-year veterinary students.12
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
The reduced consistency of the non-experts is likely related to the increased time period between the lameness scoring evaluation and the last time they had critically observed a lame horse.12 This finding indicates a possible link between recent exposure to lame horses and the ability to recognize lameness. Generally, inter-observer agreement is poor for scoring of equine lameness,10,13,14 emphasizing the need for teaching this skill. The current opportunities for veterinary students to learn lameness recognition skills are through lectures and during clinical rotations and clinical placements. Each student’s exposure to lame horses is variable and case-load dependent. To date, there is no published evidence regarding the efficacy of lameness teaching. However, there is evidence to suggest that hind limb lameness is more difficult to recognize and that comparatively simple measures (like the application of markers to the tuber coxae) may be useful to aid in hind limb lameness recognition.15 Computer-aided learning programs (CALs) started becoming part of the veterinary medicine curriculum in the early 1990s.16 They have been shown to be as effective as
1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
learning by experience through exposure to multiple clips of lame horses, mimicking the teaching during clinical placements and rotations. While there is evidence to suggest that experience improves an individual’s ability to recognize lameness,10,11 we postulated that by providing a structured framework with additional audio-visual aids, the students’ learning experience would be enhanced. We hypothesized that the tailor-made CAL1 would result in a significant improvement in the students’ ability to recognize equine lameness.
MATERIALS AND METHODS Design of CAL1
Figure 1: Freeze-frame from video footage of a right hind limb lame horse used for CAL1. This freeze-frame explains the difference between maximum and minimum pelvic height between the lame limb and the sound limb. White lines indicate maximum and minimum pelvic height on the sound side (left); gray lines indicate maximum and minimum pelvic height on the lame side (right). The voice-over at this point says, ‘‘This clip shows that there is a greater difference between the maximum pelvic height and the minimum pelvic height on the side of the lame limb, in this case the right hind limb. This is consistent with a greater degree of pelvic drop on the side of the lame limb.’’ CAL ¼ computer-aided learning program or even superior to didactic lecture teaching or writtentext methods in terms of student exam results.17,18 Students may also assimilate the same information in less time compared to learning from a textbook18 because CALs promote reduced erroneous cognitive load.19 CALs have also been used successfully to teach practical skills.20,21 The major advantage of using CALs for supporting practical teaching is their promotion of reflective learning and revision in a way that a tangible practical session cannot.21,22 CALs also have the benefit of addressing multiple learning styles in a succinct manner through the utilization of superior audio-visual techniques.20 With increasing student numbers23 and, in some hospitals, decreasing case loads becoming a concern,20 CALs may play a greater role in supplementing clinical and practical teaching. While multiple studies have compared CALs to textbook or didactic lecture teaching,17,18,20,24 none of these studies have assessed CALs for the teaching of lameness recognition skills. This study aimed to design and validate a tailor-made lameness recognition CAL (CAL1) using video footage that was recorded especially for use in a CAL to teach lameness recognition. Students have reported that footage filmed purposefully for use in a CAL is preferred to best-fit footage.22 In CAL1 specifically, the footage was edited to provide slow-motion clips, freeze-frames, annotations, and a voice-over (Figure 1). CAL1 was compared to a basic CAL (CAL2) created using archived footage of lame horses. CAL2 hence simulates lameness recognition
2
CAL1 was designed to give basic instructions in how to recognize the key signs of weight-bearing forelimb and hind limb lameness. Normal and slow-motion video footagea was collected from one weight-bearing forelimb lame horse and one hind limb lame horse. One horse was owned by the authors’ institution (14 years old, Irish Draft, gelding), and the other was a client-owned horse (18 years old, Irish Sports Horse, mare) that was referred to the authors’ institution for lameness investigation. Ethics approval was granted by the authors’ institution’s Ethics and Welfare Committee, and informed consent was gained from the owner. Both horses were filmed in walk and trot from the side, the front, and behind. The aim of this tutorial was to provide brief instructions in how to recognize the key signs of the gait alterations associated with the most commonly observed weightbearing forelimb and hind limb lameness. Head nod of weight-bearing forelimb lameness, pelvic hike and increased excursion of the tuber coxae in hind limb lameness, a shortened stride length, and fetlock drop were illustrated and described in the tutorial as these gait alterations are well documented in the literature. All four of these lameness characteristics were explained using summary slides, voice explanations, slow-motion clips, and annotation of the footage (Figure 1).
Design of CAL2 This CAL was compiled using archived video footage of lame horses from the authors’ institution. There were eight clips of left forelimb lameness, two clips of right forelimb lameness, five clips of right hind limb lameness, and four clips of left hind limb lameness. The only information provided during this CAL was which leg each horse was lame on. CAL2 was designed to simulate watching repeated lame horses as students would do when on clinical placements or clinical rotations. All video editing was performed in Final Cut Pro X, Version 10.0.05.b CAL1 was 10 minutes and 1 second in duration, and CAL2 was 6 minutes and 9 seconds in duration. Students were able to pause and repeat sections of the CALs and work at their own pace.
CAL Validation Students’ ability to recognize the lame limb was evaluated at three time points:
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
e Pre-test: before participating in either CAL e Post-test: immediately after completing the CALs e Recall test: 1 week after the CAL The material used to test the students included 12 clips of lame horses not included in either CAL (six forelimb and six hind limb lame horses from archived footage). For each clip, students had to decide which limb they believed the horse to be lame upon. They were able to watch the clip as many times as they liked. The same test material was used during each testing session. The test material and lameness CALs were embedded in an online survey.c Participants were recruited from years 3 to 5 of the veterinary course at the authors’ institution. Recruitment was via mass E-mail and was voluntary and anonymous. Students signed up to attend a 30-minute session where they were randomly allocated to either CAL1 (n ¼ 27) or CAL2 (n ¼ 34). Both pre-test and post-test took place on-site at the authors’ institution. The final test (recall test) was E-mailed to student participants 1 week after viewing the CALs. Students were able to access this testing material through a Web page link and complete the test wherever and whenever was convenient for them. Basic demographic data, including current year of study and the estimated number of lame horses the student had seen so far, was also collected. Both before and after the tutorial, students were also asked to rate their confidence in their ability to recognize lameness using a 1–5 Likert scale (1 ¼ least confident and 5 ¼ most confident).
Data Analysis
method was also used to assess for correlation between the number of lame horses previously observed by students and the pre-test overall score. SPSS, Version 20.0d and Excel were used for statistical analysis of the data. The significance level was set at a p value of less than .05.
RESULTS Test Scores Pre-Test Scores The CAL1 median pre-test score was 4 (inter-quartile range [IQR] ¼ 6) for forelimb cases, 3 (IQR ¼ 3) for hind limb cases, and 6 (IQR ¼ 6) overall (overall scores refer to the sum of correct responses for forelimb and hind limb cases within each test point) (Figure 2, Table 1). In CAL2, the median score was 4 (IQR ¼ 6) for forelimb cases, 3 (IQR ¼ 6) for hind limb cases, and 6 (IQR ¼ 5) overall (Figure 2, Table 1). There was no significant difference between the scores achieved by students in CAL1 and CAL2 for either forelimb cases (p ¼ .819), hind limb cases (p ¼ .733), or overall (p ¼ .924) (Table 1). In the pre-test, students in CAL1 and CAL2 showed equivalent ability to recognize lameness.
Post-Test Scores In CAL1, the median post-test score was 5 (IQR ¼ 1) for forelimb cases, 5 (IQR ¼ 3) for hind limb cases, and 10 (IQR ¼ 4) overall (Figure 2, Table 2). In CAL2, the median score was 6 (IQR ¼ 5) for forelimb cases, 3.5 (IQR ¼ 5) for hind limb cases, and 3.5 (IQR ¼ 5) overall (Figure 2,
Number of Correct Responses The distribution of the correct responses (scores) was assessed by the Shapiro–Wilk test, and the scores were found to be not normally distributed. In all test sessions (pre-test, post-test, and recall test), total correct responses to forelimb cases, hind limb cases, and the sum of forelimb and hind limb cases were calculated and compared between CAL1 and CAL2 using a Mann–Whitney U test. The median and average scores achieved for each CAL in each test session was calculated as the data. These are the most appropriate figures as the data were not normally distributed. The scores for the forelimb cases and the hind limb cases for each test session in each CAL were compared using a Wilcoxon signed-rank test. A Kruskal–Wallis test was used to compare the differences between forelimb case scores, hind limb case scores, and overall scores within each CAL and between each time point. To assess the effect that year of study had on the pretest overall scores, a Kruskal–Wallis test was also used.
Student Confidence and Experience The change in confidence within each CAL group was assessed using a Wilcoxon signed-rank test, and the difference in confidence between CAL1 and CAL2 was compared using a Mann–Whitney U test. A Spearman’s Rank-Order Correlation was performed to assess correlations between student confidence before undertaking the CALs and the pre-test total scores. This
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
Figure 2: Number of correct answers for forelimb cases (dark gray), hind limb cases (white), and both forelimb and hind limb cases combined (light gray) at each test session in CAL1 (left graph) and CAL2 (right graph). (boxes ¼ interquartile range; line within box ¼ median; whiskers ¼ range; circles ¼ outliers of 1.5 times the box length; stars ¼ extreme outliers of 3 times the box length) CAL ¼ computer-aided learning program
3
Table 1: Median, IQR, and p values for pre-test scores for forelimb, hind limb, and overall *
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Pre-test
Forelimb Hind limb Overall
CAL1
CAL2
p value
4 (IQR ¼ 6) 3 (IQR ¼ 6) 6 (IQR ¼ 6)
4 (IQR ¼ 6) 3 (IQR ¼ 6) 6 (IQR ¼ 5)
.819 .733 .924
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * No values were significant.
Table 2: Median, IQR, and p values for post-test scores for forelimb, hind limb, and overall * Post-test
Forelimb Hind limb Overall
CAL1
CAL2
p value
5 (IQR ¼ 1) 5 (IQR ¼ 3) 10 (IQR ¼ 4)
6 (IQR ¼ 5) 3.5 (IQR ¼ 5) 3.5 (IQR ¼ 5)
.022 .100 .008
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * Bolded text indicates a significant value.
Table 3: Median, IQR, and p values for recall test scores for forelimb, hind limb, and overall * Recall test
Forelimb Hind limb Overall
CAL1
CAL2
p value
5.5 (IQR ¼ 2) 5 (IQR ¼ 1) 10 (IQR ¼ 2)
5 (IQR ¼ 5 3.5 (IQR ¼ 2) 8 (IQR ¼ 6)
.164 .580 .049
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * Bolded text indicates a significant value.
Table 2). Significantly more (p ¼ .022) CAL1 students responded correctly to the forelimb cases compared to CAL2 students (Figure 2). There was no significant difference (p ¼ .100) between the number of correct responses to the hind limb cases in CAL1 and CAL2, though there was a significant difference in overall scores between CAL1 and CAL2 (p ¼ .008) (Table 2).
Recall Test Scores In CAL1, the median recall test score was 5.5 (IQR ¼ 2) for the forelimb cases, 5 (IQR ¼ 1) for hind limb cases, and 10 (IQR ¼ 2) overall (Figure 2, Table 3). In CAL2, the median was 5 (IQR ¼ 5) for forelimb cases, 3.5 (IQR ¼ 2) for hind limb cases, and 8 (IQR ¼ 6) overall (Figure 2, Table 3). There was no significant difference between CAL1 and CAL2 student performance in the forelimb (p ¼ .164) and hind limb (p ¼ .580) cases (Table 3). However, there was a significant difference between the overall number of correct responses achieved after CAL1
4
compared to CAL2 (CAL1 had more correct responses, p ¼ .049) (Table 3).
Forelimb Case Scores Versus Hind Limb Case Scores In the pre-test, neither CAL1 (p ¼ .951) nor CAL2 (p ¼ .320) yielded a significant difference between the number of correct responses to the forelimb and the hind limb cases. In the post-test for CAL1, there was a significant difference (p ¼ .009) between the number of correct responses to the forelimb cases compared to the hind limb cases, with students being more successful in the forelimb cases. In the post-test for CAL2, there was no significant difference (p ¼ .154) between the number of correct responses to the forelimb cases compared to the hind limb cases. In the recall test there was no significant difference between the number of correct responses for forelimb cases compared to hind limb cases in either CAL1 (p ¼ .386) or CAL2 (p ¼ .782).
Changes in Scores Between Pre-Test, PostTest, and Recall Test for CAL1 Between the pre-test and post-test, CAL1 students showed a significant improvement in their overall scores (p ¼ .004), scores for forelimb cases (p ¼ .003), and scores for hind limb cases (p ¼ .026) (Figure 2).
Changes in Scores Between Pre-Test, PostTest, and Recall Test for CAL2 There were no significant differences in overall scores (p ¼ .341), forelimb scores (p ¼ .687), or hind limb scores (p ¼ .275) across the three testing periods in CAL2. However, there was a trend toward an increase in overall scores (Figure 2).
Students’ Confidence in Recognizing Lameness Prior to viewing either CAL, both groups rated their confidence with a median of 2/5 (IQR ¼ 1), with no significant difference between the two groups (p ¼ .595). There was a significant moderate positive correlation (r ¼ .352, p ¼ .005) between the confidence rank before undertaking either CAL and the overall score achieved in the pretest. Following CAL exposure, the median confidence ranking in CAL1 was 3 (IQR ¼ 1) and in CAL2 was 2.5 (IQR ¼ 1). Students felt more confident in their ability to recognize lameness following participation in CAL1 compared to CAL2 (p ¼ .013). There was a significant difference between the confidence scores given before and after participation in CAL1 (p < .0001). Of the CAL1 students, 25.9% did not change their confidence rank, 55.6% increased their confidence by one rank, and 18.5% increased their confidence rank by two. There was no significant difference between the confidence scores before and after participation in CAL2 (p ¼ .248). Of the CAL2 students, 11.8% decreased their confidence rank by one, 64.7% did not change their
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
confidence rank, 23.5% of students increased their confidence rank by one, and none increased their confidence rank by two.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Number of Lame Horses Observed Before Data Collection in Relation to Test Scores Students were categorized into four groups according to the number of lame horses they had observed in the past (categories: 0–5, 6–10, 11–20, or >20). Each group was compared to the pre-test overall scores. Group 0–5 had a median score of 5 (IQR ¼ 3); group 6–10 had a median score of 6 (IQR ¼ 6); group 11–20 had a median score of 7 (IQR ¼ 5); and group >20 had a median score of 10.5 (IQR ¼ 5). There was a significant moderate positive correlation ( r ¼ .363, p ¼ .004) between the number of lame horses previously observed and the pre-test score.
Year of Study in Relation to Test Scores For third-year students the median score was 5 (IQR ¼ 5), for fourth-year students it was 8 (IQR ¼ 5), and for fifthyear students it was 8 (IQR ¼ 5). Fourth- and fifth-year students combined also had a median score of 8 (IQR ¼ 5). There was no significant difference in pre-test (p ¼ .115) and post-test (p ¼ .299) scores between the different year groups. However, when fourth- and fifth-year students were considered as one group (since both year groups have been exposed to clinical placements unlike thirdyear students), there was a significant difference between the combined fourth- and fifth-year overall scores in the pre-test and the third-year overall scores in the pre-test (p ¼ .040).
Gender and Test Scores The median score for males (n ¼ 6) in the pre-test period was 5 (IQR ¼ 2) and for females (n ¼ 54) was 6 (IQR ¼ 5). There was no significant difference in overall scores between males and females in the pre-test (p ¼ .088) or the post-test (p ¼ .973).
DISCUSSION The aim of this study was to determine if a tailor-made CAL (CAL1) was more effective in teaching students equine lameness recognition skills than a CAL designed to simulate learning by experience and to emulate current teaching methods (CAL2). In the post-test, immediately after the teaching session, CAL1 students were significantly better at correctly recognizing the lame limb, achieving significantly higher overall scores than the students in CAL2. CAL1 was hence more effective at teaching lameness recognition skills in the short term. While no other studies have compared lameness recognition teaching models, there is general evidence for the benefit of computer-aided learning methods compared to didactic lecture-based teaching. Computer-aided methods have also been shown to be superior in terms of student exam performance.17,25 A study on computer-aided delivery of a practical session describing the passing of a nasogastric tube in a horse
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
also found that CAL participants were more knowledgeable and able to pass the tube quicker than students taught in a traditionally structured practical session.20 However, other studies do not support the use of CALs and report no differences between CALs and current teaching methods or textbooks in terms of student knowledge.26 The design of CAL1 may have encouraged more effective learning by using a combination of visual (animation) and verbal (narration and summary text slides) material. Combining different modalities (verbal and visual) has been shown to be superior to using verbal and visual materials in succession or to using verbal material alone.27 In contrast, CAL2 contained predominately visual material, with only four slides to state on which limb each group of horses was lame. A balanced learning program addresses multiple learning preferences by using a diverse range of methods to convey information, such as using verbal and visual aids in conjunction.28 A learning preference describes the way a learner prefers to process, store, and retrieve learnt information.28 As students have a diverse range of learning preferences, the creation of balanced learning aids (such as CAL1) may enable optimal learning for a greater proportion of students compared to a learning aid that is biased toward one learning style (such as CAL2).28,29 It is important to note that while CAL2 attempts to simulate current equine lameness teaching, it does not provide a true representation of rotation teaching where it is usual for the clinician to explain which limb of the horse is lame and how this has been determined. As such, this study is unable to suggest that CAL teaching is superior to standard practical classes. In the recall test, 1 week after the learning session, CAL1 overall scores were significantly greater than CAL2 overall scores. There was, however, an increase between post-test and recall test scores for CAL2 but not for CAL1. It may be speculated that CAL2 students felt the need to revise between the post-test and the recall test period as the CAL and associated test material highlighted deficiencies in knowledge (also confirmed by the confidence scores). On the other hand, CAL1 students had already achieved comparatively high grades in the post-test and felt more confident following participation in CAL1; they therefore could have had less incentive to improve. The significantly higher CAL1 overall scores in the recall test are different to the findings in similar studies30,31 where there was no difference between the experimental and control group scores during an equivalent recall test. We reported a significant difference between the forelimb scores and the hind limb scores achieved by CAL1 students in the pre-test period. This finding suggests that students find it more difficult to recognize hind limb lameness than forelimb lameness.15 CAL1 students scored higher in the post-test forelimb cases than the CAL2 students but showed no significant difference in the hind limb cases. CAL1, which for hind limb lameness was limited to portraying the pelvic hike15 and asymmetry of tuber coxae excursion, was less successful in teaching hind limb lameness recognition. Individual clinicians
5
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
may observe pelvic asymmetry in different ways, focusing on the overall excursion of the pelvis (midline of tuber coxae) or the drop of the pelvis when the sound limb is weight bearing.32 This is also likely to be the case for forelimb lameness where difference in downward movement of the head (head nod) and/or upward movement can be observed.2 However, forelimb lameness appears to be easier to recognize reliably, with greater agreement between computer-assessed and observer-assessed lameness.33 The use of different features to recognize hind limb lameness may explain the reduced agreement between computer assessments and observer assessments33,34 as well as the reduced inter-observer agreement for hind limb lame horses.14 As hind limb lameness can be more difficult to observe through pelvic asymmetry alone, stride length, fetlock drop,35 and other parameters may be more critical for its observation. Students in CAL1 felt that their confidence significantly increased following exposure to the CAL, while the majority of CAL2 students felt less confident than or equally confident as before exposure. It is possible that the range of lame horses included in CAL2 highlighted deficiencies in student knowledge, while the guided tutorial in CAL1 with only one forelimb and one hind limb lame horse may have contributed to a false sense of security. However, the reported significant positive correlation between students’ confidence levels and pre-test overall scores contrasts to findings in anatomy research where students ranked themselves highly confident while performing relatively poorly.30 Our study indicates that student confidence was appropriate for the pre-test overall scores and shows no evidence of students overestimating their confidence. This finding differs from existing evidence suggesting that poor performers overestimate their abilities.36 We reported better performance of fourth- and fifthyear students compared to third-year students. This result is likely related to increased exposure to clinical case material and hence is in agreement with previous studies reporting that experience plays an important role in the ability to recognize lameness.11,12,15
CONCLUSION Increasing student numbers, greater financial constraints, and more technologically skilled students are leading to changes in the veterinary curriculum.23 This study successfully validated a multimedia CAL (CAL1) as a suitable learning resource for supplementing the current lameness recognition skill teaching. The CAL is, however, limited in its ability to satisfy the requirement of offering practical experience in lameness recognition, especially in regards to hind limb lameness recognition.
ACKNOWLEDGMENTS We would like to thank the Royal Veterinary College e-media team and Peter Nunn from the LIVE team for technical advice, the RCVS Trust for funding the equipment for this study, and all the students for their participation.
6
NOTES a b c d
Video footage was collected using a Nikon 1 J1 camera. Final Cut Pro X. Version 10.0.05. Cupertino, CA: Apple Inc. SurveyMonkey. Palo Alto, CA: SurveyMonkey; c1999– 2013. IBM SPSS. Version 20.0 New York, NY: IBM.
REFERENCES Egenvall A, Lo¨nnell C, Roepstorff L. Analysis of morbidity and mortality data in riding school horses, with special regard to locomotor problems. Prev Vet Med. 2009;88(3):193–204. http://dx.doi.org/10.1016/ j.prevetmed.2008.10.004. Medline:19042047 2 Ross MW, Dyson SJ. Diagnosis and management of lameness in the horse. 2nd ed. St Louis: ELSEVIER Saunders; 2011. p. 69–74. 3 Baxter GM. Adams and Stashak’s lameness in horses. 6th ed. Chichester: Wiley-Blackwell; 2011. p. 116–22. 4 Buchner HHF, Savelberg HHCM, Schamhardt HC, et al. Head and trunk movement adaptations in horses with experimentally induced fore- or hindlimb lameness. Equine Vet J. 1996;28(1):71–6. http://dx.doi.org/ 10.1111/j.2042-3306.1996.tb01592.x. Medline:8565958 5 Church EE, Walker AM, Wilson AM, et al. Evaluation of discriminant analysis based on dorsoventral symmetry indices to quantify hindlimb lameness during over ground locomotion in the horse. Equine Vet J. 2009;41(3):304–8. http://dx.doi.org/10.2746/ 042516409X397352. Medline:19469241 6 Keegan KG, Pai PF, Wilson DA, et al. Signal decomposition method of evaluating head movement to measure induced forelimb lameness in horses trotting on a treadmill. Equine Vet J. 2001;33(5):446–51. http:// dx.doi.org/10.2746/042516401776254781. Medline:11558738 7 Keegan KG. Evidence-based lameness detection and quantification. Vet Clin North Am Equine Pract. 2007;23(2):403–23. http://dx.doi.org/10.1016/ j.cveq.2007.04.008. Medline:17616320 8 Kramer J, Keegan KG, Kelmer G, et al. Objective determination of pelvic movement during hind limb lameness by use of a signal decomposition method and pelvic height differences. Am J Vet Res. 2004;65(6):741– 7. http://dx.doi.org/10.2460/ajvr.2004.65.741. Medline:15198212 9 Pfau T, Robilliard JJ, Weller R, et al. Assessment of mild hindlimb lameness during over ground locomotion using linear discriminant analysis of inertial sensor data. Equine Vet J. 2007;39(5):407–13. http://dx.doi.org/ 10.2746/042516407X185719. Medline:17910264 10 Keegan KG, Wilson DA, Wilson DJ, et al. Evaluation of mild lameness in horses trotting on a treadmill by clinicians and interns or residents and correlation of their assessments with kinematic gait analysis. Am J Vet Res. 1998;59(11):1370–7. Medline:9829392 1
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
11 Parkes RSV, Weller R, Groth AM, et al. Evidence of the development of ‘domain-restricted’ expertise in the recognition of asymmetric motion characteristics of hindlimb lameness in the horse. Equine Vet J. 2009;41(2):112–7. http://dx.doi.org/10.2746/ 042516408X343000. Medline:19418737 12 Arkell M, Archer RM, Guitian FJ, et al. Evidence of bias affecting the interpretation of the results of local anaesthetic nerve blocks when assessing lameness in horses. Vet Rec. 2006;159(11):346–8. http://dx.doi.org/ 10.1136/vr.159.11.346. Medline:16963714 13 Fuller CJ, Bladon BM, Driver AJ, et al. The intra- and inter-assessor reliability of measurement of functional outcome by lameness scoring in horses. Vet J. 2006;171(2):281–6. http://dx.doi.org/10.1016/ j.tvjl.2004.10.012. Medline:16490710 14 Keegan KG, Dent EV, Wilson DA, et al. Repeatability of subjective evaluation of lameness in horses. Equine Vet J. 2010;42(2):92–7. http://dx.doi.org/10.2746/ 042516409X479568. Medline:20156242 15 May SA, Wyn-Jones G. Identification of hindleg lameness. Equine Vet J. 1987;19(3):185–8. http:// dx.doi.org/10.1111/j.2042-3306.1987.tb01371.x. Medline:3608952 16 Holmes MA, Nicholls PK. Computer-aided veterinary learning at the University of Cambridge. Vet Rec. 1996;138(9):199–203. http://dx.doi.org/10.1136/ vr.138.9.199. Medline:8686151 17 Dale VMH, Sullivan M, Irvine DR. Computer-assisted learning as an alternative to didactic lectures: a study of teaching the physics of diagnostic imaging. ALT-J. 1999;7(3):75–86. 18 Friedl R, Ho¨ppler H, Ecard K, et al. Multimedia-driven teaching significantly improves students’ performance when compared with a print medium. Ann Thorac Surg. 2006;81(5):1760–6. http://dx.doi.org/10.1016/ j.athoracsur.2005.09.048. Medline:16631668 19 Khalil MK, Mansour MM, Wilhite DR. Evaluation of cognitive loads imposed by traditional paper-based and innovative computer-based instructional strategies. J Vet Med Educ. 2010;37(4):353–7. http://dx.doi.org/10.3138/ jvme.37.4.353. Medline:21135402 20 Abutarbush SM, Naylor JM, Parchoma G, et al. Evaluation of traditional instruction versus a selflearning computer module in teaching veterinary students how to pass a nasogastric tube in the horse. J Vet Med Educ. 2006;33(3):447–54. http://dx.doi.org/ 10.3138/jvme.33.3.447. Medline:17035223 21 Preast V, Danielson J, Bender H, et al. Effectiveness of a computer-based tutorial for teaching how to make a blood smear. Vet Clin Pathol. 2007;36(3):245–52. http:// dx.doi.org/10.1111/j.1939-165X.2007.tb00219.x. Medline:17806072 22 Roshier AL, Foster N, Jones MA. Veterinary students’ usage and perception of video teaching resources. BMC Med Educ. 2011;11(1):1–13. http://dx.doi.org/10.1186/ 1472-6920-11-1. Medline:21219639 23 Doherty ML, Jones BR. Undergraduate veterinary education at University College Dublin: a time of change. J Vet Med Educ. 2006;33(2):214–9. http:// dx.doi.org/10.3138/jvme.33.2.214. Medline:16849299
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
24 Denwood M, Dale VHM, Yam P. Development and evaluation of an online computer-aided learning (CAL) package to promote small-animal welfare. J Vet Med Educ. 2008;35(2):318–24. http://dx.doi.org/10.3138/ jvme.35.2.318. Medline:18723822 25 Beerman KA. Computer-based multimedia: new directions in teaching and learning. J Nutr Educ. 1996;28(1):15–8. http://dx.doi.org/10.1016/S00223182(96)70010-9. 26 Bloomfield J, Roberts J, While A. The effect of computerassisted learning versus conventional teaching methods on the acquisition and retention of handwashing theory and skills in pre-qualification nursing students: a randomised controlled trial. Int J Nurs Stud. 2010;47(3):287–94. http://dx.doi.org/10.1016/ j.ijnurstu.2009.08.003. Medline:19762016 27 Mayer RE, Moreno R. Aids to computer-based multimedia learning. Learn Instr. 2002;12(1):107–19. http://dx.doi.org/10.1016/S0959-4752(01)00018-4. 28 Churchill JA. Teaching nutrition to the left and right brain: an overview of learning styles. J Vet Med Educ. 2008;35(2):275–80. http://dx.doi.org/10.3138/ jvme.35.2.275. Medline:18723815 29 Neel JA, Grindem CB. Learning-style profiles of 150 veterinary medical students. J Vet Med Educ. 2010;37(4):347–52. http://dx.doi.org/10.3138/ jvme.37.4.347. Medline:21135401 30 Braid F, Williams SB, Weller R. Design and validation of a novel learning tool, the ‘‘Anato-Rug,’’ for teaching equine topographical anatomy. Anat Sci Educ. 2012;5(5):256–63. http://dx.doi.org/10.1002/ase.1295. Medline:22753138 31 Marsh KR, Giffin BF, Lowrie DJ Jr. Medical student retention of embryonic development: impact of the dimensions added by multimedia tutorials. Anat Sci Educ. 2008;1(6):252–7. http://dx.doi.org/10.1002/ ase.56. Medline:19109854 32 Dyson SJ. The clinician’s eye view of hindlimb lameness in the horse: technology and cognitive evaluation. Equine Vet J. 2009;41(2):99–100. http://dx.doi.org/ 10.2746/042516409X399963. Medline:19418734 33 Peham C, Licka T, Girtler D, et al. Supporting forelimb lameness: clinical judgement vs. computerised symmetry measurement. Equine Vet J. 1999;31(5):417– 21. http://dx.doi.org/10.1111/j.20423306.1999.tb03842.x. Medline:10505958 34 Peham C, Licka T, Girtler D, et al. Hindlimb lameness: clinical judgement versus computerised symmetry measurement. Vet Rec. 2001;148(24):750–2. http:// dx.doi.org/10.1136/vr.148.24.750. Medline:11442235 35 Buchner HHF, Savelberg HHCM, Schamhardt HC, et al. Limb movement adaptations in horses with experimentally induced fore- or hindlimb lameness. Equine Vet J. 1996;28(1):63–70. http://dx.doi.org/ 10.1111/j.2042-3306.1996.tb01591.x. Medline:8565956 36 Krueger J, Mueller RA. Unskilled, unaware or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. J Pers Soc Psychol. 2002;82(2):180–8. http://dx.doi.org/ 10.1037/0022-3514.82.2.180. Medline:11831408
7
AUTHOR INFORMATION
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Amy Barstow, BVetMed, (hons) MRCVS, graduated from the Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK in July 2013. E-mail: [email protected]. Amy conducted this project as part of her course because she has a special interest in equine orthopedics and felt that additional training in lameness recognition would be beneficial for students. Thilo Pfau, Dr.-Ing, FHEA, is a lecturer in biomedical engineering in the Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. Thilo’s background is computer engineering with an emphasis on signal processing, and his special interests are clinical locomotor biomechanics, especially sensor-based objective lameness recognition. David M. Bolt, Dr. med. vet., MS, MRCVS, DipACVS, DipECVS, ECVDI LA Assoc., FHEA, is a lecturer in equine surgery at the Equine Clinic, Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. David splits his time between clinical service, student teaching, and research.
8
Roger K. Smith, MA, VetMB, PhD, DEO, FHEA, ECVDI LA Assoc., DipECVS, MRCVS, is Professor of Equine Orthopedics, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. Roger has more than 20 years of experience in equine lameness and sees a wide range of lameness cases that are referred to the Royal Veterinary College. The videos used in this project were derived from his cases. Renate Weller, DVM, PhD, MRCVS, MScVetEd, ECVDI LA Assoc., FHEA, (corresponding author) is a senior lecturer in diagnostic imaging in the Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. She splits her time between research, teaching, and running the equine diagnostic imaging clinical service.
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
EDUCATIONAL INNOVATION IN ACTION
Design and Validation of a ComputerAided Learning Program to Enhance Students’ Ability to Recognize Lameness in the Horse Amy Barstow n Thilo Pfau n David M. Bolt n Roger K. Smith n Renate Weller ABSTRACT The ability to recognize lameness in the horse is an important skill for veterinary graduates; however, opportunities to develop this skill at the undergraduate level are limited. Computer-aided learning programs (CALs) have been successful in supplementing practical skills teaching. The aim of this study was to design and validate a CAL for the teaching of equine lameness recognition (CAL1). A control CAL was designed to simulate learning by experience (CAL2). Student volunteers were randomly assigned to either CAL and tested to establish their current ability to recognize lameness. Retesting occurred both immediately following exposure and 1 week later. At each test point, the number of correct responses for forelimb and hind limb cases was determined. Student confidence was assessed before and after CAL exposure, with previous opportunities to recognize lameness taken into account. Immediately following exposure, the number of correct responses was significantly higher for CAL1 than for CAL2, both overall and for forelimb cases but not for hind limb cases. After 1 week, the CAL1 group performed significantly better overall compared to the CAL2 group, with no significant difference between forelimb and hind limb cases. Student confidence and ability to recognize lameness were significantly improved following exposure to CAL1. When considered as one category, students in years 4 and 5 performed significantly better than year 3 students. Gender did not significantly affect performance. CAL1 could be used to supplement current lameness recognition opportunities. CAL1 is, however, limited in its ability to improve lameness recognition, especially in relation to hind limb lameness where it was unable to attain a significant difference from CAL2. Key words: equine, lameness recognition, horse, education, computer-aided learning
INTRODUCTION Lameness is one of the most common clinical problems seen in horses.1 Being able to recognize the lame limb is an essential skill for veterinary graduates as this is not only the first step in any lameness exam, but is also repeatedly used in the diagnosis and management of the lame horse.2 Visual characteristics of lameness are described in equine orthopedic textbooks2,3 and have been quantified by objective gait analysis.4–9 However, there is currently no evidence evaluating the efficacy of different methods to teach veterinary students lameness recognition.3 Current evidence suggests that more experienced observers are able to recognize equine lameness more consistently than those who are less experienced.10 For example, there is a significant difference between inexperienced and experienced observers when recognizing asymmetry based upon hind limb lame horses.11 Similarly, graduated veterinarians not working in the field of equine orthopedics are less consistent in grading lameness compared to experts in equine orthopedics and final-year veterinary students.12
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
The reduced consistency of the non-experts is likely related to the increased time period between the lameness scoring evaluation and the last time they had critically observed a lame horse.12 This finding indicates a possible link between recent exposure to lame horses and the ability to recognize lameness. Generally, inter-observer agreement is poor for scoring of equine lameness,10,13,14 emphasizing the need for teaching this skill. The current opportunities for veterinary students to learn lameness recognition skills are through lectures and during clinical rotations and clinical placements. Each student’s exposure to lame horses is variable and case-load dependent. To date, there is no published evidence regarding the efficacy of lameness teaching. However, there is evidence to suggest that hind limb lameness is more difficult to recognize and that comparatively simple measures (like the application of markers to the tuber coxae) may be useful to aid in hind limb lameness recognition.15 Computer-aided learning programs (CALs) started becoming part of the veterinary medicine curriculum in the early 1990s.16 They have been shown to be as effective as
1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
learning by experience through exposure to multiple clips of lame horses, mimicking the teaching during clinical placements and rotations. While there is evidence to suggest that experience improves an individual’s ability to recognize lameness,10,11 we postulated that by providing a structured framework with additional audio-visual aids, the students’ learning experience would be enhanced. We hypothesized that the tailor-made CAL1 would result in a significant improvement in the students’ ability to recognize equine lameness.
MATERIALS AND METHODS Design of CAL1
Figure 1: Freeze-frame from video footage of a right hind limb lame horse used for CAL1. This freeze-frame explains the difference between maximum and minimum pelvic height between the lame limb and the sound limb. White lines indicate maximum and minimum pelvic height on the sound side (left); gray lines indicate maximum and minimum pelvic height on the lame side (right). The voice-over at this point says, ‘‘This clip shows that there is a greater difference between the maximum pelvic height and the minimum pelvic height on the side of the lame limb, in this case the right hind limb. This is consistent with a greater degree of pelvic drop on the side of the lame limb.’’ CAL ¼ computer-aided learning program or even superior to didactic lecture teaching or writtentext methods in terms of student exam results.17,18 Students may also assimilate the same information in less time compared to learning from a textbook18 because CALs promote reduced erroneous cognitive load.19 CALs have also been used successfully to teach practical skills.20,21 The major advantage of using CALs for supporting practical teaching is their promotion of reflective learning and revision in a way that a tangible practical session cannot.21,22 CALs also have the benefit of addressing multiple learning styles in a succinct manner through the utilization of superior audio-visual techniques.20 With increasing student numbers23 and, in some hospitals, decreasing case loads becoming a concern,20 CALs may play a greater role in supplementing clinical and practical teaching. While multiple studies have compared CALs to textbook or didactic lecture teaching,17,18,20,24 none of these studies have assessed CALs for the teaching of lameness recognition skills. This study aimed to design and validate a tailor-made lameness recognition CAL (CAL1) using video footage that was recorded especially for use in a CAL to teach lameness recognition. Students have reported that footage filmed purposefully for use in a CAL is preferred to best-fit footage.22 In CAL1 specifically, the footage was edited to provide slow-motion clips, freeze-frames, annotations, and a voice-over (Figure 1). CAL1 was compared to a basic CAL (CAL2) created using archived footage of lame horses. CAL2 hence simulates lameness recognition
2
CAL1 was designed to give basic instructions in how to recognize the key signs of weight-bearing forelimb and hind limb lameness. Normal and slow-motion video footagea was collected from one weight-bearing forelimb lame horse and one hind limb lame horse. One horse was owned by the authors’ institution (14 years old, Irish Draft, gelding), and the other was a client-owned horse (18 years old, Irish Sports Horse, mare) that was referred to the authors’ institution for lameness investigation. Ethics approval was granted by the authors’ institution’s Ethics and Welfare Committee, and informed consent was gained from the owner. Both horses were filmed in walk and trot from the side, the front, and behind. The aim of this tutorial was to provide brief instructions in how to recognize the key signs of the gait alterations associated with the most commonly observed weightbearing forelimb and hind limb lameness. Head nod of weight-bearing forelimb lameness, pelvic hike and increased excursion of the tuber coxae in hind limb lameness, a shortened stride length, and fetlock drop were illustrated and described in the tutorial as these gait alterations are well documented in the literature. All four of these lameness characteristics were explained using summary slides, voice explanations, slow-motion clips, and annotation of the footage (Figure 1).
Design of CAL2 This CAL was compiled using archived video footage of lame horses from the authors’ institution. There were eight clips of left forelimb lameness, two clips of right forelimb lameness, five clips of right hind limb lameness, and four clips of left hind limb lameness. The only information provided during this CAL was which leg each horse was lame on. CAL2 was designed to simulate watching repeated lame horses as students would do when on clinical placements or clinical rotations. All video editing was performed in Final Cut Pro X, Version 10.0.05.b CAL1 was 10 minutes and 1 second in duration, and CAL2 was 6 minutes and 9 seconds in duration. Students were able to pause and repeat sections of the CALs and work at their own pace.
CAL Validation Students’ ability to recognize the lame limb was evaluated at three time points:
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
e Pre-test: before participating in either CAL e Post-test: immediately after completing the CALs e Recall test: 1 week after the CAL The material used to test the students included 12 clips of lame horses not included in either CAL (six forelimb and six hind limb lame horses from archived footage). For each clip, students had to decide which limb they believed the horse to be lame upon. They were able to watch the clip as many times as they liked. The same test material was used during each testing session. The test material and lameness CALs were embedded in an online survey.c Participants were recruited from years 3 to 5 of the veterinary course at the authors’ institution. Recruitment was via mass E-mail and was voluntary and anonymous. Students signed up to attend a 30-minute session where they were randomly allocated to either CAL1 (n ¼ 27) or CAL2 (n ¼ 34). Both pre-test and post-test took place on-site at the authors’ institution. The final test (recall test) was E-mailed to student participants 1 week after viewing the CALs. Students were able to access this testing material through a Web page link and complete the test wherever and whenever was convenient for them. Basic demographic data, including current year of study and the estimated number of lame horses the student had seen so far, was also collected. Both before and after the tutorial, students were also asked to rate their confidence in their ability to recognize lameness using a 1–5 Likert scale (1 ¼ least confident and 5 ¼ most confident).
Data Analysis
method was also used to assess for correlation between the number of lame horses previously observed by students and the pre-test overall score. SPSS, Version 20.0d and Excel were used for statistical analysis of the data. The significance level was set at a p value of less than .05.
RESULTS Test Scores Pre-Test Scores The CAL1 median pre-test score was 4 (inter-quartile range [IQR] ¼ 6) for forelimb cases, 3 (IQR ¼ 3) for hind limb cases, and 6 (IQR ¼ 6) overall (overall scores refer to the sum of correct responses for forelimb and hind limb cases within each test point) (Figure 2, Table 1). In CAL2, the median score was 4 (IQR ¼ 6) for forelimb cases, 3 (IQR ¼ 6) for hind limb cases, and 6 (IQR ¼ 5) overall (Figure 2, Table 1). There was no significant difference between the scores achieved by students in CAL1 and CAL2 for either forelimb cases (p ¼ .819), hind limb cases (p ¼ .733), or overall (p ¼ .924) (Table 1). In the pre-test, students in CAL1 and CAL2 showed equivalent ability to recognize lameness.
Post-Test Scores In CAL1, the median post-test score was 5 (IQR ¼ 1) for forelimb cases, 5 (IQR ¼ 3) for hind limb cases, and 10 (IQR ¼ 4) overall (Figure 2, Table 2). In CAL2, the median score was 6 (IQR ¼ 5) for forelimb cases, 3.5 (IQR ¼ 5) for hind limb cases, and 3.5 (IQR ¼ 5) overall (Figure 2,
Number of Correct Responses The distribution of the correct responses (scores) was assessed by the Shapiro–Wilk test, and the scores were found to be not normally distributed. In all test sessions (pre-test, post-test, and recall test), total correct responses to forelimb cases, hind limb cases, and the sum of forelimb and hind limb cases were calculated and compared between CAL1 and CAL2 using a Mann–Whitney U test. The median and average scores achieved for each CAL in each test session was calculated as the data. These are the most appropriate figures as the data were not normally distributed. The scores for the forelimb cases and the hind limb cases for each test session in each CAL were compared using a Wilcoxon signed-rank test. A Kruskal–Wallis test was used to compare the differences between forelimb case scores, hind limb case scores, and overall scores within each CAL and between each time point. To assess the effect that year of study had on the pretest overall scores, a Kruskal–Wallis test was also used.
Student Confidence and Experience The change in confidence within each CAL group was assessed using a Wilcoxon signed-rank test, and the difference in confidence between CAL1 and CAL2 was compared using a Mann–Whitney U test. A Spearman’s Rank-Order Correlation was performed to assess correlations between student confidence before undertaking the CALs and the pre-test total scores. This
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
Figure 2: Number of correct answers for forelimb cases (dark gray), hind limb cases (white), and both forelimb and hind limb cases combined (light gray) at each test session in CAL1 (left graph) and CAL2 (right graph). (boxes ¼ interquartile range; line within box ¼ median; whiskers ¼ range; circles ¼ outliers of 1.5 times the box length; stars ¼ extreme outliers of 3 times the box length) CAL ¼ computer-aided learning program
3
Table 1: Median, IQR, and p values for pre-test scores for forelimb, hind limb, and overall *
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Pre-test
Forelimb Hind limb Overall
CAL1
CAL2
p value
4 (IQR ¼ 6) 3 (IQR ¼ 6) 6 (IQR ¼ 6)
4 (IQR ¼ 6) 3 (IQR ¼ 6) 6 (IQR ¼ 5)
.819 .733 .924
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * No values were significant.
Table 2: Median, IQR, and p values for post-test scores for forelimb, hind limb, and overall * Post-test
Forelimb Hind limb Overall
CAL1
CAL2
p value
5 (IQR ¼ 1) 5 (IQR ¼ 3) 10 (IQR ¼ 4)
6 (IQR ¼ 5) 3.5 (IQR ¼ 5) 3.5 (IQR ¼ 5)
.022 .100 .008
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * Bolded text indicates a significant value.
Table 3: Median, IQR, and p values for recall test scores for forelimb, hind limb, and overall * Recall test
Forelimb Hind limb Overall
CAL1
CAL2
p value
5.5 (IQR ¼ 2) 5 (IQR ¼ 1) 10 (IQR ¼ 2)
5 (IQR ¼ 5 3.5 (IQR ¼ 2) 8 (IQR ¼ 6)
.164 .580 .049
IQR ¼ inter-quartile range; CAL ¼ computer-aided learning program * Bolded text indicates a significant value.
Table 2). Significantly more (p ¼ .022) CAL1 students responded correctly to the forelimb cases compared to CAL2 students (Figure 2). There was no significant difference (p ¼ .100) between the number of correct responses to the hind limb cases in CAL1 and CAL2, though there was a significant difference in overall scores between CAL1 and CAL2 (p ¼ .008) (Table 2).
Recall Test Scores In CAL1, the median recall test score was 5.5 (IQR ¼ 2) for the forelimb cases, 5 (IQR ¼ 1) for hind limb cases, and 10 (IQR ¼ 2) overall (Figure 2, Table 3). In CAL2, the median was 5 (IQR ¼ 5) for forelimb cases, 3.5 (IQR ¼ 2) for hind limb cases, and 8 (IQR ¼ 6) overall (Figure 2, Table 3). There was no significant difference between CAL1 and CAL2 student performance in the forelimb (p ¼ .164) and hind limb (p ¼ .580) cases (Table 3). However, there was a significant difference between the overall number of correct responses achieved after CAL1
4
compared to CAL2 (CAL1 had more correct responses, p ¼ .049) (Table 3).
Forelimb Case Scores Versus Hind Limb Case Scores In the pre-test, neither CAL1 (p ¼ .951) nor CAL2 (p ¼ .320) yielded a significant difference between the number of correct responses to the forelimb and the hind limb cases. In the post-test for CAL1, there was a significant difference (p ¼ .009) between the number of correct responses to the forelimb cases compared to the hind limb cases, with students being more successful in the forelimb cases. In the post-test for CAL2, there was no significant difference (p ¼ .154) between the number of correct responses to the forelimb cases compared to the hind limb cases. In the recall test there was no significant difference between the number of correct responses for forelimb cases compared to hind limb cases in either CAL1 (p ¼ .386) or CAL2 (p ¼ .782).
Changes in Scores Between Pre-Test, PostTest, and Recall Test for CAL1 Between the pre-test and post-test, CAL1 students showed a significant improvement in their overall scores (p ¼ .004), scores for forelimb cases (p ¼ .003), and scores for hind limb cases (p ¼ .026) (Figure 2).
Changes in Scores Between Pre-Test, PostTest, and Recall Test for CAL2 There were no significant differences in overall scores (p ¼ .341), forelimb scores (p ¼ .687), or hind limb scores (p ¼ .275) across the three testing periods in CAL2. However, there was a trend toward an increase in overall scores (Figure 2).
Students’ Confidence in Recognizing Lameness Prior to viewing either CAL, both groups rated their confidence with a median of 2/5 (IQR ¼ 1), with no significant difference between the two groups (p ¼ .595). There was a significant moderate positive correlation (r ¼ .352, p ¼ .005) between the confidence rank before undertaking either CAL and the overall score achieved in the pretest. Following CAL exposure, the median confidence ranking in CAL1 was 3 (IQR ¼ 1) and in CAL2 was 2.5 (IQR ¼ 1). Students felt more confident in their ability to recognize lameness following participation in CAL1 compared to CAL2 (p ¼ .013). There was a significant difference between the confidence scores given before and after participation in CAL1 (p < .0001). Of the CAL1 students, 25.9% did not change their confidence rank, 55.6% increased their confidence by one rank, and 18.5% increased their confidence rank by two. There was no significant difference between the confidence scores before and after participation in CAL2 (p ¼ .248). Of the CAL2 students, 11.8% decreased their confidence rank by one, 64.7% did not change their
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
confidence rank, 23.5% of students increased their confidence rank by one, and none increased their confidence rank by two.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Number of Lame Horses Observed Before Data Collection in Relation to Test Scores Students were categorized into four groups according to the number of lame horses they had observed in the past (categories: 0–5, 6–10, 11–20, or >20). Each group was compared to the pre-test overall scores. Group 0–5 had a median score of 5 (IQR ¼ 3); group 6–10 had a median score of 6 (IQR ¼ 6); group 11–20 had a median score of 7 (IQR ¼ 5); and group >20 had a median score of 10.5 (IQR ¼ 5). There was a significant moderate positive correlation ( r ¼ .363, p ¼ .004) between the number of lame horses previously observed and the pre-test score.
Year of Study in Relation to Test Scores For third-year students the median score was 5 (IQR ¼ 5), for fourth-year students it was 8 (IQR ¼ 5), and for fifthyear students it was 8 (IQR ¼ 5). Fourth- and fifth-year students combined also had a median score of 8 (IQR ¼ 5). There was no significant difference in pre-test (p ¼ .115) and post-test (p ¼ .299) scores between the different year groups. However, when fourth- and fifth-year students were considered as one group (since both year groups have been exposed to clinical placements unlike thirdyear students), there was a significant difference between the combined fourth- and fifth-year overall scores in the pre-test and the third-year overall scores in the pre-test (p ¼ .040).
Gender and Test Scores The median score for males (n ¼ 6) in the pre-test period was 5 (IQR ¼ 2) and for females (n ¼ 54) was 6 (IQR ¼ 5). There was no significant difference in overall scores between males and females in the pre-test (p ¼ .088) or the post-test (p ¼ .973).
DISCUSSION The aim of this study was to determine if a tailor-made CAL (CAL1) was more effective in teaching students equine lameness recognition skills than a CAL designed to simulate learning by experience and to emulate current teaching methods (CAL2). In the post-test, immediately after the teaching session, CAL1 students were significantly better at correctly recognizing the lame limb, achieving significantly higher overall scores than the students in CAL2. CAL1 was hence more effective at teaching lameness recognition skills in the short term. While no other studies have compared lameness recognition teaching models, there is general evidence for the benefit of computer-aided learning methods compared to didactic lecture-based teaching. Computer-aided methods have also been shown to be superior in terms of student exam performance.17,25 A study on computer-aided delivery of a practical session describing the passing of a nasogastric tube in a horse
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
also found that CAL participants were more knowledgeable and able to pass the tube quicker than students taught in a traditionally structured practical session.20 However, other studies do not support the use of CALs and report no differences between CALs and current teaching methods or textbooks in terms of student knowledge.26 The design of CAL1 may have encouraged more effective learning by using a combination of visual (animation) and verbal (narration and summary text slides) material. Combining different modalities (verbal and visual) has been shown to be superior to using verbal and visual materials in succession or to using verbal material alone.27 In contrast, CAL2 contained predominately visual material, with only four slides to state on which limb each group of horses was lame. A balanced learning program addresses multiple learning preferences by using a diverse range of methods to convey information, such as using verbal and visual aids in conjunction.28 A learning preference describes the way a learner prefers to process, store, and retrieve learnt information.28 As students have a diverse range of learning preferences, the creation of balanced learning aids (such as CAL1) may enable optimal learning for a greater proportion of students compared to a learning aid that is biased toward one learning style (such as CAL2).28,29 It is important to note that while CAL2 attempts to simulate current equine lameness teaching, it does not provide a true representation of rotation teaching where it is usual for the clinician to explain which limb of the horse is lame and how this has been determined. As such, this study is unable to suggest that CAL teaching is superior to standard practical classes. In the recall test, 1 week after the learning session, CAL1 overall scores were significantly greater than CAL2 overall scores. There was, however, an increase between post-test and recall test scores for CAL2 but not for CAL1. It may be speculated that CAL2 students felt the need to revise between the post-test and the recall test period as the CAL and associated test material highlighted deficiencies in knowledge (also confirmed by the confidence scores). On the other hand, CAL1 students had already achieved comparatively high grades in the post-test and felt more confident following participation in CAL1; they therefore could have had less incentive to improve. The significantly higher CAL1 overall scores in the recall test are different to the findings in similar studies30,31 where there was no difference between the experimental and control group scores during an equivalent recall test. We reported a significant difference between the forelimb scores and the hind limb scores achieved by CAL1 students in the pre-test period. This finding suggests that students find it more difficult to recognize hind limb lameness than forelimb lameness.15 CAL1 students scored higher in the post-test forelimb cases than the CAL2 students but showed no significant difference in the hind limb cases. CAL1, which for hind limb lameness was limited to portraying the pelvic hike15 and asymmetry of tuber coxae excursion, was less successful in teaching hind limb lameness recognition. Individual clinicians
5
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
may observe pelvic asymmetry in different ways, focusing on the overall excursion of the pelvis (midline of tuber coxae) or the drop of the pelvis when the sound limb is weight bearing.32 This is also likely to be the case for forelimb lameness where difference in downward movement of the head (head nod) and/or upward movement can be observed.2 However, forelimb lameness appears to be easier to recognize reliably, with greater agreement between computer-assessed and observer-assessed lameness.33 The use of different features to recognize hind limb lameness may explain the reduced agreement between computer assessments and observer assessments33,34 as well as the reduced inter-observer agreement for hind limb lame horses.14 As hind limb lameness can be more difficult to observe through pelvic asymmetry alone, stride length, fetlock drop,35 and other parameters may be more critical for its observation. Students in CAL1 felt that their confidence significantly increased following exposure to the CAL, while the majority of CAL2 students felt less confident than or equally confident as before exposure. It is possible that the range of lame horses included in CAL2 highlighted deficiencies in student knowledge, while the guided tutorial in CAL1 with only one forelimb and one hind limb lame horse may have contributed to a false sense of security. However, the reported significant positive correlation between students’ confidence levels and pre-test overall scores contrasts to findings in anatomy research where students ranked themselves highly confident while performing relatively poorly.30 Our study indicates that student confidence was appropriate for the pre-test overall scores and shows no evidence of students overestimating their confidence. This finding differs from existing evidence suggesting that poor performers overestimate their abilities.36 We reported better performance of fourth- and fifthyear students compared to third-year students. This result is likely related to increased exposure to clinical case material and hence is in agreement with previous studies reporting that experience plays an important role in the ability to recognize lameness.11,12,15
CONCLUSION Increasing student numbers, greater financial constraints, and more technologically skilled students are leading to changes in the veterinary curriculum.23 This study successfully validated a multimedia CAL (CAL1) as a suitable learning resource for supplementing the current lameness recognition skill teaching. The CAL is, however, limited in its ability to satisfy the requirement of offering practical experience in lameness recognition, especially in regards to hind limb lameness recognition.
ACKNOWLEDGMENTS We would like to thank the Royal Veterinary College e-media team and Peter Nunn from the LIVE team for technical advice, the RCVS Trust for funding the equipment for this study, and all the students for their participation.
6
NOTES a b c d
Video footage was collected using a Nikon 1 J1 camera. Final Cut Pro X. Version 10.0.05. Cupertino, CA: Apple Inc. SurveyMonkey. Palo Alto, CA: SurveyMonkey; c1999– 2013. IBM SPSS. Version 20.0 New York, NY: IBM.
REFERENCES Egenvall A, Lo¨nnell C, Roepstorff L. Analysis of morbidity and mortality data in riding school horses, with special regard to locomotor problems. Prev Vet Med. 2009;88(3):193–204. http://dx.doi.org/10.1016/ j.prevetmed.2008.10.004. Medline:19042047 2 Ross MW, Dyson SJ. Diagnosis and management of lameness in the horse. 2nd ed. St Louis: ELSEVIER Saunders; 2011. p. 69–74. 3 Baxter GM. Adams and Stashak’s lameness in horses. 6th ed. Chichester: Wiley-Blackwell; 2011. p. 116–22. 4 Buchner HHF, Savelberg HHCM, Schamhardt HC, et al. Head and trunk movement adaptations in horses with experimentally induced fore- or hindlimb lameness. Equine Vet J. 1996;28(1):71–6. http://dx.doi.org/ 10.1111/j.2042-3306.1996.tb01592.x. Medline:8565958 5 Church EE, Walker AM, Wilson AM, et al. Evaluation of discriminant analysis based on dorsoventral symmetry indices to quantify hindlimb lameness during over ground locomotion in the horse. Equine Vet J. 2009;41(3):304–8. http://dx.doi.org/10.2746/ 042516409X397352. Medline:19469241 6 Keegan KG, Pai PF, Wilson DA, et al. Signal decomposition method of evaluating head movement to measure induced forelimb lameness in horses trotting on a treadmill. Equine Vet J. 2001;33(5):446–51. http:// dx.doi.org/10.2746/042516401776254781. Medline:11558738 7 Keegan KG. Evidence-based lameness detection and quantification. Vet Clin North Am Equine Pract. 2007;23(2):403–23. http://dx.doi.org/10.1016/ j.cveq.2007.04.008. Medline:17616320 8 Kramer J, Keegan KG, Kelmer G, et al. Objective determination of pelvic movement during hind limb lameness by use of a signal decomposition method and pelvic height differences. Am J Vet Res. 2004;65(6):741– 7. http://dx.doi.org/10.2460/ajvr.2004.65.741. Medline:15198212 9 Pfau T, Robilliard JJ, Weller R, et al. Assessment of mild hindlimb lameness during over ground locomotion using linear discriminant analysis of inertial sensor data. Equine Vet J. 2007;39(5):407–13. http://dx.doi.org/ 10.2746/042516407X185719. Medline:17910264 10 Keegan KG, Wilson DA, Wilson DJ, et al. Evaluation of mild lameness in horses trotting on a treadmill by clinicians and interns or residents and correlation of their assessments with kinematic gait analysis. Am J Vet Res. 1998;59(11):1370–7. Medline:9829392 1
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
11 Parkes RSV, Weller R, Groth AM, et al. Evidence of the development of ‘domain-restricted’ expertise in the recognition of asymmetric motion characteristics of hindlimb lameness in the horse. Equine Vet J. 2009;41(2):112–7. http://dx.doi.org/10.2746/ 042516408X343000. Medline:19418737 12 Arkell M, Archer RM, Guitian FJ, et al. Evidence of bias affecting the interpretation of the results of local anaesthetic nerve blocks when assessing lameness in horses. Vet Rec. 2006;159(11):346–8. http://dx.doi.org/ 10.1136/vr.159.11.346. Medline:16963714 13 Fuller CJ, Bladon BM, Driver AJ, et al. The intra- and inter-assessor reliability of measurement of functional outcome by lameness scoring in horses. Vet J. 2006;171(2):281–6. http://dx.doi.org/10.1016/ j.tvjl.2004.10.012. Medline:16490710 14 Keegan KG, Dent EV, Wilson DA, et al. Repeatability of subjective evaluation of lameness in horses. Equine Vet J. 2010;42(2):92–7. http://dx.doi.org/10.2746/ 042516409X479568. Medline:20156242 15 May SA, Wyn-Jones G. Identification of hindleg lameness. Equine Vet J. 1987;19(3):185–8. http:// dx.doi.org/10.1111/j.2042-3306.1987.tb01371.x. Medline:3608952 16 Holmes MA, Nicholls PK. Computer-aided veterinary learning at the University of Cambridge. Vet Rec. 1996;138(9):199–203. http://dx.doi.org/10.1136/ vr.138.9.199. Medline:8686151 17 Dale VMH, Sullivan M, Irvine DR. Computer-assisted learning as an alternative to didactic lectures: a study of teaching the physics of diagnostic imaging. ALT-J. 1999;7(3):75–86. 18 Friedl R, Ho¨ppler H, Ecard K, et al. Multimedia-driven teaching significantly improves students’ performance when compared with a print medium. Ann Thorac Surg. 2006;81(5):1760–6. http://dx.doi.org/10.1016/ j.athoracsur.2005.09.048. Medline:16631668 19 Khalil MK, Mansour MM, Wilhite DR. Evaluation of cognitive loads imposed by traditional paper-based and innovative computer-based instructional strategies. J Vet Med Educ. 2010;37(4):353–7. http://dx.doi.org/10.3138/ jvme.37.4.353. Medline:21135402 20 Abutarbush SM, Naylor JM, Parchoma G, et al. Evaluation of traditional instruction versus a selflearning computer module in teaching veterinary students how to pass a nasogastric tube in the horse. J Vet Med Educ. 2006;33(3):447–54. http://dx.doi.org/ 10.3138/jvme.33.3.447. Medline:17035223 21 Preast V, Danielson J, Bender H, et al. Effectiveness of a computer-based tutorial for teaching how to make a blood smear. Vet Clin Pathol. 2007;36(3):245–52. http:// dx.doi.org/10.1111/j.1939-165X.2007.tb00219.x. Medline:17806072 22 Roshier AL, Foster N, Jones MA. Veterinary students’ usage and perception of video teaching resources. BMC Med Educ. 2011;11(1):1–13. http://dx.doi.org/10.1186/ 1472-6920-11-1. Medline:21219639 23 Doherty ML, Jones BR. Undergraduate veterinary education at University College Dublin: a time of change. J Vet Med Educ. 2006;33(2):214–9. http:// dx.doi.org/10.3138/jvme.33.2.214. Medline:16849299
doi: 10.3138/jvme.0213-040R1
JVME 41(1) 8 2014 AAVMC
24 Denwood M, Dale VHM, Yam P. Development and evaluation of an online computer-aided learning (CAL) package to promote small-animal welfare. J Vet Med Educ. 2008;35(2):318–24. http://dx.doi.org/10.3138/ jvme.35.2.318. Medline:18723822 25 Beerman KA. Computer-based multimedia: new directions in teaching and learning. J Nutr Educ. 1996;28(1):15–8. http://dx.doi.org/10.1016/S00223182(96)70010-9. 26 Bloomfield J, Roberts J, While A. The effect of computerassisted learning versus conventional teaching methods on the acquisition and retention of handwashing theory and skills in pre-qualification nursing students: a randomised controlled trial. Int J Nurs Stud. 2010;47(3):287–94. http://dx.doi.org/10.1016/ j.ijnurstu.2009.08.003. Medline:19762016 27 Mayer RE, Moreno R. Aids to computer-based multimedia learning. Learn Instr. 2002;12(1):107–19. http://dx.doi.org/10.1016/S0959-4752(01)00018-4. 28 Churchill JA. Teaching nutrition to the left and right brain: an overview of learning styles. J Vet Med Educ. 2008;35(2):275–80. http://dx.doi.org/10.3138/ jvme.35.2.275. Medline:18723815 29 Neel JA, Grindem CB. Learning-style profiles of 150 veterinary medical students. J Vet Med Educ. 2010;37(4):347–52. http://dx.doi.org/10.3138/ jvme.37.4.347. Medline:21135401 30 Braid F, Williams SB, Weller R. Design and validation of a novel learning tool, the ‘‘Anato-Rug,’’ for teaching equine topographical anatomy. Anat Sci Educ. 2012;5(5):256–63. http://dx.doi.org/10.1002/ase.1295. Medline:22753138 31 Marsh KR, Giffin BF, Lowrie DJ Jr. Medical student retention of embryonic development: impact of the dimensions added by multimedia tutorials. Anat Sci Educ. 2008;1(6):252–7. http://dx.doi.org/10.1002/ ase.56. Medline:19109854 32 Dyson SJ. The clinician’s eye view of hindlimb lameness in the horse: technology and cognitive evaluation. Equine Vet J. 2009;41(2):99–100. http://dx.doi.org/ 10.2746/042516409X399963. Medline:19418734 33 Peham C, Licka T, Girtler D, et al. Supporting forelimb lameness: clinical judgement vs. computerised symmetry measurement. Equine Vet J. 1999;31(5):417– 21. http://dx.doi.org/10.1111/j.20423306.1999.tb03842.x. Medline:10505958 34 Peham C, Licka T, Girtler D, et al. Hindlimb lameness: clinical judgement versus computerised symmetry measurement. Vet Rec. 2001;148(24):750–2. http:// dx.doi.org/10.1136/vr.148.24.750. Medline:11442235 35 Buchner HHF, Savelberg HHCM, Schamhardt HC, et al. Limb movement adaptations in horses with experimentally induced fore- or hindlimb lameness. Equine Vet J. 1996;28(1):63–70. http://dx.doi.org/ 10.1111/j.2042-3306.1996.tb01591.x. Medline:8565956 36 Krueger J, Mueller RA. Unskilled, unaware or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. J Pers Soc Psychol. 2002;82(2):180–8. http://dx.doi.org/ 10.1037/0022-3514.82.2.180. Medline:11831408
7
AUTHOR INFORMATION
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.0213-040R1 - Wednesday, May 09, 2018 1:06:57 AM - University of Adelaide IP Address:129.127.145.240
Amy Barstow, BVetMed, (hons) MRCVS, graduated from the Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK in July 2013. E-mail: [email protected]. Amy conducted this project as part of her course because she has a special interest in equine orthopedics and felt that additional training in lameness recognition would be beneficial for students. Thilo Pfau, Dr.-Ing, FHEA, is a lecturer in biomedical engineering in the Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. Thilo’s background is computer engineering with an emphasis on signal processing, and his special interests are clinical locomotor biomechanics, especially sensor-based objective lameness recognition. David M. Bolt, Dr. med. vet., MS, MRCVS, DipACVS, DipECVS, ECVDI LA Assoc., FHEA, is a lecturer in equine surgery at the Equine Clinic, Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. David splits his time between clinical service, student teaching, and research.
8
Roger K. Smith, MA, VetMB, PhD, DEO, FHEA, ECVDI LA Assoc., DipECVS, MRCVS, is Professor of Equine Orthopedics, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. Roger has more than 20 years of experience in equine lameness and sees a wide range of lameness cases that are referred to the Royal Veterinary College. The videos used in this project were derived from his cases. Renate Weller, DVM, PhD, MRCVS, MScVetEd, ECVDI LA Assoc., FHEA, (corresponding author) is a senior lecturer in diagnostic imaging in the Department of Veterinary Clinical Sciences, Royal Veterinary College, Hawkshead Lane, North Mymms, Hatfield, AL9 7TA UK. E-mail: [email protected]. She splits her time between research, teaching, and running the equine diagnostic imaging clinical service.
JVME 41(1) 8 2014 AAVMC
doi: 10.3138/jvme.0213-040R1
