2014; 36: 240–244

Influence of standardized patient body habitus on undergraduate student performance in an Objective Structured Clinical Examination VANDA YAZBECK-KARAM, SOLA AOUN BAHOUS, WISSAM FAOUR, MAYA KHAIRALLAH & NADIA ASMAR Lebanese American University, Byblos, Lebanon

Purpose: Previous studies have shown that the standardized patient’s (SP) gender may affect student performance in an Objective Structured Clinical Examination (OSCE). The aim of this study is to investigate the influence of the SPs’ body habitus on students’ performance in an OSCE counseling station. Methods: Four equally trained female SPs, with either a normal or an obese BMI participated in an OSCE counseling station for cardiovascular risk factors. Ninety-two, second year medical students were randomly assigned to one of the SPs. Station scores were compared and student behavior and opinion regarding the influence of their SP’s body habitus on their performance was assessed. Results: There was no difference in mean exam scores for students interacting with SPs with a normal BMI versus increased BMI (14.9  2.2 versus 14.01  2.2/20 respectively, p ¼ 0.06). Additionally, almost all students gave advice about healthy diets (93.5% versus 95.7%) with no specificity regarding the BMI of the SP. Conclusions: The body habitus of the SP did not significantly affect students’ performance in an undergraduate OSCE about cardiovascular risk factors, suggesting that students at that level may primarily focus on gaining points the diagnostic checklist without considering SPs as real patients.

Introduction

Practice points

The Objective Structured Clinical Examination (OSCE) was introduced in the 1970s as a tool for the measurement of clinical competencies (Harden et al. 1975). The OSCE aimed to achieve both reliability and validity by controlling many of the biases of conventional assessment methods (Collins & Harden 1998; Bartfay et al. 2004; Cleland et al. 2009). It incorporates the concept of standardized patients (SPs) (Barrows & Abrahamson 1964) to allow standardization of the type of encounter and difficulty level among students (Adamo 2003). Extensive efforts have been made to train SPs in order to reduce bias related to character variability between SPs in order to provide better psychometric assessment of student performance (McLaughlin et al. 2006; Howley et al. 2008). However, many studies have questioned the number of performance biases that can be introduced into the assessment of the examinee. Such factors include variations in SP gender, sequence of cases, day of testing (Gispert et al. 1999; Humphrey-Murto et al. 2009; Carson et al. 2010). It is important to determine these biases in OSCE grading because any variable, other than the student’s performance, that influences the rating compromises the internal validity of the evaluation process. To the authors’ knowledge, there is no published research on the influence of the SPs’ body habitus on a student’s performance in counseling stations. Therefore, in this study,

. The body habitus of the SP did not significantly affect undergraduate students’ performance regarding specific advice for weight loss. . Undergraduate medical students may have focused on gaining points without considering SPs as real patients. . Further research with larger numbers of students, SPs and stations would allow more thorough investigation about the role of the SP body habitus on students’ performance.

we looked for the effect of SP body habitus on examinee’s score in a counseling station, while controlling for case content.

20 14

Med Teach Downloaded from informahealthcare.com by University of Otago on 07/16/15 For personal use only.

Abstract

Methods This study was conducted at the Lebanese American University School of Medicine, which follows a four-year integrated curriculum. Assessment of students takes place through both summative and formative approaches, among which OSCEs. A total of 92 second-year medical students participated in a summative OSCE for the cardiovascular system, from 2010 to 2012. During orientation, students were notified that they

Correspondence: Vanda Yazbeck-Karam, MD, Associate Professor of Anesthesiology, School of Medicine, Lebanese American University, P.O. Box 36, Byblos, Lebanon. Tel: þ9613976327; email: [email protected]

240

ISSN 0142–159X print/ISSN 1466–187X online/14/030240–5 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2013.856511

Med Teach Downloaded from informahealthcare.com by University of Otago on 07/16/15 For personal use only.

SP body habitus and performance in OSCE

were part of a study involving SPs, but were not provided with further details regarding the station number in the exam. The OSCE consisted of six stations that examined clinical skills, including history taking, communication skills, physical examination and patient management. Each station was 15 minutes in length and was followed by five minutes of oral feedback provided by a doctor-examiner. To determine the existence of performance bias brought about by the SP’s body habitus, we assessed student ratings in one of the stations: a counseling station of a patient with hypertension. The use of structured checklists and trained SPs aimed at reducing error variance in performance scores. The SP program team at our institution is composed of a drama teacher responsible of the recruitment and training of SPs, and two physicians responsible for writing cases and elaborating checklists. For this case, the SPs were four professional female actors, aged between 25 and 35 years, who joined our program since its start four years ago. In preparation for this case, SPs underwent three two-hour training sessions to properly assume their roles. Also, they were given a description of their character roles and informed about possible questions they could expect from students and were instructed on appropriate responses. Two of our SPs had a body mass index (BMI) of 37.4 and 38.2 kg/m2 and two with a BMI of 21.3 and 23.0 kg/m2 [BMI was calculated using the formula: body weight (kg)/height2 (m). A person with a value between 18.5 and 24.9 kg/m2 is considered to have a normal BMI, between 25 and 29.9 kg/m2 overweight, between 30 and 35 kg/m2, obese, between 35 and 39.9 kg/m2 severely obese, and 40 kg/m2 morbid obesity]. To accommodate all students, two tracks were run simultaneously, with students and physician-raters randomly divided among the separate tracks. The real SP BMI values were given to students prior to the exam session. As for the SPs, the parallel tracks received either an SP with a normal BMI or an SP with an increased BMI. Therefore, at any point in time, there would be one SP with a normal BMI and one SP with an increased BMI running simultaneously in the parallel stations. All examiners were experienced OSCE examiners. For the purposes of this study, the examiners were chosen to be the raters, which allowed the SP portrayal to be isolated. Examiners participated in a one-hour training session and reviewed the OSCE scenarios and the checklist. This checklist was created by physicians with content expertise and each item on the checklist was linked to a learning objective (Figure 1). The number of checklist component was 16. The checklist contained items relevant to the prevention of hypertension such as alcohol intake, smoking, exercise and healthy diet as well as items related to communication skills. It also included an extra item related to obesity. This item was not graded and was only present in the checklist of students who interacted with the obese SP. Student performance was assessed on each item using a 3-point scale (0 ¼ task missed, 0.5 ¼ task attempted but incomplete, 1 ¼ task performed to the expected level). The encounter was digitally video recorded and able to be accessed by the evaluating experts, if needed. At the end of the OSCE, students who interacted with the obese SP were asked to complete a questionnaire surveying their perspective on the influence of the body habitus of their corresponding SP on their station performance.

Statistical analysis Data were analyzed using SPSS, version 19.0 for Windows (SPSS Inc., Chicago, IL). Characteristics of the study population were evaluated using descriptive statistics. Data were expressed as numbers and percentages for categorical variables and as means  SD for continuous variables. Student’s t test was used for means comparison between two groups with large sample sizes (430), the Mann–Whitney test for comparisons between groups with small sample sizes (30), and the chi-squared test for comparison of two proportions. All tests were double-sided. A value of p 5 0.05 was considered significant.

Results Ninety-two students, 58 males and 34 females, were involved in the study. Forty-six interacted with an SP with normal BMI and 46 with an SP with increased BMI. The overall mean examination score on the full OSCE was 75.5  5.9. Means for each station were representative of year 2 OSCE results at the Lebanese American University School of Medicine. The mean percentage rating for performance on the study station was not statistically significant between students exposed to an SP with a normal BMI and those interacting with an SP with an increased BMI (14.9  2.2 versus 14.01  2.2, respectively, p ¼ 0.06). Almost all students gave the SP general advice about healthy diets (93.5% and 95.7%) with no specificity regarding the actual BMI of the SP. Analysis of the students’ answers to the questionnaire showed that all students knew that obesity is a cardiovascular risk factor; however, in the group of students who interacted with the obese SP, only 17.4% advised her to lose weight while 82.6% did not. Among those who did not give this advice (38 students), 57.89% reported that they forgot to deliver a targeted weight advice (Group A), 31.57% considered that the advice to lose weight will hurt the SP’s feelings (Group B), and 10.52% elected not to advise for weight loss because this might negatively influence their grades (Group C). In a sub-analysis of students’ answers to the questionnaire, we combined Groups B and C (those who knew they should advise the SP for her current body weight but elected not to for different reasons) and compared their overall station performance to those of students in Group A. This comparison confirmed that students in Group A were significantly lower performers than those who claimed that they did not provide advice by choice (13.5  2.3 for Group A and 15  1.9 for Groups B and C combined, p ¼ 0.048).

Discussion Increasing studies have shown that the choice of SPs for OSCE cases can affect student performance and introduce rating biases (Blue et al. 2000; Guraya et al. 2010). However, recruitment of SPs for an OSCE can be a difficult task for test administrators, especially when multiple SPs are required to portray a particular role. Thus, determining the SP-associated factors that can alter OSCE ratings is crucial. Several studies clearly reported that the SP’s gender influences OSCE

241

V. Yazbeck-Karam et al.

1

0.5

0

Examiner introduces him/herself.

1

0.5

0

Examiner listens attentively and allows patient to elaborate in presenting the problem fully without interruption.

1

0.5

0

Examiner explains clearly and logically about essential hypertension.

1

0.5

0

Examiner explains the impact of healthy diet on blood pressure control.

1

0.5

0

Examiner explains the impact of exercise and physical activity on blood

Med Teach Downloaded from informahealthcare.com by University of Otago on 07/16/15 For personal use only.

pressure control. 1

0.5

0

Examiner explains the impact of smoking on blood pressure control.

1

0.5

0

Examiner explains the impact of alcohol ingestion on blood pressure control.

1

0.5

0

Examiner explains the impact of certain drugs on blood pressure control.

1

0.5

0

Examiner explains the impact of salt intake on blood pressure control.

1

0.5

0

Examiner explains about the medical treatment of hypertension.

1

0.5

0

Examiner discusses with the patient the importance of lifelong commitment to the management of hypertension.

1

0.5

0

Examiner uses repetition and summarizing to reinforce information.

1

0.5

0

Examiner uses concise, easily understood language/ avoids jargon.

1

0.5

0

Examiner checks patient’s understanding of given information.

1

0.5

0

Examiner encourages the patient to ask more questions.

1

0.5

0

Examiner shows adequate non-verbal behavior. (For students interacting with the obese SP) Student encourages the SP to lose weight

Yes ༿ No༿

Figure 1. Checklist for the counseling station.

performance ratings (Humphrey-Murto et al. 2009; Carson et al. 2010). Therefore, it was reasonable to believe that factors, such as appearance and more specifically obesity, may impact students’ performance as well. Interestingly, our data show, for the first time, that students were not influenced by the SP’s body habitus. In fact, almost all students gave the SP a general advice about healthy diets with no specificity regarding the actual BMI of the SP. BMI is a key clinical parameter that is regarded as an essential part of the routine diagnostic plan in addition to being considered a cardiovascular risk factor (Dudina et al. 2011). As such, medical students are expected to take into account high BMI value and instruct patients accordingly to lose weight. However, very few students in the group that interacted with the obese SP advised her specifically to lose weight beyond providing general information on healthy diet. In that group of students, analysis of the answers to the questionnaire taken by students who did not instruct the SP to lose weight allowed us to divide them into two categories: (i) those who did not provide advice because it did not come to their minds and (ii) those who claimed that they knew they 242

should have done so but did not want to hurt the SP’s feelings or negatively affect their grades by discussing the SP’s obesity. Analysis of their respective grades showed that those who forgot to give advice were lower overall performers than those who knew but abstained to do so. Further analysis of students’ BMI in an attempt to identify any bias related to student perception of body weight, did not reveal any significant difference between groups. Students who forgot to recall and link information about elevated BMI, development of hypertension, and weight loss could be experiencing test anxiety (Supon 2004). There is broad agreement that test anxiety is associated with lower academic performance and that a linear relationship exists between anxiety and examination performance (Cassady & Johnson 2002; Chandavarkar et al. 2007). A majority of students believe that multiple-choice question examinations give rise to less anxiety than other types of examinations. The unique qualities of the OSCE make it, understandably, a stressful experience (Reteguiz 2006). Medical students find OSCEs challenging because the simulated work environment feels too rushed and unrealistic. Furthermore, they believe that

Med Teach Downloaded from informahealthcare.com by University of Otago on 07/16/15 For personal use only.

SP body habitus and performance in OSCE

they should gain points mainly on the diagnostic checklist of the encounter, whereas personal connections with the SPs are considered less important (Blue et al. 2000). The attitude of students who worried about upsetting the SP by stressing on her increased body weight could be explained by the students’ empathy and/or sympathy toward this obese person (Halpern 2001; Benbassat & Baumal 2004). In our study involving students from the pre-clinical years, the SP is often the first ‘‘real’’ patient that the medical student interviews, examines and counsels, thus at least partly explaining the empathy/sympathy felt by our students towards the obese SP. On the other hand, students who abstained from advising the SP to lose weight fearing that stressing on her personal body image might reflect negatively on their grades viewed the SP as examiner. Although this group represents only 11% of students who interacted with an obese SP, it does question recent innovations regarding the role of SPs in student performance evaluations. In fact, many centers are now expanding the role of the SP beyond being simulated patients by using them to teach clinical skills, as well as to conduct medical interviews from a relationship-centered care approach and as examiners (Hasle et al. 1994; McGraw & O’Connor 1999; Wallace et al. 2002). SP’s assessment of examinees’ performance is already an integral part of several high-stake examinations, including the USMLE (Van Zanten et al. 2007). In this case, the SPs are trained to evaluate students’ skills based on a checklist of items for each station (Cleland et al. 2009). Regardless, it is safe to assume that our students were worried about creating an unpleasant and stressful atmosphere while interacting with the obese SP perceived as examiner, since they were concerned that she may feel stigmatized about her body weight. In fact, it is common knowledge that weight stigma and discrimination have a major impact on the mood and behavior of overweight individuals (Puhl & Heuer 2010). A major limitation to our study is the sample size and the generalizability of our results. Including students from one out of seven medical schools in Lebanon does not allow for generalizability. However, our school is one of few at our country that adopt OSCE as integral part of assessment. Understandably, larger sample sizes and wider school representation may have a varied impact on our results and warrants further investigation. In conclusion, the SP body habitus does not significantly affect student performance in an undergraduate OSCE about cardiovascular risk factors. Undergraduate medical students viewed gaining points on the diagnostic checklist as the main objective of the encounter without considering SPs as real patients. Further research with larger numbers of students, SPs and stations would allow more thorough investigation of the role of the SP’s body habitus on students’ performance. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Notes on contributors VANDA YAZBECK-KARAM, MD, is an Associate Professor of Anaesthesiology and Co-Director of the OSCE and simulation programs at the SOM, Lebanese American University. NADIA ASMAR, MD, is an Assistant Professor of Medicine at Lebanese American University. SOLA AOUN BAHOUS, MD, PhD, is an Associate Professor of Medicine at Lebanese American University. WISSAM FAOUR, PhD, is an Assistant Professor of Medicine at Lebanese American University. MAYA KHAIRALLAH, PhD, is an Assistant Professor of Medicine at Lebanese American University.

References Adamo G. 2003. Simulated and standardized patients in OSCEs: Achievements and challenges 1992–2003. Med Teach 25(3):262–270. Barrows HS, Abrahamson S. 1964. The programmed patient: A technique for appraising student performance in clinical neurology. J Med Educ 39:802–805. Bartfay WJ, Rombough R, Howse E, Leblanc R. 2004. Evaluation. The OSCE approach in nursing education. Can Nurse 100(3):18–23. Benbassat J, Baumal R. 2004. What is empathy, and how can it be promoted during clinical clerkships? Acad Med 79(9):832–839. Blue AV, Chessman AW, Gilbert GE, Mainous AG. 2000. Responding to patients emotions: Important for standardized patient satisfaction. Fam Med 32(5):326–330. Carson JA, Peets A, Grant V, McLaughlin K. 2010. The effect of gender interactions on students’ physical examination ratings in objective structured clinical examination stations. Acad Med 85(11):1772–1776. Cassady JC, Johnson RE. 2002. Cognitive test anxiety and academic performance. Contemp Educ Psychol 27(2):270–295. Chandavarkar U, Azzam A, Mathews CA. 2007. Anxiety symptoms and perceived performance in medical students. Depress Anxiety 24(2):103–111. Cleland JA, Abe K, Rethans J. 2009. The use of simulated patients in medical education: AMEE Guide No 42. Med Teach 31(6):477–486. Collins JP, Harden RM. 1998. AMEE Medical Education Guide No. 13: Real patients, simulated patients and simulators in clinical examinations. Med Teach 20(6):508–521. Dudina A, Cooney MT, Bacquer DD, Backer GD, Ducimetiere P, Jousilahti P, Keil U, Menotti A, Njolstad I, Oganov R, et al. 2011. Relationships between body mass index, cardiovascular mortality, and risk factors: A report from the SCORE investigators. Eur J Cardiovasc Prev Rehabil 18:731–742. Gispert R, Rue´ M, Roma J, Martinez-Carretero JM. 1999. Gender, sequence of cases and day effects on clinical skills assessment with standardized patients. Med Educ 33(7):499–503. Guraya SY, Alzobydi AH, Salman S. 2010. Objective structured clinical examination: Examiners’ bias and recommendations to improve its reliability. J Med Sci 1(7):269–272. Halpern J. 2001. From detached concern to empathy: Humanizing medical practice. New York: Oxford University Press. Harden RM, Stevenson M, Downie WW, Wilson GM. 1975. Assessment of clinical competence using objective structured examination. Br Med J 1:447–451. Hasle JL, Anderson DS, Szerlip HM. 1994. Analysis of the costs and benefits of using standardized patients to help teach physical diagnosis. Acad Med 69(7):567–570. Howley L, Szauter K, Perkowski L, Clifton M, McNaughton N, Association of Standardized Patient Educators (ASPE). 2008. Quality of standardised patient research reports in the medical education literature: Review and recommendations. Med Educ 42(4):350–358. Humphrey-Murto S, Touchie C, Wood TJ, Smee S. 2009. Does the gender of the standardised patient influence candidate performance in an objective structured clinical examination? Med Educ 43(6):521–525.

243

V. Yazbeck-Karam et al.

Med Teach Downloaded from informahealthcare.com by University of Otago on 07/16/15 For personal use only.

McGraw RC, O’Connor HM. 1999. Standardized patients in the early acquisition of clinical skills. Med Educ 33(8):572–578. McLaughlin K, Gregor L, Jones A, Coderre S. 2006. Can standardized patients replace physicians as OSCE examiners? BMC Med Educ 6:12. Puhl RM, Heuer CA. 2010. Obesity stigma: Important considerations for public health. Am J Public Health 100(6):1019–1028. Reteguiz JA. 2006. Relationship between anxiety and standardized patient test performance in the medicine clerkship. J Gen Intern Med 21(5):415–418.

244

Supon V. 2004. Implementing strategies to assist test-anxious students. J Instr Psychol 31(4):292–296. Van Zanten M, Boulet JR, McKinley DW, DeChamplain A, Jobe AC. 2007. Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam. Acad Med 82(10 Suppl):S65–S68. Wallace J, Rao R, Haslam R. 2002. Simulated patients and objective structured clinical examinations: Review of their use in medical education. Adv Psychiatry Treat 8:342–350.

Influence of standardized patient body habitus on undergraduate student performance in an Objective Structured Clinical Examination.

Previous studies have shown that the standardized patient's (SP) gender may affect student performance in an Objective Structured Clinical Examination...
158KB Sizes 0 Downloads 0 Views