ORIGINAL REPORTS
The Surgical Clerkship and Medical Student Performance in a Standardized Patient Case of Acute Cholecystitis Eric Low, MPH,* Robert Tessler, MD,† Karen E. Hauer, MD,‡ Andrew D. Leavitt, MD,‡,§ Bernie Miller, BA,* and John Maa, MD, FACS† School of Medicine, University of California, San Francisco, California; †Department of Surgery, University of California, San Francisco, California; ‡Department of Medicine, University of California, San Francisco, California; and §Department Laboratory Medicine, University of California, San Francisco, California *
BACKGROUND: Although an Observed Structured Clinical Examination (OSCE) has been used to evaluate patient interaction and general knowledge competencies of thirdyear students during their required surgical clerkships, whether surgical clerkship experience predicts satisfactory performance with a surgical patient in an OSCE environment has not been investigated. OBJECTIVE: We hypothesized that completion of the
third-year surgery clerkship would improve student ability to diagnose acute cholecystitis and recognize the further need for hospital admission and treatment. DESIGN: An observational study design was used to determine student skills in evaluating a simulated surgical patient with abdominal pain from acute cholecystitis. The skills included key data gathering, physical examination, and information-sharing tasks. SETTING: Tertiary care academic medical center. PARTICIPANTS: Performance was compared between a cohort of 101 medical students who had completed the third-year surgical clerkship and 72 who had not. A secondary analysis compared performance for 91 students who had completed their third-year clerkship in sites near the University of California, San Francisco School of Medicine, and 10 who did so at a regional campus geographically distant from the medical school. RESULTS: Of the 173 students who participated in the
OSCE, only 42% correctly identified the diagnosis of acute
Disclaimers: This study was performed in compliance with our institutional Ethical Board and/or Committee. This work was presented at the AAMC Group on Regional Medical Campuses meeting in Philadelphia in 2013. Correspondence: Inquiries to John Maa, MD, FACS, University of California, Office of the President, 300 Lakeside Drive, 6th Floor, Oakland, CA 94612-3550 and Marin General Hospital; E-mail: [email protected], [email protected]
cholecystitis, though 71% did suggest the possibility of a biliary process to the standardized patient. Most of the students who identified the condition as acute cholecystitis or gallbladder-related process had completed their third-year surgical clerkship (odds ratio [OR] ¼ 3.26). Students who completed their surgical clerkship were also better able to recommend appropriate treatment for the patient (OR ¼ 2.35), and recommend admission to the hospital or emergency department (OR ¼ 2.00). Approximately one-third (35.3%) of all students documented a positive Murphy’s sign, but only 6.4% identified the triad of leukocytosis, fever, and a Murphy’s sign as diagnostic of acute cholecystitis and the need for surgical intervention. Student performance on the clinical examination did not differ depending on whether the students completed their clerkship at a Bay Area or regional hospital. CONCLUSIONS: Student recognition of the key physical
examination and laboratory findings diagnostic of acute cholecystitis was low, but students were better able to recommend further treatment for a patient with acute cholecystitis after completing the third-year surgical clerkship. Our study reveals areas where surgical educators can improve medical student ability to accurately diagnose acute cholecystitis and evaluate acute abdominal processes. ( J Surg Ed 72:1045C 2015 Association of Program Directors in Surgery. 1051. J Published by Elsevier Inc. All rights reserved.)
KEY WORDS: medical student, surgery clerkship, stand-
ardized patient, abdominal pain, regional campus COMPETENCIES: Patient Care, Medical Knowledge, Interpersonal and Communication Skills
INTRODUCTION Undergraduate medical education addresses 6 core competencies: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills,
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 1045 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2015.04.019
professionalism, and systems-based practice. In medical school, progress in each of these competencies is evaluated in several ways. Medical students must pass written examinations, clinical clerkships, and simulated patient experiences, which are critically important for providing a structured environment for students to demonstrate their ability to integrate the various core competencies into practice.1 The third-year surgical clerkship is the primary opportunity for medical students to learn about surgical disease and treatment in medical school. Previous studies have used an Observed Structured Clinical Examination (OSCE) to evaluate clinical and patient interaction skills and general knowledge competencies of third-year students during their required surgical clerkships2-8; the OSCE has been proven to be a valid method to assess the 6 core competencies outlined by the Accreditation Council for Graduate Medical Education in surgery.2 Student OSCE performance varies depending on several key factors, including previous completion of a surgical clerkship, surgical clerkship length, and clerkship content.5-8 Multiple studies have examined the performance of medical students when evaluating simulated general surgical patients, including those with an acute abdominal pain. Student performance has been compared based on exposure to type of patients,6 number of days on service,5 and location of the clerkship6 with little difference noted in the performance with a simulated patient. This contrasts with the study by Lind et al.,8 which examined the OSCE scores of 487 students and demonstrated that OSCE performance (particularly in an abdominal pain scenario) was higher in students who completed an 8-week surgical clerkship compared with those who completed a 6-week clerkship.8 The surgical education literature, however, does not include investigations into whether individual surgical clerkship experience predicts superior performance with a surgical patient in an OSCE environment to diagnose cholecystitis or to recommend appropriate treatment. In this study, we sought to characterize how third-year students did in evaluating and managing a standardized case of the acute abdominal pain. For this, we used a checklist of expected history taking, physical examination, clinical reasoning behaviors, standardized patient (SP) feedback, and interstation exercises. We aimed to identify the specific ways that would improve the ability of students who had completed the third-year surgery clerkship to: (1) diagnose acute cholecystitis, and (2) recognize the further need for hospital admission and treatment.
MATERIAL AND METHODS Clinical Scenario At the University of California, San Francisco (UCSF) School of Medicine in the Fall of 2012, the faculty of the UCSF Surgery and Medical Education departments designed a new 1046
miniClinical Performance Examination (mini-CPX) case, structured similarly to other OSCE exercises used across all 4 years of medical education at UCSF as well as in other U.S. medical schools. The scenario involved a patient with acute cholecystitis, which (1) commonly requires surgical intervention, (2) is addressed in the UCSF medical school curriculum, and (3) is associated with a physical examination finding (Murphy’s sign) that the SPs could reproduce on physical examination. Biliary tract disease was introduced in the first-year gastroenterology curriculum at UCSF, and sickle cell disease was discussed in the second-year hematology course. The management objective for the student was to recognize 3 abnormalities at presentation—fever, leukocytosis, and a Murphy’s sign—and recommend admission to either the hospital or emergency department. The scenario of a man with upper abdominal pain prompts a broad differential diagnosis, and we designed the case particularly to challenge students to rule out another common etiology such as peptic ulcer disease (PUD). Additional responses from the SP would clearly have shown a history of a normal endoscopy, negative helicobacter serology, and no improvement after oral H2 blocker therapy, thereby excluding PUD. The 4 SPs used in the mini-CPX were either African or Asian (sickle cell disease occurs in both but is less common in Asian populations) and were provided a script of responses to student inquiries. SPs were prepared with 20 hours of training and practice before the student exercise and were trained to reproduce a positive Murphy’s sign on examination. A total of 173 third-year UCSF medical students participated in an orientation to the mini-CPX and then completed a 17-minute interaction with the SPs as part of the mini-CPX. After the exercise, students underwent a debriefing session where the key teaching points and differential diagnosis were reviewed. The costs of designing and implementing the acute abdomen case were minimal and included training SPs, the percent time and effort of the physician educators, and the School of Medicine resources to house the exercise and assist in the data analysis. Performance Evaluation SPs used a checklist to evaluate medical student performance during the mini-CPX. This checklist contained 30 binary (“yes” or “no”) items; 7 of these items pertained to the history taking (e.g., did the student ask the SP about pain after eating or relevant past surgeries), and 23 of these items pertained to physical examination, information sharing, or communication techniques (e.g., did the student press for a Murphy’s sign or verbally express empathy). Following the patient encounter, the SP scored if the student completed the action (“yes”) or did not for “no” on the checklist.
Journal of Surgical Education Volume 72/Number 5 September/October 2015
During written interstation exercises, which followed the student-patient interaction, students logged on to a computer for 10 minutes and were prompted to complete a patient note with pertinent findings, clinical reasoning, and a treatment plan. The student information, SP feedback, and interstation responses were collected for the acute abdomen case and deidentified to ensure student confidentiality. Statistical Analysis To test the efficacy of the third-year surgical clerkship, students were divided into 2 categories based on whether they had or had not completed a third-year surgical clerkship. A secondary analysis was performed to distinguish those students who had completed their clerkship at (a) any of the following 7 Bay Area teaching sites: (1) UCSF Medical Center at Parnassus, (2) San Francisco General Hospital, (3) San Francisco Veterans Affairs Medical Center, (4) California Pacific Medical Center, (5) UCSF Medical Center at Mount Zion, (6) Kaiser Oakland, and (7) Alameda County Medical Center; or (b) the regional clerkship site at UCSF-Fresno. At each clinical site, students attended a faculty-led seminar on biliary tract disease, which focuses on the core concepts in the management of acute cholecystitis. Data from all 173 medical students were used for both analyses. Descriptive statistics was used to summarize medical students’ mini-CPX performance measures, including history taking, physical examination, information sharing, differential diagnosis, and patient assessment and plan. Binomial proportion z tests were used to compare miniCPX performance measures between students who had completed a third-year surgical clerkship and those who had not. This statistical procedure was also used to compare students who had completed their clerkship at a Bay Area or
regional clerkship site. Statistical significance was evaluated using chi-square tests. All statistical tests were performed with Statistical Analysis System (SAS) 9.4, using an α ¼ 0.05. The UCSF Committee of Human Research approved the study.
RESULTS Overall student performance as assessed by SPs and through the interstation activities is summarized in the Figure. Of the 173 students, 71.1% explained that acute cholecystitis or another process involving the gallbladder (either cholelithiasis, choledocholithiasis, or cholangitis) might be causing the patient symptoms, but only 41.6% explained to the SP that sickle cell disease might be contributing. Patient admission to the hospital or emergency department was recommended by 44.5% of students. Only 11.0% asked to perform a rectal examination (to which the students would have been directed to printed results available in the drawer). Approximately one-third (35.3%) of the students documented a positive Murphy’s sign, but only 6.4% recognized the triad of leukocytosis, fever, and a Murphy’s sign as diagnostic of acute cholecystitis and the need for surgical intervention. The differential diagnoses generated by the students in their interstation exercises (Table 1) were as follows: 42.8% correctly listed cholecystitis as the leading diagnosis, and 57.3% included it in their top 3 differential diagnoses. Approximately half (55.5%) of the students determined that choledocholithiasis, cholelithiasis, or cholangitis might be the cause of the patient’s symptoms, though these were all incorrect diagnoses. A total of 46% of the students documented that the patient’s condition might relate to the underlying sickle cell disease, though most of these students believed that the patient was experiencing a sickle cell crisis or other unrelated condition. A hepatic process,
80 70 60 50 40 30 20 10 0
Recommended Admission to ER or Hospital
Shared with SP possible Explained that Sickle diagnosis of acute Cell Disease may be cholecystitis or related to current gallbladder symptoms inflammation
Asked to perform rectal Documented a positive Documented the classic exam Murphy's sign triad of acute cholecystitis
FIGURE. Percentage of correct responses during the mini-CPX from the cohort of 173 third-year medical students. ER, emergency room. Journal of Surgical Education Volume 72/Number 5 September/October 2015
1047
TABLE 1. Differential Diagnoses Reported by 173 Medical Students During Interstation Exercises*
Diagnosis Cholecystitis Cholelithiasis/ choledocholithiasis/ cholangitis Complications of sickle cell Hepatic process (infarction, abscess, and inflammation) Pancreatitis Appendicitis Peptic ulcer disease Other
DDx #1, Number (%)
DDx #2 or #3, Number (%)
74 (42.8) 44 (25.4)
25 (14.5) 52 (30.1)
21 (12.1) 17 (9.8)
59 (34.1) 32 (18.5)
13 7 3 6
50 15 30 83
(7.5) (4.0) (1.7) (3.5)
(28.9) (8.7) (17.3) (48.0)
*Listed diagnoses total 4173 owing to students listing multiple diagnoses. DDx #1, #2, and #3 indicate the preferred order of a student's differential diagnosis. DDx #1 is the primary diagnosis on the differential.
pancreatitis, appendicitis, and PUD were the other leading diagnoses, though these were all incorrect—the scenario had been constructed to exclude these alternate diagnoses through additional information the SPs would provide if queried. Table 2 summarizes the SP’s assessment of the 173 students. Student performance in history-taking tasks to characterize the abdominal symptoms ranged from 66% to 91% for all 173 students, with 91% of students inquiring about changes in urination or bowel habits and 66% inquiring about previous surgical history. Student performance on the physical examination indicated that 89% auscultated the abdomen, 73% palpated the abdomen, and 65% palpated in the right upper quadrant, where a Murphy’s sign was demonstrated. But only 35% of students (approximately half of those for whom a Murphy’s sign had been displayed) recognized and documented the importance of this finding. Only 11% of the students requested a rectal examination be performed. Students performed at a high level in displaying empathy and maintaining a respectful tone when interacting with the SP. Medical student performance assessed by the SP across all 30 variables (Table 2), comparing surgical clerkship status, showed that completion of the clerkship had a statistically significant, favorable effect on the following 6 areas: (1) auscultating the lung fields on the patient’s back, (2) recommending admission to the emergency room or hospital, (3) suggesting the correct diagnosis of a gallbladder process, (4) involving the SP in deciding a treatment plan, (5) eliciting and addressing the SP’s concerns about the treatment plan and (6) recommending antibiotics, x-rays or surgery. As the mini-CPX had not previously included a case that required immediate surgical intervention, we do not have direct comparison data to assess overall student 1048
performance. We did compare the scores on a comparable mini-CPX case of an abdominal complaint in a general practice setting, and found that the average history taking, communication skills, and SP overall satisfaction were similar. Physical examination scores were significantly lower, which may reflect the case complexity. In our analysis of the interstation exercises, 120 students (69.4%) correctly recognized the patient to have a fever, and 23 students (13.3%) noted the presence of leukocytosis. Of the 11 students who correctly documented the triad of acute cholecystitis in their interstation exercise, 7 had completed the surgery core clerkship. All 11 of the students who correctly documented this triad of the Murphy’s sign also inquired about alcohol consumption, any associated changes in urination, and bilirubinuria or acholic stools; 9 inquired about the association of the abdominal pain with eating, and 8 palpated the abdomen; 6 students probed the possibility of peptic ulcer symptoms and 8 probed the possibility of sickle cell disease. Our secondary analysis of the results for medical student performance across all 30 variables as a function of clerkship location (Bay Area vs regional) revealed no statistically significant difference in student performance. All 10 students at the regional medical center site (with a rural patient population) suggested the correct diagnosis to the patient, compared with 72% of the 91 students at the 6 Bay Area urban sites. Fewer students at the regional site recognized that sickle cell disease may have contributed to the patient’s presentation.
DISCUSSION Our study shows that most students had difficulty recognizing and documenting the key objective clinical findings of acute cholecystitis in interstation activities regardless of whether they had completed their third-year surgery clerkship. Students who had completed the surgical clerkship were more likely to suggest a possible acute process involving the gallbladder to the SP. More than half of the students did not perform key physical examination and counseling items for a patient with an acute abdominal pain. Completion of the third-year clerkship did not improve performance in the following areas: (1) inquiring about surgical history, (2) recognizing a Murphy’s sign, (3) recognizing the relationship between sickle cell disease and pigment stone formation, or (4) asking to perform a rectal examination. These findings may be specific areas for improvement in the surgical curriculum locally and perhaps nationally, although it is unclear if the underlying problem relates to content or content delivery. Our secondary analysis shows that students who completed their surgical clerkship were better able to recognize the acuity of patient illness, the need for hospital admission, and further assessment and treatment. These findings suggest that surgical educators should study further whether
Journal of Surgical Education Volume 72/Number 5 September/October 2015
TABLE 2. Mini-CPX Response Frequencies of 173 Medical Students Characteristics*
All Students, N ¼ 173 (%)
The student asked if the standardized patient (SP) … Had pain while eating 122 (70.5) Had an endoscopy or upper GI tract 120 (69.4) test or H. pylori test Had changes in urination and color of 158 (91.3) either urine or bowel movements Had any complications with sickle cell 134 (77.5) disease Had any past surgeries 115 (66.5) Consumes alcohol 150 (86.7) The student … Suggested the SP has acute 123 (71.1) cholecystitis or acute gallbladder process Recommended x-rays or antibiotics or 109 (63.0) surgery Involved the SP in deciding on a 126 (72.8) treatment plan Elicited and addressed the SP’s 110 (63.6) concerns about the plan Listed to the SP’s lungs on his/her back 59 (34.1) Recommended the SP be admitted to 77 (44.5) ER or hospital Tapped on the SP’s liver 91 (52.6) Listened to the SP’s abdomen 154 (89.0) Pressed on the SP’s abdomen 127 (73.4) Pressed for Murphy’s sign 113 (65.3) Documented a positive Murphy’s sign 61 (35.3) during interstation session Listened to the SP’s heart in 4 places 51 (29.5) Asked to perform a rectal examination 19 (11.0) Explained sickle cell disease may have 72 (41.6) contributed to the SP’s condition Made a personal connection with the 140 (80.9) SP Verbally expressed empathy 173 (100)
Preclerkship, Postclerkship, N ¼ 72 (%) N ¼ 101 (%)
Odds Ratio (95% CI)†
p Value‡
47 (65.3) 53 (73.6)
75 (74.3) 67 (66.3)
1.53 (0.79-2.97) 0.71 (0.36-1.38)
0.20 0.31
63 (87.5)
95 (94.1)
2.26 (0.77-6.67)
0.13
55 (76.4)
79 (78.2)
1.11 (0.54-2.28)
0.78
49 (68.1) 62 (86.1)
66 (65.3) 88 (87.1)
0.89 (0.47-1.68) 1.09 (0.45-2.65)
0.71 0.85
41 (56.9)
82 (81.2)
3.26 (1.65-6.46) o0.001§
37 (51.4)
72 (71.3)
2.35 (1.25-4.42) o0.01§
45 (62.5)
81 (80.2)
2.43 (1.23-4.81) o0.01§
36 (50.0)
74 (73.3)
2.74 (1.45-5.19) o0.01§
17 (23.6) 25 (34.7)
42 (41.6) 52 (51.5)
2.30 (1.18-4.51) 2.00 (1.07-3.72)
0.01§ 0.03§
39 65 57 43 20
52 89 70 70 41
0.90 0.80 0.59 1.52 1.78
(0.49-1.65) (0.30-2.14) (0.29-1.21) (0.81-2.87) (0.93-3.41)
0.73 0.65 0.15 0.19 0.08
(54.2) (90.3) (79.2) (59.7) (27.8)
(51.5) (88.1) (69.3) (69.3) (40.6)
18 (25.0) 8 (11.1) 32 (44.4)
33 (32.7) 11 (10.9) 40 (39.6)
1.46 (0.74-2.86) 0.98 (0.37-2.57) 0.82 (0.44-1.51)
0.28 0.96 0.52
61 (84.7)
79 (78.2)
0.65 (0.29-1.44)
0.28
72 (100)
101 (100)
-
-
ER, emergency room; GI, gastrointestinal; H. pylori, Helicobacter pylori. *Binomial proportion z tests were used to compare mini-CPX performance measures between the students who completed their surgical clerkship and the students who had not. † Univariate logistic regression analysis comparing students who had completed their surgical clerkship to students who had not. ‡ Chi-square test, α ¼ 0.05. § Statistical significance at α ¼ 0.05.
illness-specific education about patient management may be suboptimal at early stages of training. Regardless of whether someone plans a career in surgery or a different field, it is critical to distinguish which patients require higher levels of care for common general surgical conditions such as the acute abdominal pain. Our results corroborate those from an observational study across 222 medical schools that assessed medical student performance on the OSCE after an 8-week surgical clerkship and found no significant difference between OSCE scores of medical students who had completed their surgical clerkship at an academic site compared to those at a community site.6 Our results suggest comparable educational medical student experiences between metropolitan and regional clerkship sites even in a challenging surgical
case, underscoring the value of continued equivalence of education in regional settings. Students who completed the clerkship were more likely to auscultate the lung fields on the back of the simulated patients, a task that is included in the first- and second-year medical student physical examination exercises, but is not emphasized in the surgery clerkship orientation. It is unclear whether a site director or single educator may have played a role in the difference we observed. York et al.9 assessed thirdyear medical students’ ability to perform physical examinations at the beginning and end of their surgical clerkship. Performance of back, breast, and thyroid examinations significantly improved over the duration of the clerkship,9 but abdominal examination skills did not.9 It is unclear whether the difference we observed in lung auscultation is
Journal of Surgical Education Volume 72/Number 5 September/October 2015
1049
related to what York observed in the back examination, which may have included both the spine and lung examinations. The performance of the 7 students who recognized the triad of acute cholecystitis is noteworthy, as is the surprisingly low number of students (61) who identified the Murphy’s sign on examination. Curiously, some students documented a Murphy’s sign without palpating the right upper quadrant; this may be an indication that some students struggled with time management, or that there may be a disconnection between what occurred in the patient encounter and the student documentation of the interaction. Another study identified discrepancies between medical student data gathering and their documentation of an SP encounter; this finding suggests the need for curricular attention to accurate documentation as a component of professional behavior.10 An understanding of hemolysis in the formation of pigment stones was not absolutely necessary to make the final diagnosis. The final test of student thinking would have been to recognize the use of further radiologic tests such as either a hepatobiliary iminodiacetic acid scan or right upper quadrant ultrasound, which would lead to the recommendation for admission to the hospital for further testing or surgery. Rather than attempting to focus upon the content of a specific illness, a better foundation for students may be learning to recognize illness acuity. Our study has a number of limitations. We conducted the study at a single medical school, which likely limits generalizability. We used a single case to assess students’ surgical skills. The nature and stress of the testing environment may have confused some students despite our orientation; although it is clearly stated that the simulated scenario takes place in an outpatient clinic, it is feasible that some students may not have understood the resources of that clinic, and believed that the clinical setting was capable of providing inpatient care. Sickle cell disease in this scenario, which as a disease itself is uncommon, was not intended to distract the students but rather provide a recognized risk factor for the formation of pigment gallstones (the prevalence of cholelithiasis in sickle cell patients is between 34% and 70% in the United States).11-13 Although only 1 UCSF clerkship site (the San Francisco General Hospital) has a designated sickle cell clinic, sickle cell patients can be treated at any of the Bay Area or regional hospitals. Recognizing the relationship between sickle cell disease, hemolysis, and pigment stone formation was not necessary to correctly diagnose acute cholecystitis, and would have indicated superior performance beyond our expectations for passing performance of the student’s synthesis of the case. In future exercises, superior student performance may be achieved by recognizing the merits of hematology consultation. Finally, there may have been insufficient power to determine a difference in student performance at the Bay Area and regional campuses. Despite these limitations, our study suggests 3 areas of improvement for surgical clerkship educators; principally, that the clerkship curriculum should focus on teaching core 1050
concepts in general surgery such as physical examination skills and the assessment and management of patients with acute abdominal pain. The student performance we observed may reflect deficiencies in medical student education before the surgery rotation, the quality of teaching during the clerkship, or the challenge for students to integrate their knowledge and skills with a complex patient in a time-limited situation. The current practice of exposing medical students to diverse subspeciality rotations may limit their understanding of core principles of general surgery8 and students may fail to correctly examine a patient who is in need of further diagnostic tests or emergent surgery. Second, students may benefit from further teaching regarding the information they use in clinical reasoning. Students may rely on patient demographics more than history and physical methods to reach a diagnosis, as our patient was demographically atypical for both acute cholecystitis and sickle cell disease. Third, it may benefit students to focus on time management in clinical encounters, and continually improve their ability to conduct a comprehensive, yet focused history and physical examinations. As many clinicians regularly see patients in 15-minute clinic appointments, effective time management is an important skill, which should be demonstrated and practiced during clerkship training. Surgical education can likely be strengthened by increasing opportunities for clinical reasoning through a combination of paper cases, computer-based simulations, and interactions with SPs and actual patients.
CONCLUSIONS We conclude that medical student abilities to diagnose acute cholecystitis and recognize the need for hospital admission were improved after completing the third-year clerkship, but that these abilities should be strengthened further. Educators who design third-year medical student surgical education could focus first on teaching students to accurately diagnose acute cholecystitis and evaluate acute abdominal processes, and to further recognize which patients are ill and warrant further evaluation before discharge home.
ACKNOWLEDGMENTS The authors thank Mark Lovett in the UCSF Office of Medical Education for database support; Anne Lazar for statistical analysis; Brian George, MD, for design of the study; and Pamela Derish in the UCSF Department of Surgery for editing advice.
REFERENCES 1. Chang A, Boscardin C, Chou CL, Loeser H, Hauer KE.
Predicting failing performance on a standardized patient
Journal of Surgical Education Volume 72/Number 5 September/October 2015
clinical performance examination: the importance of deficits. Acad Med. 2009;84(10):101-104. 2. Sidhu RS, Grober ED, Musselman LJ, Reznick RK.
Assessing competency in surgery: where to begin? Surgery. 2004;135(1):6-20. 3. Tervo RC, Dimitrievich E, Trujillo AL, Whittle K,
Redinius P, Wellman L. The Objective Structured Clinical Examination (OSCE) in the clinical clerkship: an overview. S D J Med. 1997;50(5):153-156. 4. Falcone JL, Schenarts KD, Ferson PF, Day HD. Using
elements from an acute abdominal pain Objective Structured Clinical Examination (OSCE) leads to more standardized grading in the surgical clerkship for third-year medical students. J Surg Educ. 2011;68(5): 408-413. 5. Falcone JL, Watson GA. Differential diagnosis in a
3-station acute abdominal pain Objective Structured Clinical Examination (OSCE): a needs assessment in third-year medical student performance and summative evaluation in the surgical clerkship. J Surg Educ. 2011;68(4):266-269. 6. Williams M, Ambrose M, Carlin AM, Tyburski JG,
Steffes CP. Evaluation of academic and community surgery clerkships at a Midwestern medical school. J Surg Educ. 2004;116(1):11-13.
7. Al-Naami MY. Reliability, validity, and feasibility of
the Objective Structured Clinical Examination in assessing clinical skills of final year surgical clerkship. Saudi Med J. 2008;29(12):1802-1807. 8. Lind DS, Marum T, Ledbetter D, Flynn TC, Romrell LJ,
Copeland EM. The effect of the duration and structure of a surgery clerkship on student performance. J Surg Educ. 1999;84(1):106-111. 9. York NL, Niehaus AH, Markwell SJ, Folse JR.
Evaluation of students’ physical examination skills during their surgery clerkship. Am J Surg. 1999;177(3): 240-243. 10. Szauter KM, Ainsworth MA, Holden MD, Mercado AC.
Do students do what they write and write what they do? The match between the patient encounter and patient note Acad Med. 2006;81(10):S44-S47. 11. Barrett-Connor E. Cholelithiasis in sickle cell anemia.
Am J Med. 1968;45(8):89-98. 12. Stephens CG, Scott RB. Cholelithiasis in sickle cell
anemia: surgical or medical management. Arch Intern Med. 1980;140:648-651. 13. Cameron JL, Maddrey WC, Zuidema GD. Biliary
tract disease in sickle cell anemia: surgical considerations. Ann Surg. 1971;174:702-710.
Journal of Surgical Education Volume 72/Number 5 September/October 2015
1051
The Surgical Clerkship and Medical Student Performance in a Standardized Patient Case of Acute Cholecystitis Eric Low, MPH,* Robert Tessler, MD,† Karen E. Hauer, MD,‡ Andrew D. Leavitt, MD,‡,§ Bernie Miller, BA,* and John Maa, MD, FACS† School of Medicine, University of California, San Francisco, California; †Department of Surgery, University of California, San Francisco, California; ‡Department of Medicine, University of California, San Francisco, California; and §Department Laboratory Medicine, University of California, San Francisco, California *
BACKGROUND: Although an Observed Structured Clinical Examination (OSCE) has been used to evaluate patient interaction and general knowledge competencies of thirdyear students during their required surgical clerkships, whether surgical clerkship experience predicts satisfactory performance with a surgical patient in an OSCE environment has not been investigated. OBJECTIVE: We hypothesized that completion of the
third-year surgery clerkship would improve student ability to diagnose acute cholecystitis and recognize the further need for hospital admission and treatment. DESIGN: An observational study design was used to determine student skills in evaluating a simulated surgical patient with abdominal pain from acute cholecystitis. The skills included key data gathering, physical examination, and information-sharing tasks. SETTING: Tertiary care academic medical center. PARTICIPANTS: Performance was compared between a cohort of 101 medical students who had completed the third-year surgical clerkship and 72 who had not. A secondary analysis compared performance for 91 students who had completed their third-year clerkship in sites near the University of California, San Francisco School of Medicine, and 10 who did so at a regional campus geographically distant from the medical school. RESULTS: Of the 173 students who participated in the
OSCE, only 42% correctly identified the diagnosis of acute
Disclaimers: This study was performed in compliance with our institutional Ethical Board and/or Committee. This work was presented at the AAMC Group on Regional Medical Campuses meeting in Philadelphia in 2013. Correspondence: Inquiries to John Maa, MD, FACS, University of California, Office of the President, 300 Lakeside Drive, 6th Floor, Oakland, CA 94612-3550 and Marin General Hospital; E-mail: [email protected], [email protected]
cholecystitis, though 71% did suggest the possibility of a biliary process to the standardized patient. Most of the students who identified the condition as acute cholecystitis or gallbladder-related process had completed their third-year surgical clerkship (odds ratio [OR] ¼ 3.26). Students who completed their surgical clerkship were also better able to recommend appropriate treatment for the patient (OR ¼ 2.35), and recommend admission to the hospital or emergency department (OR ¼ 2.00). Approximately one-third (35.3%) of all students documented a positive Murphy’s sign, but only 6.4% identified the triad of leukocytosis, fever, and a Murphy’s sign as diagnostic of acute cholecystitis and the need for surgical intervention. Student performance on the clinical examination did not differ depending on whether the students completed their clerkship at a Bay Area or regional hospital. CONCLUSIONS: Student recognition of the key physical
examination and laboratory findings diagnostic of acute cholecystitis was low, but students were better able to recommend further treatment for a patient with acute cholecystitis after completing the third-year surgical clerkship. Our study reveals areas where surgical educators can improve medical student ability to accurately diagnose acute cholecystitis and evaluate acute abdominal processes. ( J Surg Ed 72:1045C 2015 Association of Program Directors in Surgery. 1051. J Published by Elsevier Inc. All rights reserved.)
KEY WORDS: medical student, surgery clerkship, stand-
ardized patient, abdominal pain, regional campus COMPETENCIES: Patient Care, Medical Knowledge, Interpersonal and Communication Skills
INTRODUCTION Undergraduate medical education addresses 6 core competencies: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills,
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 1045 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2015.04.019
professionalism, and systems-based practice. In medical school, progress in each of these competencies is evaluated in several ways. Medical students must pass written examinations, clinical clerkships, and simulated patient experiences, which are critically important for providing a structured environment for students to demonstrate their ability to integrate the various core competencies into practice.1 The third-year surgical clerkship is the primary opportunity for medical students to learn about surgical disease and treatment in medical school. Previous studies have used an Observed Structured Clinical Examination (OSCE) to evaluate clinical and patient interaction skills and general knowledge competencies of third-year students during their required surgical clerkships2-8; the OSCE has been proven to be a valid method to assess the 6 core competencies outlined by the Accreditation Council for Graduate Medical Education in surgery.2 Student OSCE performance varies depending on several key factors, including previous completion of a surgical clerkship, surgical clerkship length, and clerkship content.5-8 Multiple studies have examined the performance of medical students when evaluating simulated general surgical patients, including those with an acute abdominal pain. Student performance has been compared based on exposure to type of patients,6 number of days on service,5 and location of the clerkship6 with little difference noted in the performance with a simulated patient. This contrasts with the study by Lind et al.,8 which examined the OSCE scores of 487 students and demonstrated that OSCE performance (particularly in an abdominal pain scenario) was higher in students who completed an 8-week surgical clerkship compared with those who completed a 6-week clerkship.8 The surgical education literature, however, does not include investigations into whether individual surgical clerkship experience predicts superior performance with a surgical patient in an OSCE environment to diagnose cholecystitis or to recommend appropriate treatment. In this study, we sought to characterize how third-year students did in evaluating and managing a standardized case of the acute abdominal pain. For this, we used a checklist of expected history taking, physical examination, clinical reasoning behaviors, standardized patient (SP) feedback, and interstation exercises. We aimed to identify the specific ways that would improve the ability of students who had completed the third-year surgery clerkship to: (1) diagnose acute cholecystitis, and (2) recognize the further need for hospital admission and treatment.
MATERIAL AND METHODS Clinical Scenario At the University of California, San Francisco (UCSF) School of Medicine in the Fall of 2012, the faculty of the UCSF Surgery and Medical Education departments designed a new 1046
miniClinical Performance Examination (mini-CPX) case, structured similarly to other OSCE exercises used across all 4 years of medical education at UCSF as well as in other U.S. medical schools. The scenario involved a patient with acute cholecystitis, which (1) commonly requires surgical intervention, (2) is addressed in the UCSF medical school curriculum, and (3) is associated with a physical examination finding (Murphy’s sign) that the SPs could reproduce on physical examination. Biliary tract disease was introduced in the first-year gastroenterology curriculum at UCSF, and sickle cell disease was discussed in the second-year hematology course. The management objective for the student was to recognize 3 abnormalities at presentation—fever, leukocytosis, and a Murphy’s sign—and recommend admission to either the hospital or emergency department. The scenario of a man with upper abdominal pain prompts a broad differential diagnosis, and we designed the case particularly to challenge students to rule out another common etiology such as peptic ulcer disease (PUD). Additional responses from the SP would clearly have shown a history of a normal endoscopy, negative helicobacter serology, and no improvement after oral H2 blocker therapy, thereby excluding PUD. The 4 SPs used in the mini-CPX were either African or Asian (sickle cell disease occurs in both but is less common in Asian populations) and were provided a script of responses to student inquiries. SPs were prepared with 20 hours of training and practice before the student exercise and were trained to reproduce a positive Murphy’s sign on examination. A total of 173 third-year UCSF medical students participated in an orientation to the mini-CPX and then completed a 17-minute interaction with the SPs as part of the mini-CPX. After the exercise, students underwent a debriefing session where the key teaching points and differential diagnosis were reviewed. The costs of designing and implementing the acute abdomen case were minimal and included training SPs, the percent time and effort of the physician educators, and the School of Medicine resources to house the exercise and assist in the data analysis. Performance Evaluation SPs used a checklist to evaluate medical student performance during the mini-CPX. This checklist contained 30 binary (“yes” or “no”) items; 7 of these items pertained to the history taking (e.g., did the student ask the SP about pain after eating or relevant past surgeries), and 23 of these items pertained to physical examination, information sharing, or communication techniques (e.g., did the student press for a Murphy’s sign or verbally express empathy). Following the patient encounter, the SP scored if the student completed the action (“yes”) or did not for “no” on the checklist.
Journal of Surgical Education Volume 72/Number 5 September/October 2015
During written interstation exercises, which followed the student-patient interaction, students logged on to a computer for 10 minutes and were prompted to complete a patient note with pertinent findings, clinical reasoning, and a treatment plan. The student information, SP feedback, and interstation responses were collected for the acute abdomen case and deidentified to ensure student confidentiality. Statistical Analysis To test the efficacy of the third-year surgical clerkship, students were divided into 2 categories based on whether they had or had not completed a third-year surgical clerkship. A secondary analysis was performed to distinguish those students who had completed their clerkship at (a) any of the following 7 Bay Area teaching sites: (1) UCSF Medical Center at Parnassus, (2) San Francisco General Hospital, (3) San Francisco Veterans Affairs Medical Center, (4) California Pacific Medical Center, (5) UCSF Medical Center at Mount Zion, (6) Kaiser Oakland, and (7) Alameda County Medical Center; or (b) the regional clerkship site at UCSF-Fresno. At each clinical site, students attended a faculty-led seminar on biliary tract disease, which focuses on the core concepts in the management of acute cholecystitis. Data from all 173 medical students were used for both analyses. Descriptive statistics was used to summarize medical students’ mini-CPX performance measures, including history taking, physical examination, information sharing, differential diagnosis, and patient assessment and plan. Binomial proportion z tests were used to compare miniCPX performance measures between students who had completed a third-year surgical clerkship and those who had not. This statistical procedure was also used to compare students who had completed their clerkship at a Bay Area or
regional clerkship site. Statistical significance was evaluated using chi-square tests. All statistical tests were performed with Statistical Analysis System (SAS) 9.4, using an α ¼ 0.05. The UCSF Committee of Human Research approved the study.
RESULTS Overall student performance as assessed by SPs and through the interstation activities is summarized in the Figure. Of the 173 students, 71.1% explained that acute cholecystitis or another process involving the gallbladder (either cholelithiasis, choledocholithiasis, or cholangitis) might be causing the patient symptoms, but only 41.6% explained to the SP that sickle cell disease might be contributing. Patient admission to the hospital or emergency department was recommended by 44.5% of students. Only 11.0% asked to perform a rectal examination (to which the students would have been directed to printed results available in the drawer). Approximately one-third (35.3%) of the students documented a positive Murphy’s sign, but only 6.4% recognized the triad of leukocytosis, fever, and a Murphy’s sign as diagnostic of acute cholecystitis and the need for surgical intervention. The differential diagnoses generated by the students in their interstation exercises (Table 1) were as follows: 42.8% correctly listed cholecystitis as the leading diagnosis, and 57.3% included it in their top 3 differential diagnoses. Approximately half (55.5%) of the students determined that choledocholithiasis, cholelithiasis, or cholangitis might be the cause of the patient’s symptoms, though these were all incorrect diagnoses. A total of 46% of the students documented that the patient’s condition might relate to the underlying sickle cell disease, though most of these students believed that the patient was experiencing a sickle cell crisis or other unrelated condition. A hepatic process,
80 70 60 50 40 30 20 10 0
Recommended Admission to ER or Hospital
Shared with SP possible Explained that Sickle diagnosis of acute Cell Disease may be cholecystitis or related to current gallbladder symptoms inflammation
Asked to perform rectal Documented a positive Documented the classic exam Murphy's sign triad of acute cholecystitis
FIGURE. Percentage of correct responses during the mini-CPX from the cohort of 173 third-year medical students. ER, emergency room. Journal of Surgical Education Volume 72/Number 5 September/October 2015
1047
TABLE 1. Differential Diagnoses Reported by 173 Medical Students During Interstation Exercises*
Diagnosis Cholecystitis Cholelithiasis/ choledocholithiasis/ cholangitis Complications of sickle cell Hepatic process (infarction, abscess, and inflammation) Pancreatitis Appendicitis Peptic ulcer disease Other
DDx #1, Number (%)
DDx #2 or #3, Number (%)
74 (42.8) 44 (25.4)
25 (14.5) 52 (30.1)
21 (12.1) 17 (9.8)
59 (34.1) 32 (18.5)
13 7 3 6
50 15 30 83
(7.5) (4.0) (1.7) (3.5)
(28.9) (8.7) (17.3) (48.0)
*Listed diagnoses total 4173 owing to students listing multiple diagnoses. DDx #1, #2, and #3 indicate the preferred order of a student's differential diagnosis. DDx #1 is the primary diagnosis on the differential.
pancreatitis, appendicitis, and PUD were the other leading diagnoses, though these were all incorrect—the scenario had been constructed to exclude these alternate diagnoses through additional information the SPs would provide if queried. Table 2 summarizes the SP’s assessment of the 173 students. Student performance in history-taking tasks to characterize the abdominal symptoms ranged from 66% to 91% for all 173 students, with 91% of students inquiring about changes in urination or bowel habits and 66% inquiring about previous surgical history. Student performance on the physical examination indicated that 89% auscultated the abdomen, 73% palpated the abdomen, and 65% palpated in the right upper quadrant, where a Murphy’s sign was demonstrated. But only 35% of students (approximately half of those for whom a Murphy’s sign had been displayed) recognized and documented the importance of this finding. Only 11% of the students requested a rectal examination be performed. Students performed at a high level in displaying empathy and maintaining a respectful tone when interacting with the SP. Medical student performance assessed by the SP across all 30 variables (Table 2), comparing surgical clerkship status, showed that completion of the clerkship had a statistically significant, favorable effect on the following 6 areas: (1) auscultating the lung fields on the patient’s back, (2) recommending admission to the emergency room or hospital, (3) suggesting the correct diagnosis of a gallbladder process, (4) involving the SP in deciding a treatment plan, (5) eliciting and addressing the SP’s concerns about the treatment plan and (6) recommending antibiotics, x-rays or surgery. As the mini-CPX had not previously included a case that required immediate surgical intervention, we do not have direct comparison data to assess overall student 1048
performance. We did compare the scores on a comparable mini-CPX case of an abdominal complaint in a general practice setting, and found that the average history taking, communication skills, and SP overall satisfaction were similar. Physical examination scores were significantly lower, which may reflect the case complexity. In our analysis of the interstation exercises, 120 students (69.4%) correctly recognized the patient to have a fever, and 23 students (13.3%) noted the presence of leukocytosis. Of the 11 students who correctly documented the triad of acute cholecystitis in their interstation exercise, 7 had completed the surgery core clerkship. All 11 of the students who correctly documented this triad of the Murphy’s sign also inquired about alcohol consumption, any associated changes in urination, and bilirubinuria or acholic stools; 9 inquired about the association of the abdominal pain with eating, and 8 palpated the abdomen; 6 students probed the possibility of peptic ulcer symptoms and 8 probed the possibility of sickle cell disease. Our secondary analysis of the results for medical student performance across all 30 variables as a function of clerkship location (Bay Area vs regional) revealed no statistically significant difference in student performance. All 10 students at the regional medical center site (with a rural patient population) suggested the correct diagnosis to the patient, compared with 72% of the 91 students at the 6 Bay Area urban sites. Fewer students at the regional site recognized that sickle cell disease may have contributed to the patient’s presentation.
DISCUSSION Our study shows that most students had difficulty recognizing and documenting the key objective clinical findings of acute cholecystitis in interstation activities regardless of whether they had completed their third-year surgery clerkship. Students who had completed the surgical clerkship were more likely to suggest a possible acute process involving the gallbladder to the SP. More than half of the students did not perform key physical examination and counseling items for a patient with an acute abdominal pain. Completion of the third-year clerkship did not improve performance in the following areas: (1) inquiring about surgical history, (2) recognizing a Murphy’s sign, (3) recognizing the relationship between sickle cell disease and pigment stone formation, or (4) asking to perform a rectal examination. These findings may be specific areas for improvement in the surgical curriculum locally and perhaps nationally, although it is unclear if the underlying problem relates to content or content delivery. Our secondary analysis shows that students who completed their surgical clerkship were better able to recognize the acuity of patient illness, the need for hospital admission, and further assessment and treatment. These findings suggest that surgical educators should study further whether
Journal of Surgical Education Volume 72/Number 5 September/October 2015
TABLE 2. Mini-CPX Response Frequencies of 173 Medical Students Characteristics*
All Students, N ¼ 173 (%)
The student asked if the standardized patient (SP) … Had pain while eating 122 (70.5) Had an endoscopy or upper GI tract 120 (69.4) test or H. pylori test Had changes in urination and color of 158 (91.3) either urine or bowel movements Had any complications with sickle cell 134 (77.5) disease Had any past surgeries 115 (66.5) Consumes alcohol 150 (86.7) The student … Suggested the SP has acute 123 (71.1) cholecystitis or acute gallbladder process Recommended x-rays or antibiotics or 109 (63.0) surgery Involved the SP in deciding on a 126 (72.8) treatment plan Elicited and addressed the SP’s 110 (63.6) concerns about the plan Listed to the SP’s lungs on his/her back 59 (34.1) Recommended the SP be admitted to 77 (44.5) ER or hospital Tapped on the SP’s liver 91 (52.6) Listened to the SP’s abdomen 154 (89.0) Pressed on the SP’s abdomen 127 (73.4) Pressed for Murphy’s sign 113 (65.3) Documented a positive Murphy’s sign 61 (35.3) during interstation session Listened to the SP’s heart in 4 places 51 (29.5) Asked to perform a rectal examination 19 (11.0) Explained sickle cell disease may have 72 (41.6) contributed to the SP’s condition Made a personal connection with the 140 (80.9) SP Verbally expressed empathy 173 (100)
Preclerkship, Postclerkship, N ¼ 72 (%) N ¼ 101 (%)
Odds Ratio (95% CI)†
p Value‡
47 (65.3) 53 (73.6)
75 (74.3) 67 (66.3)
1.53 (0.79-2.97) 0.71 (0.36-1.38)
0.20 0.31
63 (87.5)
95 (94.1)
2.26 (0.77-6.67)
0.13
55 (76.4)
79 (78.2)
1.11 (0.54-2.28)
0.78
49 (68.1) 62 (86.1)
66 (65.3) 88 (87.1)
0.89 (0.47-1.68) 1.09 (0.45-2.65)
0.71 0.85
41 (56.9)
82 (81.2)
3.26 (1.65-6.46) o0.001§
37 (51.4)
72 (71.3)
2.35 (1.25-4.42) o0.01§
45 (62.5)
81 (80.2)
2.43 (1.23-4.81) o0.01§
36 (50.0)
74 (73.3)
2.74 (1.45-5.19) o0.01§
17 (23.6) 25 (34.7)
42 (41.6) 52 (51.5)
2.30 (1.18-4.51) 2.00 (1.07-3.72)
0.01§ 0.03§
39 65 57 43 20
52 89 70 70 41
0.90 0.80 0.59 1.52 1.78
(0.49-1.65) (0.30-2.14) (0.29-1.21) (0.81-2.87) (0.93-3.41)
0.73 0.65 0.15 0.19 0.08
(54.2) (90.3) (79.2) (59.7) (27.8)
(51.5) (88.1) (69.3) (69.3) (40.6)
18 (25.0) 8 (11.1) 32 (44.4)
33 (32.7) 11 (10.9) 40 (39.6)
1.46 (0.74-2.86) 0.98 (0.37-2.57) 0.82 (0.44-1.51)
0.28 0.96 0.52
61 (84.7)
79 (78.2)
0.65 (0.29-1.44)
0.28
72 (100)
101 (100)
-
-
ER, emergency room; GI, gastrointestinal; H. pylori, Helicobacter pylori. *Binomial proportion z tests were used to compare mini-CPX performance measures between the students who completed their surgical clerkship and the students who had not. † Univariate logistic regression analysis comparing students who had completed their surgical clerkship to students who had not. ‡ Chi-square test, α ¼ 0.05. § Statistical significance at α ¼ 0.05.
illness-specific education about patient management may be suboptimal at early stages of training. Regardless of whether someone plans a career in surgery or a different field, it is critical to distinguish which patients require higher levels of care for common general surgical conditions such as the acute abdominal pain. Our results corroborate those from an observational study across 222 medical schools that assessed medical student performance on the OSCE after an 8-week surgical clerkship and found no significant difference between OSCE scores of medical students who had completed their surgical clerkship at an academic site compared to those at a community site.6 Our results suggest comparable educational medical student experiences between metropolitan and regional clerkship sites even in a challenging surgical
case, underscoring the value of continued equivalence of education in regional settings. Students who completed the clerkship were more likely to auscultate the lung fields on the back of the simulated patients, a task that is included in the first- and second-year medical student physical examination exercises, but is not emphasized in the surgery clerkship orientation. It is unclear whether a site director or single educator may have played a role in the difference we observed. York et al.9 assessed thirdyear medical students’ ability to perform physical examinations at the beginning and end of their surgical clerkship. Performance of back, breast, and thyroid examinations significantly improved over the duration of the clerkship,9 but abdominal examination skills did not.9 It is unclear whether the difference we observed in lung auscultation is
Journal of Surgical Education Volume 72/Number 5 September/October 2015
1049
related to what York observed in the back examination, which may have included both the spine and lung examinations. The performance of the 7 students who recognized the triad of acute cholecystitis is noteworthy, as is the surprisingly low number of students (61) who identified the Murphy’s sign on examination. Curiously, some students documented a Murphy’s sign without palpating the right upper quadrant; this may be an indication that some students struggled with time management, or that there may be a disconnection between what occurred in the patient encounter and the student documentation of the interaction. Another study identified discrepancies between medical student data gathering and their documentation of an SP encounter; this finding suggests the need for curricular attention to accurate documentation as a component of professional behavior.10 An understanding of hemolysis in the formation of pigment stones was not absolutely necessary to make the final diagnosis. The final test of student thinking would have been to recognize the use of further radiologic tests such as either a hepatobiliary iminodiacetic acid scan or right upper quadrant ultrasound, which would lead to the recommendation for admission to the hospital for further testing or surgery. Rather than attempting to focus upon the content of a specific illness, a better foundation for students may be learning to recognize illness acuity. Our study has a number of limitations. We conducted the study at a single medical school, which likely limits generalizability. We used a single case to assess students’ surgical skills. The nature and stress of the testing environment may have confused some students despite our orientation; although it is clearly stated that the simulated scenario takes place in an outpatient clinic, it is feasible that some students may not have understood the resources of that clinic, and believed that the clinical setting was capable of providing inpatient care. Sickle cell disease in this scenario, which as a disease itself is uncommon, was not intended to distract the students but rather provide a recognized risk factor for the formation of pigment gallstones (the prevalence of cholelithiasis in sickle cell patients is between 34% and 70% in the United States).11-13 Although only 1 UCSF clerkship site (the San Francisco General Hospital) has a designated sickle cell clinic, sickle cell patients can be treated at any of the Bay Area or regional hospitals. Recognizing the relationship between sickle cell disease, hemolysis, and pigment stone formation was not necessary to correctly diagnose acute cholecystitis, and would have indicated superior performance beyond our expectations for passing performance of the student’s synthesis of the case. In future exercises, superior student performance may be achieved by recognizing the merits of hematology consultation. Finally, there may have been insufficient power to determine a difference in student performance at the Bay Area and regional campuses. Despite these limitations, our study suggests 3 areas of improvement for surgical clerkship educators; principally, that the clerkship curriculum should focus on teaching core 1050
concepts in general surgery such as physical examination skills and the assessment and management of patients with acute abdominal pain. The student performance we observed may reflect deficiencies in medical student education before the surgery rotation, the quality of teaching during the clerkship, or the challenge for students to integrate their knowledge and skills with a complex patient in a time-limited situation. The current practice of exposing medical students to diverse subspeciality rotations may limit their understanding of core principles of general surgery8 and students may fail to correctly examine a patient who is in need of further diagnostic tests or emergent surgery. Second, students may benefit from further teaching regarding the information they use in clinical reasoning. Students may rely on patient demographics more than history and physical methods to reach a diagnosis, as our patient was demographically atypical for both acute cholecystitis and sickle cell disease. Third, it may benefit students to focus on time management in clinical encounters, and continually improve their ability to conduct a comprehensive, yet focused history and physical examinations. As many clinicians regularly see patients in 15-minute clinic appointments, effective time management is an important skill, which should be demonstrated and practiced during clerkship training. Surgical education can likely be strengthened by increasing opportunities for clinical reasoning through a combination of paper cases, computer-based simulations, and interactions with SPs and actual patients.
CONCLUSIONS We conclude that medical student abilities to diagnose acute cholecystitis and recognize the need for hospital admission were improved after completing the third-year clerkship, but that these abilities should be strengthened further. Educators who design third-year medical student surgical education could focus first on teaching students to accurately diagnose acute cholecystitis and evaluate acute abdominal processes, and to further recognize which patients are ill and warrant further evaluation before discharge home.
ACKNOWLEDGMENTS The authors thank Mark Lovett in the UCSF Office of Medical Education for database support; Anne Lazar for statistical analysis; Brian George, MD, for design of the study; and Pamela Derish in the UCSF Department of Surgery for editing advice.
REFERENCES 1. Chang A, Boscardin C, Chou CL, Loeser H, Hauer KE.
Predicting failing performance on a standardized patient
Journal of Surgical Education Volume 72/Number 5 September/October 2015
clinical performance examination: the importance of deficits. Acad Med. 2009;84(10):101-104. 2. Sidhu RS, Grober ED, Musselman LJ, Reznick RK.
Assessing competency in surgery: where to begin? Surgery. 2004;135(1):6-20. 3. Tervo RC, Dimitrievich E, Trujillo AL, Whittle K,
Redinius P, Wellman L. The Objective Structured Clinical Examination (OSCE) in the clinical clerkship: an overview. S D J Med. 1997;50(5):153-156. 4. Falcone JL, Schenarts KD, Ferson PF, Day HD. Using
elements from an acute abdominal pain Objective Structured Clinical Examination (OSCE) leads to more standardized grading in the surgical clerkship for third-year medical students. J Surg Educ. 2011;68(5): 408-413. 5. Falcone JL, Watson GA. Differential diagnosis in a
3-station acute abdominal pain Objective Structured Clinical Examination (OSCE): a needs assessment in third-year medical student performance and summative evaluation in the surgical clerkship. J Surg Educ. 2011;68(4):266-269. 6. Williams M, Ambrose M, Carlin AM, Tyburski JG,
Steffes CP. Evaluation of academic and community surgery clerkships at a Midwestern medical school. J Surg Educ. 2004;116(1):11-13.
7. Al-Naami MY. Reliability, validity, and feasibility of
the Objective Structured Clinical Examination in assessing clinical skills of final year surgical clerkship. Saudi Med J. 2008;29(12):1802-1807. 8. Lind DS, Marum T, Ledbetter D, Flynn TC, Romrell LJ,
Copeland EM. The effect of the duration and structure of a surgery clerkship on student performance. J Surg Educ. 1999;84(1):106-111. 9. York NL, Niehaus AH, Markwell SJ, Folse JR.
Evaluation of students’ physical examination skills during their surgery clerkship. Am J Surg. 1999;177(3): 240-243. 10. Szauter KM, Ainsworth MA, Holden MD, Mercado AC.
Do students do what they write and write what they do? The match between the patient encounter and patient note Acad Med. 2006;81(10):S44-S47. 11. Barrett-Connor E. Cholelithiasis in sickle cell anemia.
Am J Med. 1968;45(8):89-98. 12. Stephens CG, Scott RB. Cholelithiasis in sickle cell
anemia: surgical or medical management. Arch Intern Med. 1980;140:648-651. 13. Cameron JL, Maddrey WC, Zuidema GD. Biliary
tract disease in sickle cell anemia: surgical considerations. Ann Surg. 1971;174:702-710.
Journal of Surgical Education Volume 72/Number 5 September/October 2015
1051
