Research Report
Standardized Patient Assessment of Medical Student Empathy: Ethnicity and Gender Effects in a Multi-Institutional Study Katherine Berg, MD, MPH, Benjamin Blatt, MD, Joseph Lopreiato, MD, MPH, Julianna Jung, MD, Arielle Schaeffer, Daniel Heil, Tamara Owens, MEd, Pamela L. Carter-Nolan, PhD, MPH, Dale Berg, MD, Jon Veloski, MS, Elizabeth Darby, and Mohammadreza Hojat, PhD
Abstract Purpose To examine, primarily, the effects of ethnicity and gender, which could introduce bias into scoring, on standardized patient (SP) assessments of medical students and, secondarily, to examine medical students’ self-reported empathy for ethnicity and gender effects so as to compare self-perception with the perceptions of SPs. Method Participants were 577 students from four medical schools in 2012: 373 (65%) were white, 79 (14%) black/African American, and 125 (22%) Asian/Pacific Islander. These students were assessed by 84 SPs: 62 (74%) were white and 22 (26%) were black/African American.
E
mpathy is a pillar of the patient– physician relationship. Sir William Osler1 stated that a physician’s work “is arduous and complex, requiring the exercise of the very highest faculties of the mind, while constantly appealing to the emotions and finer feelings.” The importance of empathy in patient care has caused both the Association of American Medical Colleges and the American Board of Internal Medicine to recommend that empathy be cultivated and assessed as an essential medical education outcome.2,3 Empathy in the context of medicine is defined as a predominantly cognitive Please see the end of this article for information about the authors. Correspondence should be addressed to Dr. K. Berg, Sidney Kimmel Medical College at Thomas Jefferson University, University Clinical Skills and Simulation Center, Suite 309E, Hamilton Building, 1001 Locust St., Philadelphia, PA 19107; telephone: (215) 5034234; e-mail: [email protected]. Acad Med. 2015;90:105–111. First published online October 21, 2014 doi: 10.1097/ACM.0000000000000529
Academic Medicine, Vol. 90, No. 1 / January 2015
SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) and the Global Ratings of Empathy tool. Students completed the Jefferson Scale of Empathy and two Interpersonal Reactivity Index subscales. The investigators used 2,882 student–SP encounters in their analyses.
on the JSPPPE than their respective male counterparts. Male black/African American students obtained the lowest SP assessment scores of empathy regardless of SP ethnicity. Black/African American students obtained the highest mean scores on self-reported empathy.
Results Analyses of SPs’ assessments of students’ empathy indicated significant interaction effects of gender and ethnicity. Female students, regardless of ethnicity, obtained significantly higher mean JSPPPE scores than men. Female black/African American, female white, and female Asian/Pacific Islander students scored significantly higher
Conclusions The significant interaction effects of ethnicity and gender in clinical encounters, plus the inconsistencies observed between SPs’ assessments of students’ empathy and students’ self-reported empathy, raise questions about possible ethnicity and gender biases in the SPs’ assessments of medical students’ clinical skills.
attribute, involving an understanding of the experiences, concerns, and perspective of each patient, combined with a capacity to communicate this understanding and an intention to help.4–6 Many medical schools in the United States and elsewhere use standardized patients (SPs) in objective structured clinical examinations (OSCEs) to assess medical students’ clinical competencies, including their empathy. Currently, 94% of U.S. medical schools involve SPs in both clinical training and performance-based assessments, such as OSCEs.7
examinations and of the SP assessment approach in general.9,10
Since 1992, the Consensus Conference on the Use of SPs has recommended the need for studies evaluating how racial, ethnic, and cultural factors affect SP ratings of students’ performances during clinical skills testing.8 OSCEs and U.S. medical licensure exams alike involve SPs in clinical skills assessment; therefore, the possibility of a significant interaction between the SP’s and the student’s ethnicity and/or gender could threaten the validity of these high-stakes
Although at least one study has reported no ethnicity influence in SPs’ assessment scores,11 others have shown significant interaction effects.9,10 Some of us conducted an earlier study and found that Asian American medical students obtained lower empathy ratings from SPs than did white students.12 These findings raised questions about the possible effect ethnicity might have on SP assessments of students (and, more broadly, about bias in testing). That earlier study, however, had several shortcomings. It was from a single institution, involved few African American students as participants, and did not consider the ethnicity and gender of the SPs. The present study was designed to overcome those shortcomings by including four medical schools with more diverse student populations. The primary purpose of this study, therefore, was to examine the effects of ethnicity and gender, which could bias
105
Research Report
scoring, on SP assessments of medical students’ empathy. A secondary purpose was to examine medical students’ self-reported empathy for ethnicity and gender effects so as to compare their self-perceptions with the external perceptions of SPs. Method
Participants Participants included medical students from four medical schools: the George Washington University (GWU), Howard University (HU), Sidney Kimmel Medical College at Thomas Jefferson University (TJU), and the Uniformed Services University of the Health Sciences (USU). Instruments To assess student empathy, the SPs completed two instruments during the OSCE: (1) the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE), and (2) the Global Ratings of Empathy (GRE) tool. The JSPPPE. The JSPPPE contains five items measuring SPs’ perception of an individual student’s empathic engagement. Each item (e.g., “This physician [or student] understands my emotions, feelings, and concerns”) is scored on a seven-point Likert-type scale (1 = strongly disagree, 7 = strongly agree). The score is calculated from the sum of the item scores received from SPs; the higher the score, the more favorably the SPs viewed the student’s empathy. Multiple previous studies have supported the psychometric validity of this scale.13–16 The GRE. The GRE is a two-item tool, completed by the SPs, using a five-point Likert scale (1 = strongly disagree, 5 = strongly agree), after each encounter with students. The two items are “This student understands my perspective” and “The student supports my emotions.” The score, calculated from the sum of the item scores received from all SPs, serves as another measure of student empathy. Again, the higher the score, the more favorably the evaluators, in this case the SPs, view the student’s empathy. Medical students also completed two instruments to assess their own empathy: the Jefferson Scale of Empathy (JSE) and the Interpersonal Reactivity Index (IRI).
106
The JSE. The JSE measures students’ selfassessment of their own empathy. Like the JSPPPE, each item is scored on a sevenpoint Likert-type scale (1 = strongly disagree, 7 = strongly agree). Some items (e.g., “Because people are different, it is difficult to see problems from patients’ perspectives”) are reverse scored. Previous publications have provided extensive psychometric evidence in support of the JSE as a valid instrument to use with medical students.4,17–19 Similar underlying construct and psychometric properties have been reported among dental students,20 nursing students,21 and medical students from several different countries.22–25 The score is calculated from the sum of item scores (possible range 20–140); the higher the score, the more empathic a student assesses him- or herself to be. The IRI. The IRI, developed by M.H. Davis, is a four-subscale (perspectivetaking, empathic concern, fantasy, and personal distress) instrument that has been used to measure empathy in the general population.26 This study used the perspective-taking (as an indicator of cognitive empathy in general context) and empathic concern (as an indicator of effective empathy, or sympathy) subscales—each consisting of seven items. A typical item from the perspective-taking subscale is “I sometimes try to understand my friends better by imagining how things look from their perspective,” and a typical item from the empathic concern subscale is “I often have tender, concerned feelings for people less fortunate than me”). Correlations between the JSE and subscale scores of the IRI have been reported.27 The higher the IRI score, the more empathy the medical students in our study assessed themselves as having. Procedures Students’ clinical skills were assessed at the end of the third year or the beginning of the fourth year of medical school in four schools that are members of the Mid-Atlantic Consortium. All schools used the same six-station OSCE, which was designed to be similar to the United States Medical Licensing Exam Step 2 Clinical Skills assessment.28 The case content was determined by the MidAtlantic Consortium based on the objectives of the medical schools. The cases had been piloted previously and had undergone systematic review to improve
their reliability and validity. The four schools used identical training materials (and three even used the same trainer) to train SPs for their cases. A passing score is required at all institutions. Data were collected at each institution and deidentified prior to being sent to TJU for analysis. The institutional review boards of all four institutions granted this study exempt status. Students and SPs alike signed appropriate consent forms and participated voluntarily. Students were not compensated and were informed that individual responses would be confidential and not part of their academic records. We examined the effects of ethnicity and gender by using data from patient encounters, defined as one SP’s assessment of one medical student in each OSCE station. After each encounter, the SPs assessed students using both the JSPPPE and the GRE. The students took the JSE and IRI the same day they took their OSCE. The unit of analysis was the student–SP encounter. Statistical analysis To analyze students’ self-reported empathy, we used multivariate analysis of variance (MANOVA). Students’ ethnicity and gender were the independent variables, while their scores on the JSE and IRI were the dependent variables (two-way 3 × 2 design). For the patient encounters, we also used MANOVA in a four-way (3 × 2 × 2 × 2) design in which a student’s ethnicity (white, black/ African American, and Asian/Pacific Islander), the SP’s ethnicity (white, black/ African American), the student’s gender (male, female), and the SP’s gender (male, female) were four independent variables. Scores on the JSPPPE and GRE were the dependent variables. We made comparisons simultaneously for all groups classified by ethnicity and gender. When multivariate F ratios were statistically significant, we used univariate analysis of variance for each dependent variable followed by post hoc mean comparisons. The probability of type I (alpha) error was set at .05 to determine the significance of statistical findings. We analyzed the data using the Statistical Analysis System (SAS version 9.3 for Windows, Cary, North Carolina) to determine the mean, standard deviation,
Academic Medicine, Vol. 90, No. 1 / January 2015
Research Report
and number of encounters for each of the groups by student ethnicity and gender and by SP ethnicity and gender.
Table 1 Number and Distribution of Patient Encounters Between Medical Students and Standardized Patients (SPs) by Ethnicity in a Study of the Interaction Effects of Ethnicity and Gender on SP Assessments of Medical Student Empathy, 2012
Results
SPs
Participants Of the 577 medical students who participated, 290 (50%) were men, and 145 (25%) were from GWU, 57 (10%) from HU, 238 (41%) from TJU, and 137 (24%) from USU. The sample consisted of 373 (65%) white students, 79 (14%) black/African American students, and 125 (22%) Asian/Pacific Islander students. Also, 84 SPs participated in the study: 39 (46%) were men; 62 (74%) were white, and 22 (26%) were black/African American (none were Asian/Pacific Islander). Of the 62 white SPs, 30 (48%) were men, and of the 22 black/African American SPs, 9 (41%) were men.
Students White Black/African American Asian/Pacific Islander Total
Total
1,137 256
854 142
1,991 398
343
321
664
1,736
1,317
3,053a
The investigators used 2,882 (94%) of the SP–student encounters in their analyses.
In the multivariate analyses, the interaction of students’ ethnicity and SPs’ ethnicity was statistically significant (P < .05). In the interactions of students’ ethnicity and gender and SPs’ ethnicity and gender, the results were highly significant (P < .001). No significant interaction effect was obtained for students’ gender and SPs’ ethnicity (P = .13). See Table 3. For our interpretation, we emphasized the interactions that we found to be statistically significant (i.e., students’ ethnicity and gender, interacting with SPs’ ethnicity and gender; Table 3). Post hoc comparisons of the mean scores of the JSPPPE and the GRE showed a number of patterns in these four-way interactions. First, in most instances, SPs of all categories rated female students higher than male students. The male student scores, though sometimes equal, never significantly exceeded the scores of female students of any ethnicity. Second, SPs of all categories rated female black/
Patient encounters: Main and interaction effects of ethnicity and gender
Academic Medicine, Vol. 90, No. 1 / January 2015
Black/African American
a
The total number of student–SP encounters was 3,053, and complete data for analysis were available for 2,882 (94%) of them. The number and percent distributions of all encounters by student and SP ethnicity are reported in Table 1. Table 2 provides the number of encounters used in the analysis by student and SP ethnicity and by student and SP gender.
In the multivariate analysis, the following main effects were all statistically significant at P < .001: students’ ethnicity, SPs’ ethnicity, students’ gender, and SPs’ gender (Table 3). Compared with their male counterparts, female students—regardless of ethnicity—obtained better assessments on both the JSPPPE and the GRE. The JSPPPE mean scores for men and women were, respectively, 24.1 and 25.6 (P < .0001); on the GRE, men’s and women’s scores were, respectively, 6.9 and 7.4 (P < .0001 [not shown]). Regardless of the ethnicity and gender of the SPs, black/ African American students received significantly lower assessments on the JSPPPE compared with the other ethnic groups; the mean score for black/African American students was 21.9; for white students, 25.0; and for Asian/Pacific Islander students, 25.2; P < .0001 [not shown]). A similar pattern was observed for the GRE; mean scores for black/African American students were 6.8; for white students, 7.2; and for Asian/Pacific Islander students, 7.2 (P white); the main effect of student gender is F(1, 2874) = 13.3 (women > men)—all statistically significant with P < .001. The interaction effect of student ethnicity and SP ethnicity is F(2, 2874) = 3.5 (P = .03), and the interaction effect of student gender and SP ethnicity is F(1, 2874) = 0.45 (P = .50, not significant). d The results for the univariate analysis for the GRE are as follows: the main effect of student ethnicity is F(2, 2874) = 8.6 (black/African American < white = Asian/Pacific Islanders); the main effect of SP ethnicity is F(1, 2874) = 16.7 (black/ African American > white); and the main effect of student gender: F(1, 2874) = 15.3 (women > men)—all statistically significant with P < .001. The interaction effect of student ethnicity and SP ethnicity is F(2, 2874) = 4.0 (P = .01), and the interaction effect of student gender and SP ethnicity is F(1, 2874) = 0.04 (P = .83, not significant).
ethnicity and gender (P < .04) were all statistically significant.
(mean scores were, respectively, 56.8, 53.8, and 53.3; P = .005; results not shown).
Students’ self-reported empathy: Effect of ethnicity and gender
Our results showed that female students rated themselves higher in empathy than did male students; mean scores on the JSE for male students and female students were, respectively, 109.3 and 114.9 (P = .003; not shown). Similar patterns were found for the IRI scores; mean scores for men and women were, respectively, 52.2 and 56.1 (P = .0004; results not shown). The gender difference in empathy in favor of women, consistent with previous findings, has been attributed to evolutionary and social learning factors.17–19
Means and standard deviations of the self-reported empathy measures for students by ethnicity and gender, and summary results of statistical analyses, are reported in Table 4. The multivariate analysis for the main effects of students’ ethnicity (P < .02), students’ gender (P white = Asian/Pacific Islanders, significant with P = .03); the main effect of gender is F(1, 550) = 13.2 (women > men, significant with P = .0003); and the interaction effect of ethnicity and gender is F(2, 550) = 3.4 (significant with P = .03). d The results of the univariate analysis for the IRI are as follows: the main effect of ethnicity is F(2, 550) = 5.3 (black/African American > white = Asian/Pacific Islanders, significant with P = .005); the main effect of gender is F(1, 550) = 12.8 (women > men, significant with P = .0004); and the interaction effect of ethnicity and gender is F(2, 550) = 3.5 (significant with P = .03).
and ethnicity in SPs’ assessment of students’ empathy, which may introduce bias into SP exams. First, female students may have an advantage over male students. SPs of all categories rated female students, regardless of ethnicity, significantly higher than they rated male students. Second, black/ African American students may be at a disadvantage compared with white and Asian/Pacific Islander students. SPs of all categories rated male black/African American students lower than their white and Asian/Pacific Islander counterparts. Third, black/African American male students may be particularly disadvantaged when interacting with black/African American female SPs. Fourth, all students, except black/African American men, may have an advantage if they are being assessed by a male black/ African American SP, because these SPs, as a group, were generally more lenient graders than other SPs. A secondary purpose of our study was to examine students’ self-reported empathy for ethnicity and gender effects so as to compare self-perception with the external perception of SPs. When analyzed by gender, our results indicated that female students’ self-perception was congruent with SPs’ assessment. Women of all ethnicities obtained significantly higher mean scores than their male counterparts on both SP-assessed and self-assessed ratings. When analyzed by ethnicity,
Academic Medicine, Vol. 90, No. 1 / January 2015
the self-perception of black/African American students was incongruent with SP assessments. Though they received significantly lower SP ratings than other ethnicities, both male and female black/African American students rated themselves higher in empathy than either male white and male and female Asian/ Pacific Islander students. Potential causes of the interaction effects The significant interaction effects of ethnicity and gender in SP encounters that we detected raise potential concerns about fairness in assessing medical students’ interpersonal skills. Though our study was not designed to provide evidence of why these effects occur, exploring not only possible reasons but also possible means for addressing and preventing them is important. The causes of the effects may stem from the students, the SPs, the empathy evaluation instruments, or a combination of these factors. Students. Some evidence shows that environmental factors can affect the way in which medical students manifest empathy. Gender culture and ethnic culture may encourage or discourage the degree and manner of students’ expression of empathy.4 These gender and ethnic differences may or may not align well with the way SPs are trained to evaluate empathy. Female medical students may be scoring higher than
males because women are, on average, more empathic than men because of issues of gender socialization.29 Further, SPs may be more likely to view women as empathetic because of gender stereotypes.4,17 In the cases of male gender and black/African American ethnicity, students in these groups may demonstrate empathy less than or differently than their educators expect. SPs might interpret the gender/ethnic differences as a lack of empathy, which could confound the validity of such assessments. Possibly, black/African American male students may have lower self-expectations for empathy than all of the other groups, which, if true, may account for the generous empathy ratings that SPs of this demographic gave medical students. Alternatively, our findings could be the result of the possible inaccuracy that black/African American male students have assessing their own empathy. Previous studies have demonstrated that many students who achieve poor ratings on the OSCE have limited insight into their performance and rate themselves higher than they actually perform.30 Students who express empathy less or differently than expected might benefit from additional training in mainstream empathy skills, which has been shown in studies to positively affect patient outcomes.5 Some research demonstrates that empathy can be enhanced effectively by dedicated educational programs.4 SPs. SP factors may also be responsible for the gender/ethnicity effects we detected. Some studies have reported that SPs can better assess well-defined technical skills such as history taking and physical examination rather than interpersonal skills such as empathy and teamwork.31,32 Another possibility is that black/African American male students are in fact more empathetic, as demonstrated by their self-rating, but that the SPs are biased against them—resulting in the lower empathy scores. SPs or those who train them may have unconscious biases involving certain categories of students, including perhaps a stereotype against black/African American men. The fact that male black/African American SPs rated students noticeably different than did their white and Asian/Pacific Islander counterparts also requires further examination. It may be beneficial for the SPs to undergo unconscious bias
109
Research Report
training to help decrease the potential for bias. Rubrics and standardized processes targeted at unconscious bias for all SPs could help to keep unconscious biases from affecting the outcome of OSCEs. Assessment instruments. Finally, the interaction effects could also be the result of flaws in the assessment instruments. Although they have been extensively validated,4,13–19 the instruments may not have been sufficiently assessed with students of some ethnicities or ethnic– gender combinations, confounding their validity. The instruments used by the SPs (the JSPPPE and the GRE) could be inherently biased against black/African American males if they lack questions that reflect the empathic qualities black/African American male students demonstrate. The causes of the interaction effects can be explored further through analysis of clinical skills exam videos and qualitative research. More research is needed to confirm our findings and to better understand causality. Strengths and limitations This study offers unique advantages. First, its multi-institutional scope increases our confidence in the external validity (generalizability) and credibility of our findings. Second, the large sample size adds to the reliability of the results. Third, the use of two dependent variables for students’ self-reported empathy (JSE and IRI), two variables for SPs’ assessment of students’ empathy (JSPPPE and GRE), and, importantly, the generally consistent findings on these measures increase our confidence in the internal validity of the results. Fourth, the availability of information about both the gender and the ethnicity of students and SPs alike enabled us to simultaneously examine the interaction effects of these demographic characteristics. Fifth, the availability of students’ self-reported empathy and SPs’ assessments of the same students’ empathy provided the opportunity to compare the pattern of students’ selfreported empathy scores and the pattern of students’ empathy as assessed by SPs. The study also has some limitations. First, the OSCE is an inherently regimented checklist-driven assessment.12,33 Second, the study includes institutional and geographical disparity in the number
110
of ethnic minority students who were not equally represented in each school in the Mid-Atlantic Consortium sample population. For example, whereas 91% of HU students were black/African American, the proportion was less than 14% in the other participating schools. Because the sample size of students from HU was smaller than for the other schools (10% of the sample, versus 24%– 41%), their exclusion from the analysis would not change the overall pattern of the findings. Third, the observations are not independent because one SP assesses multiple students. This design is fundamental to the OSCE as using fewer SPs increases the reliability of OSCEs and it improves standardization of the cases and scoring.
Mr. Heil is a research assistant, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
Conclusions
Dr. Hojat is research professor of psychiatry and human behavior and director, Jefferson Longitudinal Study, Center for Research in Medical Education and Health Care, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
Our findings that assessment of medical students’ empathy by SPs in clinical skills examinations may be biased by gender and ethnicity raises concerns about the fairness of the methods used to assess this competency. Thus, the findings of this study call for further research to address possible ethnic- and gender-related biases in SPs’ assessments of medical students’ interpersonal skills in OSCEs. Acknowledgments: The authors wish to thank Dorissa Bolinski for her editorial assistance. They would also like to acknowledge the Mid-Atlantic Consortium Study Group, whose membership includes Karen Lewis, Carol Fleishman, Rita Duboyce, and Amy Flanagan (in addition to authors K.B., B.B., J.L., J.J., T.O., and P.C.-N.). Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This study was exempted by the internal review boards of all participating institutions. Dr. K. Berg is professor of medicine and codirector, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Dr. Blatt is professor of medicine and director, Clinical Learning and Simulation Skills Center, George Washington University School of Medicine, Washington, DC. Dr. Lopreiato is professor of pediatrics and associate dean for simulation education, Uniformed Services University of the Health Sciences, Bethesda, Maryland. Dr. Jung is assistant professor of emergency medicine and associate director, Simulation Center, Johns Hopkins University, Baltimore, Maryland. Ms. Schaeffer is a medical student, Tufts University, Boston, Massachusetts.
Ms. Owens is director, Clinical Skills and Simulation Center, Howard University, Washington, DC. Dr. Carter-Nolan is assistant dean for medical education, vice chair for education, and assistant professor, Department of Community and Family Medicine, Howard University, Washington, DC. Dr. D. Berg is G. Fritz Blechschmidt, MD, Professor of Clinical Skills and codirector, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Mr. Veloski is director, Medical Education Research, Center for Research in Medical Education and Health Care, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Ms. Darby is standardized patient educator, Uniform Services University of the Health Sciences, Bethesda, Maryland.
References 1 Osler W. Teaching and thinking. In: Aequanimitas, With Other Addresses to Medical Students, Nurses, and Practitioners of Medicine. Philadelphia, Pa: Blakiston’s Son & Co.; 1904:121–136. 2 Association of American Medical Colleges. Learning Objectives for Medical Student Education: Guidelines for Medical Schools. 1998. https://www.aamc.org/initiatives/ msop/. Accessed August 28, 2014. 3 American Board of Internal Medicine. Evaluation of humanistic qualities in the internists. Ann Intern Med. 1983;99:720–724. 4 Hojat M. Empathy in Patient Care: Antecedents, Development, Measurement, and Outcomes. New York, NY: Springer; 2007. 5 Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS. Physicians’ empathy and clinical outcomes for diabetic patients. Acad Med. 2011;86:359–364. 6 Hojat M, Vergare MJ, Maxwell K, et al. The devil is in the third year: A longitudinal study of erosion of empathy in medical school. Acad Med. 2009;84:1182–1191. 7 Passiment M, Sacks H, Huang G. Medical Simulation in Medical Education: Results of an AAMC Survey. September 2011. https:// www.aamc.org/download/259760/data. Accessed August 28, 2014. 8 Terrill A, Anderson MB. Special section: Annex to the proceedings of the AAMC consensus conference on the use of standardized patients in teaching and evaluation of clinical skills. Teach Learn Med. 1994;6:2–35. 9 Colliver JA, Swartz MH, Robbs RS. The effect of examinee and patient ethnicity in clinical-skills assessment with standardized patients. Adv Health Sci Educ Theory Pract. 2001;6:5–13. 10 Swartz MH, Colliver JA, Robbs RS. The interaction of examinee’s ethnicity
Academic Medicine, Vol. 90, No. 1 / January 2015
Research Report
11
12
13
14
15
16
17
and standardized patient’s ethnicity: An extended analysis. Acad Med. 2001;76(10 suppl):S96–S98. De Champlain AF, Schoeneberger J, Boulet JR. Assessing the impact of examinee and standardized patient ethnicity on test scores in a large-scale clinical skills examination: Gathering evidence for the consequential aspect of validity. Acad Med. 2004;79(10 suppl): S12–S14. Berg K, Majdan JF, Berg D, Veloski J, Hojat M. Medical students’ self-reported empathy and simulated patients’ assessments of student empathy: An analysis by gender and ethnicity. Acad Med. 2011;86:984–988. Kane GC, Gotto JL, Mangione S, West S, Hojat M. Jefferson Scale of Patient’s Perceptions of Physician Empathy: Preliminary psychometric data. Croat Med J. 2007;48:81–86. Glaser KM, Markham FW, Adler HM, McManus PR, Hojat M. Relationships between scores on the Jefferson Scale of Physician Empathy, patient perceptions of physician empathy, and humanistic approaches to patient care: A validity study. Med Sci Monit. 2007;13:CR291–CR294. Hojat M, Louis DZ, Maxwell K, Markham F, Wender R, Gonnella JS. Patient perceptions of physician empathy, satisfaction with physician, interpersonal trust, and compliance. Int J Med Educ. 2010;1:83–87. Dorflinger L, Schumann K. Critical Synthesis Package: Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE). MedEdPORTAL. August 2013. https://www. mededportal.org/publication/9485. Accessed August 28, 2014. Hojat M, Gonnella JS, Nasca TJ, Mangione S, Vergare M, Magee M. Physician empathy: Definition, components, measurement, and
Academic Medicine, Vol. 90, No. 1 / January 2015
18
19
20 21
22
23
24
25
relationship to gender and specialty. Am J Psychiatry. 2002;159:1563–1569. Hojat M, Mangione S, Nasca TJ, et al. The Jefferson Scale of Physician Empathy: Development and preliminary psychometric data. Educ Psychol Meas. 2001;61: 349–365. Hojat M, Gonnella JS, Mangione S, et al. Empathy in medical students as related to academic performance, clinical competence and gender. Med Educ. 2002;36:522–527. Sherman JJ, Cramer A. Measurement of changes in empathy during dental school. J Dent Educ. 2005;69:338–345. Ward J, Schaal M, Sullivan J, Bowen ME, Erdmann JB, Hojat M. Reliability and validity of the Jefferson Scale of Empathy in undergraduate nursing students. J Nurs Meas. 2009;17:73–88. Alcorta-Garza A, González-Guerrero JF, Tavitas-Herrera SE, Rodríguez-Lara FJ, Hojat M. Validity and reliability of the Jefferson Scale of Physician Empathy in Mexican medical students [in Spanish]. Salud Ment [Mex]. 2005;28:57–63. Kataoka HU, Koide N, Ochi K, Hojat M, Gonnella JS. Measurement of empathy among Japanese medical students: Psychometrics and score differences by gender and level of medical education. Acad Med. 2009;84:1192–1197. Roh MS, Hahm BJ, Lee DH, Suh DH. Evaluation of empathy among Korean medical students: A cross-sectional study using the Korean version of the Jefferson Scale of Physician Empathy. Teach Learn Med. 2010;22:167–171. Di Lillo M, Cicchetti A, Lo Scalzo A, Taroni F, Hojat M. The Jefferson Scale of Physician Empathy: Preliminary psychometrics and
26 27
28
29
30
31
32
33
group comparisons in Italian physicians. Acad Med. 2009;84:1198–1202. Davis MH. Measuring individual differences in empathy: Evidence for a multidimensional approach. J Pers Soc Psychol. 1983;44:113–126. Hojat M, Mangione S, Kane GC, Gonnella JS. Relationships between scores of the Jefferson Scale of Physician Empathy (JSPE) and the Interpersonal Reactivity Index (IRI). Med Teach. 2005;27:625–628. United States Medical Licensing Examinations. Step 2 Clinical Skills Content Description and General Information. January 2014. http://www.usmle.org/pdfs/ step-2-cs/cs-info-manual.pdf. Accessed August 28, 2014. Garaigordobil M. A comparative analysis of empathy in childhood and adolescence: Gender differences and associated socioemotional variables. Int J Psychol Psychol Ther. 2009;9:217–235. Martin D, Regehr G, Hodges B, McNaughton N. Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Acad Med. 1998;73:1201–1206. Carlson J, Min E, Bridges D. The impact of leadership and team behavior on standard of care delivered during human patient simulation: A pilot study for undergraduate medical students. Teach Learn Med. 2009;21:24–32. Flin R, Maran R. Identifying and training non-technical skills for teams in acute medicine. Qual Saf Health Care (QSHC). 2004;13(suppl):S80–S84. Berg K, Majdan JF, Berg D, Veloski J, Hojat M. A comparison of medical students’ selfreported empathy with simulated patients’ assessments of the students’ empathy. Med Teach. 2011;33:388–391.
111
Standardized Patient Assessment of Medical Student Empathy: Ethnicity and Gender Effects in a Multi-Institutional Study Katherine Berg, MD, MPH, Benjamin Blatt, MD, Joseph Lopreiato, MD, MPH, Julianna Jung, MD, Arielle Schaeffer, Daniel Heil, Tamara Owens, MEd, Pamela L. Carter-Nolan, PhD, MPH, Dale Berg, MD, Jon Veloski, MS, Elizabeth Darby, and Mohammadreza Hojat, PhD
Abstract Purpose To examine, primarily, the effects of ethnicity and gender, which could introduce bias into scoring, on standardized patient (SP) assessments of medical students and, secondarily, to examine medical students’ self-reported empathy for ethnicity and gender effects so as to compare self-perception with the perceptions of SPs. Method Participants were 577 students from four medical schools in 2012: 373 (65%) were white, 79 (14%) black/African American, and 125 (22%) Asian/Pacific Islander. These students were assessed by 84 SPs: 62 (74%) were white and 22 (26%) were black/African American.
E
mpathy is a pillar of the patient– physician relationship. Sir William Osler1 stated that a physician’s work “is arduous and complex, requiring the exercise of the very highest faculties of the mind, while constantly appealing to the emotions and finer feelings.” The importance of empathy in patient care has caused both the Association of American Medical Colleges and the American Board of Internal Medicine to recommend that empathy be cultivated and assessed as an essential medical education outcome.2,3 Empathy in the context of medicine is defined as a predominantly cognitive Please see the end of this article for information about the authors. Correspondence should be addressed to Dr. K. Berg, Sidney Kimmel Medical College at Thomas Jefferson University, University Clinical Skills and Simulation Center, Suite 309E, Hamilton Building, 1001 Locust St., Philadelphia, PA 19107; telephone: (215) 5034234; e-mail: [email protected]. Acad Med. 2015;90:105–111. First published online October 21, 2014 doi: 10.1097/ACM.0000000000000529
Academic Medicine, Vol. 90, No. 1 / January 2015
SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) and the Global Ratings of Empathy tool. Students completed the Jefferson Scale of Empathy and two Interpersonal Reactivity Index subscales. The investigators used 2,882 student–SP encounters in their analyses.
on the JSPPPE than their respective male counterparts. Male black/African American students obtained the lowest SP assessment scores of empathy regardless of SP ethnicity. Black/African American students obtained the highest mean scores on self-reported empathy.
Results Analyses of SPs’ assessments of students’ empathy indicated significant interaction effects of gender and ethnicity. Female students, regardless of ethnicity, obtained significantly higher mean JSPPPE scores than men. Female black/African American, female white, and female Asian/Pacific Islander students scored significantly higher
Conclusions The significant interaction effects of ethnicity and gender in clinical encounters, plus the inconsistencies observed between SPs’ assessments of students’ empathy and students’ self-reported empathy, raise questions about possible ethnicity and gender biases in the SPs’ assessments of medical students’ clinical skills.
attribute, involving an understanding of the experiences, concerns, and perspective of each patient, combined with a capacity to communicate this understanding and an intention to help.4–6 Many medical schools in the United States and elsewhere use standardized patients (SPs) in objective structured clinical examinations (OSCEs) to assess medical students’ clinical competencies, including their empathy. Currently, 94% of U.S. medical schools involve SPs in both clinical training and performance-based assessments, such as OSCEs.7
examinations and of the SP assessment approach in general.9,10
Since 1992, the Consensus Conference on the Use of SPs has recommended the need for studies evaluating how racial, ethnic, and cultural factors affect SP ratings of students’ performances during clinical skills testing.8 OSCEs and U.S. medical licensure exams alike involve SPs in clinical skills assessment; therefore, the possibility of a significant interaction between the SP’s and the student’s ethnicity and/or gender could threaten the validity of these high-stakes
Although at least one study has reported no ethnicity influence in SPs’ assessment scores,11 others have shown significant interaction effects.9,10 Some of us conducted an earlier study and found that Asian American medical students obtained lower empathy ratings from SPs than did white students.12 These findings raised questions about the possible effect ethnicity might have on SP assessments of students (and, more broadly, about bias in testing). That earlier study, however, had several shortcomings. It was from a single institution, involved few African American students as participants, and did not consider the ethnicity and gender of the SPs. The present study was designed to overcome those shortcomings by including four medical schools with more diverse student populations. The primary purpose of this study, therefore, was to examine the effects of ethnicity and gender, which could bias
105
Research Report
scoring, on SP assessments of medical students’ empathy. A secondary purpose was to examine medical students’ self-reported empathy for ethnicity and gender effects so as to compare their self-perceptions with the external perceptions of SPs. Method
Participants Participants included medical students from four medical schools: the George Washington University (GWU), Howard University (HU), Sidney Kimmel Medical College at Thomas Jefferson University (TJU), and the Uniformed Services University of the Health Sciences (USU). Instruments To assess student empathy, the SPs completed two instruments during the OSCE: (1) the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE), and (2) the Global Ratings of Empathy (GRE) tool. The JSPPPE. The JSPPPE contains five items measuring SPs’ perception of an individual student’s empathic engagement. Each item (e.g., “This physician [or student] understands my emotions, feelings, and concerns”) is scored on a seven-point Likert-type scale (1 = strongly disagree, 7 = strongly agree). The score is calculated from the sum of the item scores received from SPs; the higher the score, the more favorably the SPs viewed the student’s empathy. Multiple previous studies have supported the psychometric validity of this scale.13–16 The GRE. The GRE is a two-item tool, completed by the SPs, using a five-point Likert scale (1 = strongly disagree, 5 = strongly agree), after each encounter with students. The two items are “This student understands my perspective” and “The student supports my emotions.” The score, calculated from the sum of the item scores received from all SPs, serves as another measure of student empathy. Again, the higher the score, the more favorably the evaluators, in this case the SPs, view the student’s empathy. Medical students also completed two instruments to assess their own empathy: the Jefferson Scale of Empathy (JSE) and the Interpersonal Reactivity Index (IRI).
106
The JSE. The JSE measures students’ selfassessment of their own empathy. Like the JSPPPE, each item is scored on a sevenpoint Likert-type scale (1 = strongly disagree, 7 = strongly agree). Some items (e.g., “Because people are different, it is difficult to see problems from patients’ perspectives”) are reverse scored. Previous publications have provided extensive psychometric evidence in support of the JSE as a valid instrument to use with medical students.4,17–19 Similar underlying construct and psychometric properties have been reported among dental students,20 nursing students,21 and medical students from several different countries.22–25 The score is calculated from the sum of item scores (possible range 20–140); the higher the score, the more empathic a student assesses him- or herself to be. The IRI. The IRI, developed by M.H. Davis, is a four-subscale (perspectivetaking, empathic concern, fantasy, and personal distress) instrument that has been used to measure empathy in the general population.26 This study used the perspective-taking (as an indicator of cognitive empathy in general context) and empathic concern (as an indicator of effective empathy, or sympathy) subscales—each consisting of seven items. A typical item from the perspective-taking subscale is “I sometimes try to understand my friends better by imagining how things look from their perspective,” and a typical item from the empathic concern subscale is “I often have tender, concerned feelings for people less fortunate than me”). Correlations between the JSE and subscale scores of the IRI have been reported.27 The higher the IRI score, the more empathy the medical students in our study assessed themselves as having. Procedures Students’ clinical skills were assessed at the end of the third year or the beginning of the fourth year of medical school in four schools that are members of the Mid-Atlantic Consortium. All schools used the same six-station OSCE, which was designed to be similar to the United States Medical Licensing Exam Step 2 Clinical Skills assessment.28 The case content was determined by the MidAtlantic Consortium based on the objectives of the medical schools. The cases had been piloted previously and had undergone systematic review to improve
their reliability and validity. The four schools used identical training materials (and three even used the same trainer) to train SPs for their cases. A passing score is required at all institutions. Data were collected at each institution and deidentified prior to being sent to TJU for analysis. The institutional review boards of all four institutions granted this study exempt status. Students and SPs alike signed appropriate consent forms and participated voluntarily. Students were not compensated and were informed that individual responses would be confidential and not part of their academic records. We examined the effects of ethnicity and gender by using data from patient encounters, defined as one SP’s assessment of one medical student in each OSCE station. After each encounter, the SPs assessed students using both the JSPPPE and the GRE. The students took the JSE and IRI the same day they took their OSCE. The unit of analysis was the student–SP encounter. Statistical analysis To analyze students’ self-reported empathy, we used multivariate analysis of variance (MANOVA). Students’ ethnicity and gender were the independent variables, while their scores on the JSE and IRI were the dependent variables (two-way 3 × 2 design). For the patient encounters, we also used MANOVA in a four-way (3 × 2 × 2 × 2) design in which a student’s ethnicity (white, black/ African American, and Asian/Pacific Islander), the SP’s ethnicity (white, black/ African American), the student’s gender (male, female), and the SP’s gender (male, female) were four independent variables. Scores on the JSPPPE and GRE were the dependent variables. We made comparisons simultaneously for all groups classified by ethnicity and gender. When multivariate F ratios were statistically significant, we used univariate analysis of variance for each dependent variable followed by post hoc mean comparisons. The probability of type I (alpha) error was set at .05 to determine the significance of statistical findings. We analyzed the data using the Statistical Analysis System (SAS version 9.3 for Windows, Cary, North Carolina) to determine the mean, standard deviation,
Academic Medicine, Vol. 90, No. 1 / January 2015
Research Report
and number of encounters for each of the groups by student ethnicity and gender and by SP ethnicity and gender.
Table 1 Number and Distribution of Patient Encounters Between Medical Students and Standardized Patients (SPs) by Ethnicity in a Study of the Interaction Effects of Ethnicity and Gender on SP Assessments of Medical Student Empathy, 2012
Results
SPs
Participants Of the 577 medical students who participated, 290 (50%) were men, and 145 (25%) were from GWU, 57 (10%) from HU, 238 (41%) from TJU, and 137 (24%) from USU. The sample consisted of 373 (65%) white students, 79 (14%) black/African American students, and 125 (22%) Asian/Pacific Islander students. Also, 84 SPs participated in the study: 39 (46%) were men; 62 (74%) were white, and 22 (26%) were black/African American (none were Asian/Pacific Islander). Of the 62 white SPs, 30 (48%) were men, and of the 22 black/African American SPs, 9 (41%) were men.
Students White Black/African American Asian/Pacific Islander Total
Total
1,137 256
854 142
1,991 398
343
321
664
1,736
1,317
3,053a
The investigators used 2,882 (94%) of the SP–student encounters in their analyses.
In the multivariate analyses, the interaction of students’ ethnicity and SPs’ ethnicity was statistically significant (P < .05). In the interactions of students’ ethnicity and gender and SPs’ ethnicity and gender, the results were highly significant (P < .001). No significant interaction effect was obtained for students’ gender and SPs’ ethnicity (P = .13). See Table 3. For our interpretation, we emphasized the interactions that we found to be statistically significant (i.e., students’ ethnicity and gender, interacting with SPs’ ethnicity and gender; Table 3). Post hoc comparisons of the mean scores of the JSPPPE and the GRE showed a number of patterns in these four-way interactions. First, in most instances, SPs of all categories rated female students higher than male students. The male student scores, though sometimes equal, never significantly exceeded the scores of female students of any ethnicity. Second, SPs of all categories rated female black/
Patient encounters: Main and interaction effects of ethnicity and gender
Academic Medicine, Vol. 90, No. 1 / January 2015
Black/African American
a
The total number of student–SP encounters was 3,053, and complete data for analysis were available for 2,882 (94%) of them. The number and percent distributions of all encounters by student and SP ethnicity are reported in Table 1. Table 2 provides the number of encounters used in the analysis by student and SP ethnicity and by student and SP gender.
In the multivariate analysis, the following main effects were all statistically significant at P < .001: students’ ethnicity, SPs’ ethnicity, students’ gender, and SPs’ gender (Table 3). Compared with their male counterparts, female students—regardless of ethnicity—obtained better assessments on both the JSPPPE and the GRE. The JSPPPE mean scores for men and women were, respectively, 24.1 and 25.6 (P < .0001); on the GRE, men’s and women’s scores were, respectively, 6.9 and 7.4 (P < .0001 [not shown]). Regardless of the ethnicity and gender of the SPs, black/ African American students received significantly lower assessments on the JSPPPE compared with the other ethnic groups; the mean score for black/African American students was 21.9; for white students, 25.0; and for Asian/Pacific Islander students, 25.2; P < .0001 [not shown]). A similar pattern was observed for the GRE; mean scores for black/African American students were 6.8; for white students, 7.2; and for Asian/Pacific Islander students, 7.2 (P white); the main effect of student gender is F(1, 2874) = 13.3 (women > men)—all statistically significant with P < .001. The interaction effect of student ethnicity and SP ethnicity is F(2, 2874) = 3.5 (P = .03), and the interaction effect of student gender and SP ethnicity is F(1, 2874) = 0.45 (P = .50, not significant). d The results for the univariate analysis for the GRE are as follows: the main effect of student ethnicity is F(2, 2874) = 8.6 (black/African American < white = Asian/Pacific Islanders); the main effect of SP ethnicity is F(1, 2874) = 16.7 (black/ African American > white); and the main effect of student gender: F(1, 2874) = 15.3 (women > men)—all statistically significant with P < .001. The interaction effect of student ethnicity and SP ethnicity is F(2, 2874) = 4.0 (P = .01), and the interaction effect of student gender and SP ethnicity is F(1, 2874) = 0.04 (P = .83, not significant).
ethnicity and gender (P < .04) were all statistically significant.
(mean scores were, respectively, 56.8, 53.8, and 53.3; P = .005; results not shown).
Students’ self-reported empathy: Effect of ethnicity and gender
Our results showed that female students rated themselves higher in empathy than did male students; mean scores on the JSE for male students and female students were, respectively, 109.3 and 114.9 (P = .003; not shown). Similar patterns were found for the IRI scores; mean scores for men and women were, respectively, 52.2 and 56.1 (P = .0004; results not shown). The gender difference in empathy in favor of women, consistent with previous findings, has been attributed to evolutionary and social learning factors.17–19
Means and standard deviations of the self-reported empathy measures for students by ethnicity and gender, and summary results of statistical analyses, are reported in Table 4. The multivariate analysis for the main effects of students’ ethnicity (P < .02), students’ gender (P white = Asian/Pacific Islanders, significant with P = .03); the main effect of gender is F(1, 550) = 13.2 (women > men, significant with P = .0003); and the interaction effect of ethnicity and gender is F(2, 550) = 3.4 (significant with P = .03). d The results of the univariate analysis for the IRI are as follows: the main effect of ethnicity is F(2, 550) = 5.3 (black/African American > white = Asian/Pacific Islanders, significant with P = .005); the main effect of gender is F(1, 550) = 12.8 (women > men, significant with P = .0004); and the interaction effect of ethnicity and gender is F(2, 550) = 3.5 (significant with P = .03).
and ethnicity in SPs’ assessment of students’ empathy, which may introduce bias into SP exams. First, female students may have an advantage over male students. SPs of all categories rated female students, regardless of ethnicity, significantly higher than they rated male students. Second, black/ African American students may be at a disadvantage compared with white and Asian/Pacific Islander students. SPs of all categories rated male black/African American students lower than their white and Asian/Pacific Islander counterparts. Third, black/African American male students may be particularly disadvantaged when interacting with black/African American female SPs. Fourth, all students, except black/African American men, may have an advantage if they are being assessed by a male black/ African American SP, because these SPs, as a group, were generally more lenient graders than other SPs. A secondary purpose of our study was to examine students’ self-reported empathy for ethnicity and gender effects so as to compare self-perception with the external perception of SPs. When analyzed by gender, our results indicated that female students’ self-perception was congruent with SPs’ assessment. Women of all ethnicities obtained significantly higher mean scores than their male counterparts on both SP-assessed and self-assessed ratings. When analyzed by ethnicity,
Academic Medicine, Vol. 90, No. 1 / January 2015
the self-perception of black/African American students was incongruent with SP assessments. Though they received significantly lower SP ratings than other ethnicities, both male and female black/African American students rated themselves higher in empathy than either male white and male and female Asian/ Pacific Islander students. Potential causes of the interaction effects The significant interaction effects of ethnicity and gender in SP encounters that we detected raise potential concerns about fairness in assessing medical students’ interpersonal skills. Though our study was not designed to provide evidence of why these effects occur, exploring not only possible reasons but also possible means for addressing and preventing them is important. The causes of the effects may stem from the students, the SPs, the empathy evaluation instruments, or a combination of these factors. Students. Some evidence shows that environmental factors can affect the way in which medical students manifest empathy. Gender culture and ethnic culture may encourage or discourage the degree and manner of students’ expression of empathy.4 These gender and ethnic differences may or may not align well with the way SPs are trained to evaluate empathy. Female medical students may be scoring higher than
males because women are, on average, more empathic than men because of issues of gender socialization.29 Further, SPs may be more likely to view women as empathetic because of gender stereotypes.4,17 In the cases of male gender and black/African American ethnicity, students in these groups may demonstrate empathy less than or differently than their educators expect. SPs might interpret the gender/ethnic differences as a lack of empathy, which could confound the validity of such assessments. Possibly, black/African American male students may have lower self-expectations for empathy than all of the other groups, which, if true, may account for the generous empathy ratings that SPs of this demographic gave medical students. Alternatively, our findings could be the result of the possible inaccuracy that black/African American male students have assessing their own empathy. Previous studies have demonstrated that many students who achieve poor ratings on the OSCE have limited insight into their performance and rate themselves higher than they actually perform.30 Students who express empathy less or differently than expected might benefit from additional training in mainstream empathy skills, which has been shown in studies to positively affect patient outcomes.5 Some research demonstrates that empathy can be enhanced effectively by dedicated educational programs.4 SPs. SP factors may also be responsible for the gender/ethnicity effects we detected. Some studies have reported that SPs can better assess well-defined technical skills such as history taking and physical examination rather than interpersonal skills such as empathy and teamwork.31,32 Another possibility is that black/African American male students are in fact more empathetic, as demonstrated by their self-rating, but that the SPs are biased against them—resulting in the lower empathy scores. SPs or those who train them may have unconscious biases involving certain categories of students, including perhaps a stereotype against black/African American men. The fact that male black/African American SPs rated students noticeably different than did their white and Asian/Pacific Islander counterparts also requires further examination. It may be beneficial for the SPs to undergo unconscious bias
109
Research Report
training to help decrease the potential for bias. Rubrics and standardized processes targeted at unconscious bias for all SPs could help to keep unconscious biases from affecting the outcome of OSCEs. Assessment instruments. Finally, the interaction effects could also be the result of flaws in the assessment instruments. Although they have been extensively validated,4,13–19 the instruments may not have been sufficiently assessed with students of some ethnicities or ethnic– gender combinations, confounding their validity. The instruments used by the SPs (the JSPPPE and the GRE) could be inherently biased against black/African American males if they lack questions that reflect the empathic qualities black/African American male students demonstrate. The causes of the interaction effects can be explored further through analysis of clinical skills exam videos and qualitative research. More research is needed to confirm our findings and to better understand causality. Strengths and limitations This study offers unique advantages. First, its multi-institutional scope increases our confidence in the external validity (generalizability) and credibility of our findings. Second, the large sample size adds to the reliability of the results. Third, the use of two dependent variables for students’ self-reported empathy (JSE and IRI), two variables for SPs’ assessment of students’ empathy (JSPPPE and GRE), and, importantly, the generally consistent findings on these measures increase our confidence in the internal validity of the results. Fourth, the availability of information about both the gender and the ethnicity of students and SPs alike enabled us to simultaneously examine the interaction effects of these demographic characteristics. Fifth, the availability of students’ self-reported empathy and SPs’ assessments of the same students’ empathy provided the opportunity to compare the pattern of students’ selfreported empathy scores and the pattern of students’ empathy as assessed by SPs. The study also has some limitations. First, the OSCE is an inherently regimented checklist-driven assessment.12,33 Second, the study includes institutional and geographical disparity in the number
110
of ethnic minority students who were not equally represented in each school in the Mid-Atlantic Consortium sample population. For example, whereas 91% of HU students were black/African American, the proportion was less than 14% in the other participating schools. Because the sample size of students from HU was smaller than for the other schools (10% of the sample, versus 24%– 41%), their exclusion from the analysis would not change the overall pattern of the findings. Third, the observations are not independent because one SP assesses multiple students. This design is fundamental to the OSCE as using fewer SPs increases the reliability of OSCEs and it improves standardization of the cases and scoring.
Mr. Heil is a research assistant, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
Conclusions
Dr. Hojat is research professor of psychiatry and human behavior and director, Jefferson Longitudinal Study, Center for Research in Medical Education and Health Care, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
Our findings that assessment of medical students’ empathy by SPs in clinical skills examinations may be biased by gender and ethnicity raises concerns about the fairness of the methods used to assess this competency. Thus, the findings of this study call for further research to address possible ethnic- and gender-related biases in SPs’ assessments of medical students’ interpersonal skills in OSCEs. Acknowledgments: The authors wish to thank Dorissa Bolinski for her editorial assistance. They would also like to acknowledge the Mid-Atlantic Consortium Study Group, whose membership includes Karen Lewis, Carol Fleishman, Rita Duboyce, and Amy Flanagan (in addition to authors K.B., B.B., J.L., J.J., T.O., and P.C.-N.). Funding/Support: None reported. Other disclosures: None reported. Ethical approval: This study was exempted by the internal review boards of all participating institutions. Dr. K. Berg is professor of medicine and codirector, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Dr. Blatt is professor of medicine and director, Clinical Learning and Simulation Skills Center, George Washington University School of Medicine, Washington, DC. Dr. Lopreiato is professor of pediatrics and associate dean for simulation education, Uniformed Services University of the Health Sciences, Bethesda, Maryland. Dr. Jung is assistant professor of emergency medicine and associate director, Simulation Center, Johns Hopkins University, Baltimore, Maryland. Ms. Schaeffer is a medical student, Tufts University, Boston, Massachusetts.
Ms. Owens is director, Clinical Skills and Simulation Center, Howard University, Washington, DC. Dr. Carter-Nolan is assistant dean for medical education, vice chair for education, and assistant professor, Department of Community and Family Medicine, Howard University, Washington, DC. Dr. D. Berg is G. Fritz Blechschmidt, MD, Professor of Clinical Skills and codirector, University Clinical Skills and Simulation Center, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Mr. Veloski is director, Medical Education Research, Center for Research in Medical Education and Health Care, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Ms. Darby is standardized patient educator, Uniform Services University of the Health Sciences, Bethesda, Maryland.
References 1 Osler W. Teaching and thinking. In: Aequanimitas, With Other Addresses to Medical Students, Nurses, and Practitioners of Medicine. Philadelphia, Pa: Blakiston’s Son & Co.; 1904:121–136. 2 Association of American Medical Colleges. Learning Objectives for Medical Student Education: Guidelines for Medical Schools. 1998. https://www.aamc.org/initiatives/ msop/. Accessed August 28, 2014. 3 American Board of Internal Medicine. Evaluation of humanistic qualities in the internists. Ann Intern Med. 1983;99:720–724. 4 Hojat M. Empathy in Patient Care: Antecedents, Development, Measurement, and Outcomes. New York, NY: Springer; 2007. 5 Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS. Physicians’ empathy and clinical outcomes for diabetic patients. Acad Med. 2011;86:359–364. 6 Hojat M, Vergare MJ, Maxwell K, et al. The devil is in the third year: A longitudinal study of erosion of empathy in medical school. Acad Med. 2009;84:1182–1191. 7 Passiment M, Sacks H, Huang G. Medical Simulation in Medical Education: Results of an AAMC Survey. September 2011. https:// www.aamc.org/download/259760/data. Accessed August 28, 2014. 8 Terrill A, Anderson MB. Special section: Annex to the proceedings of the AAMC consensus conference on the use of standardized patients in teaching and evaluation of clinical skills. Teach Learn Med. 1994;6:2–35. 9 Colliver JA, Swartz MH, Robbs RS. The effect of examinee and patient ethnicity in clinical-skills assessment with standardized patients. Adv Health Sci Educ Theory Pract. 2001;6:5–13. 10 Swartz MH, Colliver JA, Robbs RS. The interaction of examinee’s ethnicity
Academic Medicine, Vol. 90, No. 1 / January 2015
Research Report
11
12
13
14
15
16
17
and standardized patient’s ethnicity: An extended analysis. Acad Med. 2001;76(10 suppl):S96–S98. De Champlain AF, Schoeneberger J, Boulet JR. Assessing the impact of examinee and standardized patient ethnicity on test scores in a large-scale clinical skills examination: Gathering evidence for the consequential aspect of validity. Acad Med. 2004;79(10 suppl): S12–S14. Berg K, Majdan JF, Berg D, Veloski J, Hojat M. Medical students’ self-reported empathy and simulated patients’ assessments of student empathy: An analysis by gender and ethnicity. Acad Med. 2011;86:984–988. Kane GC, Gotto JL, Mangione S, West S, Hojat M. Jefferson Scale of Patient’s Perceptions of Physician Empathy: Preliminary psychometric data. Croat Med J. 2007;48:81–86. Glaser KM, Markham FW, Adler HM, McManus PR, Hojat M. Relationships between scores on the Jefferson Scale of Physician Empathy, patient perceptions of physician empathy, and humanistic approaches to patient care: A validity study. Med Sci Monit. 2007;13:CR291–CR294. Hojat M, Louis DZ, Maxwell K, Markham F, Wender R, Gonnella JS. Patient perceptions of physician empathy, satisfaction with physician, interpersonal trust, and compliance. Int J Med Educ. 2010;1:83–87. Dorflinger L, Schumann K. Critical Synthesis Package: Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE). MedEdPORTAL. August 2013. https://www. mededportal.org/publication/9485. Accessed August 28, 2014. Hojat M, Gonnella JS, Nasca TJ, Mangione S, Vergare M, Magee M. Physician empathy: Definition, components, measurement, and
Academic Medicine, Vol. 90, No. 1 / January 2015
18
19
20 21
22
23
24
25
relationship to gender and specialty. Am J Psychiatry. 2002;159:1563–1569. Hojat M, Mangione S, Nasca TJ, et al. The Jefferson Scale of Physician Empathy: Development and preliminary psychometric data. Educ Psychol Meas. 2001;61: 349–365. Hojat M, Gonnella JS, Mangione S, et al. Empathy in medical students as related to academic performance, clinical competence and gender. Med Educ. 2002;36:522–527. Sherman JJ, Cramer A. Measurement of changes in empathy during dental school. J Dent Educ. 2005;69:338–345. Ward J, Schaal M, Sullivan J, Bowen ME, Erdmann JB, Hojat M. Reliability and validity of the Jefferson Scale of Empathy in undergraduate nursing students. J Nurs Meas. 2009;17:73–88. Alcorta-Garza A, González-Guerrero JF, Tavitas-Herrera SE, Rodríguez-Lara FJ, Hojat M. Validity and reliability of the Jefferson Scale of Physician Empathy in Mexican medical students [in Spanish]. Salud Ment [Mex]. 2005;28:57–63. Kataoka HU, Koide N, Ochi K, Hojat M, Gonnella JS. Measurement of empathy among Japanese medical students: Psychometrics and score differences by gender and level of medical education. Acad Med. 2009;84:1192–1197. Roh MS, Hahm BJ, Lee DH, Suh DH. Evaluation of empathy among Korean medical students: A cross-sectional study using the Korean version of the Jefferson Scale of Physician Empathy. Teach Learn Med. 2010;22:167–171. Di Lillo M, Cicchetti A, Lo Scalzo A, Taroni F, Hojat M. The Jefferson Scale of Physician Empathy: Preliminary psychometrics and
26 27
28
29
30
31
32
33
group comparisons in Italian physicians. Acad Med. 2009;84:1198–1202. Davis MH. Measuring individual differences in empathy: Evidence for a multidimensional approach. J Pers Soc Psychol. 1983;44:113–126. Hojat M, Mangione S, Kane GC, Gonnella JS. Relationships between scores of the Jefferson Scale of Physician Empathy (JSPE) and the Interpersonal Reactivity Index (IRI). Med Teach. 2005;27:625–628. United States Medical Licensing Examinations. Step 2 Clinical Skills Content Description and General Information. January 2014. http://www.usmle.org/pdfs/ step-2-cs/cs-info-manual.pdf. Accessed August 28, 2014. Garaigordobil M. A comparative analysis of empathy in childhood and adolescence: Gender differences and associated socioemotional variables. Int J Psychol Psychol Ther. 2009;9:217–235. Martin D, Regehr G, Hodges B, McNaughton N. Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Acad Med. 1998;73:1201–1206. Carlson J, Min E, Bridges D. The impact of leadership and team behavior on standard of care delivered during human patient simulation: A pilot study for undergraduate medical students. Teach Learn Med. 2009;21:24–32. Flin R, Maran R. Identifying and training non-technical skills for teams in acute medicine. Qual Saf Health Care (QSHC). 2004;13(suppl):S80–S84. Berg K, Majdan JF, Berg D, Veloski J, Hojat M. A comparison of medical students’ selfreported empathy with simulated patients’ assessments of the students’ empathy. Med Teach. 2011;33:388–391.
111
