Teaching and Learning in Medicine, 26(4), 366–372 C 2014, Taylor & Francis Group, LLC Copyright ISSN: 1040-1334 print / 1532-8015 online DOI: 10.1080/10401334.2014.945392
RESEARCH BASIC TO TEACHING AND LEARNING Developmental-Behavioral Pediatric Teaching of Medical Students: A National COMSEP Survey Neelkamal Soares Department of Pediatrics, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California, USA
Qishan Wu Department of Biostatistics, University of Kentucky, Lexington, Kentucky, USA
Shibani Kanungo UCLA Intercampus Genetics Training program, Los Angeles, California, USA
Background: The Council on Medical Student Education in Pediatrics (COMSEP) pediatric clerkship curriculum is widely followed. To date, there are no known studies on clerkship instruction related to developmental-behavioral pediatric (DBP) curricular elements. Purposes: The goals of this study are to examine pediatric clerkships’ current DBP teaching methods and to identify barriers and solutions to recommended curriculum implementation. Methods: Electronic survey was conducted with COMSEPmember pediatric clerkship directors. Descriptive statistics and qualitative data analysis was conducted. Results: Response rate was 66%. General Pediatricians (87.1%) were mostly responsible for clerkship DBP teaching. Around 18% of directors reported not assessing DBP competencies. Most clerkship directors report time constraints (61.8%) as a barrier to implementing the curriculum, along with faculty availability and resources. Suggested solutions included DBP faculty collaboration and resources. Conclusions: General pediatricians should collaborate with DBP faculty for instructional content creation, and community-based observational opportunities and web-based shared resources could help clerkship directors achieve the COMSEP DBP curriculum competencies. Keywords
developmental-behavioral pediatrics, medical student education, curriculum
BACKGROUND The Liaison Committee on Medical Education (LCME) is the nationally recognized accrediting authority for medical
We thank the COMSEP Annual Survey Committee for inclusion of our questionnaire in their annual survey. We acknowledge Sam Zinner, Carol Weitzman, and Franklin Trimm for their inspiration, and Linda Nield and Sherilyn Smith for the manuscript review. Correspondence may be sent to Neelkamal Soares, 120 Hamm Drive, Suite 2A, Lewisburg, PA 17837, USA. E-mail: nssoares@ geisinger.edu
education programs at the U.S. and Canadian medical schools represented by the Association of American Medical Colleges.1 LCME recommends clinical experiences (clerkships) in pediatrics, family medicine, internal medicine, obstetrics and gynecology, psychiatry, and surgery, which usually occur in the 3rd year of undergraduate medical education. LCME also wants clerkship directors to actively manage their curricula, develop competency-based learning objectives, and monitor students’ progress in the clerkship.2 The Council on Medical Student Education in Pediatrics (COMSEP) is composed of pediatric clerkship directors and others interested in medical student education in pediatrics. In association with the Academic Pediatric Association (formerly the Ambulatory Pediatric Association), COMSEP developed a national pediatric clerkship curriculum in 1995, which has been revised most recently in 2005.3 The curriculum, which has been adopted by more than 90% of pediatric clerkships, aims to provide an educational roadmap for medical student education in pediatrics, not just in the clerkship year but throughout the entire medical school experience. Foundational knowledge of typical and atypical child development and behavior and of psychosocial frameworks are important to understanding pathological processes in childhood growth, learning, and general health. Ultimately this prepares future clinicians to engage families with children as partners for effective healthcare management. In contrast to postgraduate pediatric education, training during the undergraduate pediatric clerkship has no mandatory developmental-behavioral pediatric (DBP) component. Pediatric clerkships provide opportunities for students to participate in clinic-based and community-based DBP learning opportunities. To date, the extent and nature of these opportunities have not been published. The COMSEP curriculum delineates the expected competencies in DBP topics upon completion of the pediatrics clerkship, with suggested
366
367
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
strategies in instruction and in evaluating student performance.3 To our knowledge, there are no studies that determine if, and how, programs are implementing the COMSEP curriculum to achieve the delineated competencies and barriers these programs face in the process.
PURPOSE The goals of this study are to examine pediatric clerkships’ DBP teaching methods and assessment of suggested DBP curricular competencies, to identify potential barriers to implementing the recommended curriculum, and to explore strategies to address the barriers.
METHODS In the fall of 2010, COMSEP conducted a survey of its membership, consisting of leaders of pediatric education for medical students, including clerkship directors, site directors, academic pediatric deans, and other pediatric faculty interested in medical student education. The confidential electronic survey consisted of 60 questions from five investigator teams (including ours). The first 20 questions elicited general demographic information. The other investigators were studying social networking use, professionalism remediation, expectations about oral case presentations, and educational impact of family-centered rounds. COMSEP members were electronically invited to complete the survey, and nonresponders were sent a maximum of three reminders at 4-week intervals from September 2010 to December 2010. Survey participants’ deidentified responses were saved in a confidential database, and no incentives were provided to participate. Approval was obtained from both Institutional Review Boards at the University of Kentucky and the University of Washington, where the final survey was compiled. Our DBP component included 17 multiple-choice questions and elicited binomial and qualitative responses about methods of instruction, assessment, respondent perception of students’ competencies, and perceived barriers to implementing the DBP curriculum (see the supplementary file). All statistical analyses were done using SAS 9.3 (SAS Institute Inc, Cary, NC). Descriptive statistical methods were used to compute all frequency tables for categorical variables and mean, standard deviation, minimum, and maximum for continuous variables. McNemar’s test4 was used to analyze if developmental competencies and behavioral competencies have different patterns of attainment achievement (excellent, above average, average, below average, and poor). Qualitative analysis of barriers and suggested resources was performed independently by two authors (NS and SK) with absolute agreement on the categories. Descriptive statistics with frequency counts (number and percentages) were conducted.
RESULTS Respondent Characteristics Although the survey was disseminated to all COMSEP members, our study focused only on pediatric clerkship directors. Response rate for the clerkship directors was 66% (95/144), 53% (n = 50) were female. The predominant respondents were M.D./D.O. board certified in pediatrics, and the majority were general pediatricians, at either Associate or Assistant Professor rank, almost half of whom had been directing the clerkship for more than 5 years (Table 1). Clerkship Characteristics Among respondents, 93.7% (n = 89) indicated that their clerkships followed the COMSEP curriculum. There were an average of 20 medical students on the pediatric clerkship at a time (M = 20.30, SD = 8.46, minimum = four students, maximum = 50 students). The most common clerkship durations were 6 weeks (40.4%) and 8 weeks (43.6%; M = 6.88, SD = 1.13, minimum = 4 weeks, maximum = 9 weeks).
TABLE 1 Survey respondent characteristics Respondent Characteristics M.D./D.O. Degree Female Academic Rank Instructor Assistant Professor Associate Professor Full Professor Professor Emeritus/Other Board Certified in Pediatrics General Pediatricians Certified in a Subspecialty Area Years Directing the Pediatric Clerkship 10 Years Hours Devoted to the Administrative Responsibility of the Clerkship per Month 20 Hours Respondent Unable to Compute a
N = 95.
Clerkship Directorsa 95 (100%) 50 (52.63%) 0 37 (38.95%) 39 (41.05%) 19 (20.0%) 0 92 (97.87%) 66 (69.47%) 29 (30.53) 7 (7.37%) 19 (20%) 22 (23.16%) 24 (25.26%) 23 (24.21%)
13 (13.68%) 19 (20.0%) 4 (4.21%) 0 57 (60.0%) 2 (2.11%)
368
N. SOARES, Q. WU, S. KANUNGO
TABLE 2 How are developmental-behavioral topics taught in the clerkship?
Clinical Experiences Didactic Lectures Book Chapter/Article Readings Case Readings (CLIPP, etc.) Small-Group Discussion Other (Highlights Listed Below) Developed a Baby 411 Jeopardy! Game and Use COMSEP Problem Sets Online Interactive Video Students Perform Ages and Stages Evaluations in a Clinical Setting Clinical Skills Workshop Practice Developed History With a Standardized Patient Tour of a School for Developmentally Challenged Students a
Frequency
%a
83 71 21 71 40 9 1
89.25% 76.34% 22.58% 76.34% 43.01% 9.68%
1 1
1 1 1
N = 93.
Predominantly, general pediatricians 87.1% (n = 81) were responsible for DBP topics in the clerkship, whereas 52.7% (n = 49) reported that developmental-behavioral pediatricians or neurodevelopmental disabilities physicians participated in the teaching of DBP topics. Others involved were child psychiatrists, child neurologists, and psychologists. Clinical experiences, case readings, and didactic lectures were the primary teaching choices for developmental-behavioral topics (Table 2). However, some innovative approaches were also reported like “Baby 411” Jeopardy! game, online videos, and clinical skills workshop among them. Of the various activities that students were reported to engage in, the most frequent was adolescent HEADSS (Home, Education/Employment, Activities, Drugs, Sexuality, and Suicide/depression) 5 interview (76.7%), followed by a neonatal neuromotor examination (65.5%). Of the various activities that students were reported to observe, the most frequent was counseling on common behavior problems (55.6%), followed by administration of a parent report screening tool (40.7%). Most clerkships did not participate in naturalistic observations in community settings (61%; Table 3). Respondents were asked how they felt their instruction enabled students to achieve the DBP curricula competencies (Table 4). Respondents were more likely to indicate excellent, above average, or average for competencies in medical and developmental activities compared to behavioral and psychosocial activities (McNemar’s test = 3.2667, p = .11), though the difference was not statistically significant. About 18.3% (17/93) of respondents reported not assessing competencies with two
TABLE 3 Student clerkship activities n (%) Denver Development Screening Testa Observe 26 (28.89%) Conduct 24 (26.67%) Don’t Participate 40 (44.44%) Parent Report Screening Toolb Observe 35 (40.70%) Conduct 29 (33.72%) Don’t Participate 22 (25.58%) Neonatal Neuromotor Examc Observe 18 (20.69%) Conduct 57 (65.52%) Don’t Participate 12 (13.79%) Adolescent HEADSS Interviewa Observe 16 (17.78%) Conduct 69 (76.67%) Don’t Participate 5 (5.56%) Preschool/School Evaluationd Observe 23 (26.14%) Conduct 42 (47.73%) Don’t Participate 23 (26.14%) Naturalistic Observations in Community Settingsc Observe 22 (25.29%) Conduct 12 (13.79%) Don’t Participate 53 (60.92%) Counseling on Common Behavior Problemsa Observe 50 (55.56%) Conduct 33 (36.67%) Don’t Participate 7 (7.78%) Note: HEADSS = Home, Education/Employment, Activities, Drugs, Sexuality, and Suicide/depression. a N = 90. bN = 86. cN = 87. dN = 88.
nonrespondents. Posttests and feedback from faculty supervisors were the most commonly reported forms of assessment (Table 5). Of interest, 96% (16/17) of respondents who did not assess competencies indicated that their clerkships followed the COMSEP curriculum. Barriers and Solutions Clerkship directors most frequently report time constraints (61.8%, duration of the clerkship primarily) as the barrier to implementing the COMSEP DBP curriculum, with availability of faculty (27.6%), resources (7.9%, money, materials), and location of clerkship experiences (7.9%) as the other barriers (Table 6). Proposed solutions included resources (48.1%), faculty involvement (27.7%), and curricular elements (20.4%) (see Table 7). Although few reported lack of funds as a barrier, many more suggested it as a solution to alleviate the perceived barriers.
369
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
TABLE 4 How well do you feel your clerkship helps the students achieve these competencies? Competency Developmental Describe the four developmental domains of childhooda Describe how abnormal findings on screening would suggest a diagnosis of developmental delayb Identify normal pattern of behaviors in the developing childa Demonstrate an ability to assess development in pediatric patients using interview and screening toolsc Behavioral and Psychosocial Describe typical presentation of common behavioral problems in different age groupsb Describe the emotional or medical conditions that may affect school performance/peer/family relationshipsb Describe how somatic complaints may represent psychosocial problemsb Describe the types of situations where family pathology contributes to childhood behavior problemsb Identify behavioral and psychosocial problems of childhood using medical history and physical examinationc a
Excellent n (%)
Above Average n (%)
Average n (%)
Below Average n (%)
Poor (%)
16 (17.39%)
30 (32.61%)
45 (48.91%)
1 (1.09%)
0 (0%)
5 (5.49%)
24 (26.37%)
60 (65.93%)
2 (2.20%)
0 (0%)
5 (5.43%)
32 (34.78%)
53 (57.61%)
2 (2.17%)
0 (0%)
5 (5.56%)
23 (25.56%)
48 (53.33%)
13 (14.44%)
4 (4.4%)
19 (20.88%)
59 (64.84%)
9 (9.89%)
0 (0%)
5 (5.49%)
24 (26.37%)
51 (56.04%)
11 (12.09%)
0 (0%)
5 (5.49%)
17 (18.68%)
56 (61.54%)
13 (14.29%)
0 (0%)
7 (7.69%)
18 (19.78%)
52 (57.14%)
14 (15.38%)
0 (0%)
3 (3.33%)
31 (34.44%)
51 (56.67%)
5 (5.56%)
0 (0%)
1 (1.11%)
N = 92. bN = 91. cN = 90.
DISCUSSION This study describes prevailing methods of teaching DBP topics during Pediatric clerkship for medical students. A revision TABLE 5 Assessment of developmental and behavioral competencies of students
Portfolio Entries Case Log Observing Demonstrated Skill (Screening, Exam, Interview) Assessment of Knowledge (Posttest) Feedback From Faculty Supervisors Do Not Assess Competencies a
Frequency
%a
3 39 26
3.23% 41.94% 27.96%
40
43.01%
41
44.09%
17
18.26%
Multiple responses permitted, so totals are greater than 100%.
to the COMSEP curriculum is currently under way; however, it is unclear how much of the current content will change. We intend conducting a follow-up study at least a year after the revision to determine how the clerkship directors are responding to the changes. Until that time, the current study could serve as a baseline. The 66% response rate for this study compares favorably with previously published surveys of clerkship directors.2,6,7 Clerkship directors are predominantly female (continuing an increasing national trend from previous studies).2,8 Increasing number of directors are general pediatricians (69.47% in our study vs. 65% in 2004), and fewer subspecialists (30.53% in our study vs. 35% in 2004). Almost 95% of clerkships report following the COMSEP curriculum, increased from the 2004 study of 84%. The clerkships have approximately the same number of students and duration as in the previous study. Only 50% of DBP faculty are reportedly involved in DBP teaching, whereas general pediatric faculty appear to have most of the responsibility of the clerkship DBP teaching. All medical schools with associated pediatric residency programs are
370
N. SOARES, Q. WU, S. KANUNGO
TABLE 6 Themes of barriers to implementing curriculum Theme Time 61.8% (47/76)
Subthemes
Representative Responses
Clerkship time Faculty time
Faculty Availability & Training 27.6% (21 /76)
General faculty Specialty faculty
Resources/Tools 7.9% (6/76)
Resources
Location/Site 7.9% (6/76)
Funding Amount
Variability Curricular Elements 2.6% (2/76)
Lack of awareness
• Short clerkship length • Time during our 6-week rotation • Time constraints • Takes time to do it • Time limited • Not enough experienced clinicians available to teach • Paucity of faculty • We lack a neurodevelopmental specialist that dedicates time to students within the clinics • Small DBP department and difficult to ensure exposure of all students to clinical behavior issues • Lack of resources • Assessment tools • Lack of funding • Multiple outpatient sites which is where most of the appropriate clinical exposure would take place • Lack of primary care site • Diversity of experiences at distant sites • Variations in screening practices across sites • I am not aware of this curriculum • Need more information regarding it
Note: N = 76. Multiple themes identified for some respondents, totals more than 100%. DBP = developmental and behavioral pediatric.
required to have at least one faculty member with subspecialty DBP training, board certification, and expertise in DBP topics for residency accreditation. Engaging DBP faculty in medical student education is reported as a potential barrier, possibly due to time constraints and pediatric resident mandatory DBP teaching responsibilities. The Society for Developmental and Behavioral Pediatrics is placing more emphasis on DBP physician engagement in undergraduate medical education. As expected, didactics, case discussions and clinical encounters were the primary ways to instruct, though innovative practices such as web portals and interactive activities were also reported. There are 32 Computer-assisted Learning in Pediatrics Program9 cases that are used in instruction, but only two cases cover developmental (#28: an 18-month-old with developmental delay) and behavioral (#4: an 8-year-old with attention deficit hyperactivity disorder) topics. Clinicians may enhance engagement in DBP medical student education through collaboration in the development of Computer-assisted Learning in Pediatrics Program cases that cover common developmental-behavioral topics like temper tantrums and elimination problems. E-learning10 promises a delivery system for DBP teaching, and modalities such as webcasts allow for archival of materials for later access. Video conferencing (tele-education) can be conducted through “live” streamed lectures or clinical observations of patients through a telehealth modality.11 These strategies enable students at off-site locations for clerkships to avail of the same
learning opportunities as those at primary sites. We note that parent report tools such as Parents’ Evaluation of Developmental Status12 and Ages and Stages Questionnaire13 are being observed and conducted more than traditional Denver Developmental Screening Test.14 This could be due to the fact that the parent report tools have reasonable validity and reliability15 and are more convenient to administer than the Denver Developmental Screening Test. However, there is no study to quantify or validate this shift in practice or to explore and confirm the reasons. For this study, we chose to focus on only those developmental and behavioral competencies regarded as “Universal” and “Core Pediatric,” which are relevant throughout the medical school curriculum and essential for the pediatric clerkship, respectively.3 Most respondents reported that their students were at least average or better in acquiring the competencies related to developmental topics, but less so for behavioral topics. One explanation could be the increasing trend of pediatricians’ use of developmental screening tools16 and the comfort level of academicians with teaching developmental milestones. Another explanation could be the perceived barriers to identification and management of psychosocial issues in pediatrics particularly by general pediatricians.17 Some other explanations could be limited use of standardized instruments, lack of behavioral evaluation training, limited time, and poor reimbursement for behavioral screening.18
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
371
TABLE 7 Themes of resources to address barriers Theme Time 22.2% (12/54)
Subthemes
Representative Responses
Increased overall time
Redistribution of time
Faculty Involvement 27.7% (15/54)
General faculty
Specialty faculty
Resources 48.1% (26/54)
Materials
Monetary
Curricular Elements 20.4% (11/54)
Clinical curricular elements
Other curricular elements
• Increase the length of the clerkship • Adding 2 more weeks to the rotation • More time • More general pediatrics time • Increase the amount of time for medical students on outpatient • Experienced faculty • Encourage our general pediatrics to teach in the outpatient setting • Faculty participation • Getting community ambulatory preceptors to actively ensure this content is covered • Willingness of DBP faculty (or ancillary staff) to shift efforts from residents to med student core clerkship • A DBP training program • A full time developmental pediatrician • Self-study modules • CLIPP cases, simulations • Online learning modules • Funds for community preceptors • Money • Funding for CLIPP • Protected time for teachers • Fun ways to introduce DB issues on different areas of the rotation • Dependable clinical experiences • More integration of development into general pediatrics • Specific strategies for implementing this that have been successful at other institutions • Short educational package of information developed for students • Robust online curriculum
Note: N = 54. Multiple themes identified for some respondents, totals are more than 100%. DBP = developmental and behavioral pediatric; CLIPP = Computer-assisted Learning in Pediatrics Program.
Assessment of competencies is a key component of the COMSEP curriculum and strategies for assessment are outlined in the curriculum.3 Almost 18% of respondents did not assess DBP competencies, and most of these reported following the COMSEP curriculum, but this study was not designed to identify barriers to assessment. After introduction of the current curriculum in 2005, suggestions were made regarding possible solutions to anticipated implementation barriers.19 In our study, majority of clerkship directors felt that inadequate time (both the duration of the clerkship and time for faculty to provide instruction) was a barrier to implementing the DBP curriculum. Although it is hard to increase clerkship time relative to the
other clinical experiences that medical students need, “time” could be a proxy for faculty availability and time protected for educational activities. In many institutions, billable clinical activities seem to take precedence over clerkship didactic instruction, especially in the current healthcare economic environment. Strategies like “educational value units” could be used to fairly measure clinician-educators teaching activities.20 Without that, faculty may not be inclined to teach as it compromises their clinical productivity, which is often linked to their compensation. Apart from protecting time, faculty development may also be necessary for both general and DBP clinicians to deliver DBP content to medical students, which necessitates a focus
372
N. SOARES, Q. WU, S. KANUNGO
on typical development and basic variant processes, compared to the teaching needs of pediatric residents and DBP fellows. Most DBP teaching appears to occur in outpatient and inpatient medical settings; however, naturalistic observations of children occur best in community settings like schools and child care locations. A shift in paradigm with emphasis on these sites for student learning experiences will be needed. Directors also cited lack of materials and instructional resources, both for teaching and assessment. National organizations like COMSEP and Society for Developmental and Behavioral Pediatrics could serve as portals for sharing information both at conferences and on the Internet. The Association of American Medical Colleges’ MedEd Portal21 is a suggested venue that facilitates the open exchange of peer-reviewed teaching and assessment resources. Limitations We chose to focus only on the primary clerkship director for each of the COMSEP member programs, so our findings may not reflect the views of nondirectors and others involved in undergraduate pediatric medical education. It is possible that nondirectors’ perspectives could provide valuable information on barriers to implementing the curriculum. Also, there may be a response bias when asking directors to reflect on their students’ abilities to achieve the COMSEP competencies. Perhaps a concurrent survey of the medical students and directors/educators at individual medical schools eliciting responses on competency achievement could have mitigated this bias. However, this was beyond the scope of the current study. SUMMARY This study provides the first glimpse into how developmental and behavioral pediatric topics are taught and assessed during the medical student pediatric clerkship. With general pediatricians mostly involved in clerkship DBP teaching, collaboration with DBP clinicians for content creation, faculty development, and utilization of electronic resources will be necessary. Utilizing community-based naturalistic opportunities for learning normal child development and behavior, and instructional and assessment strategies shared through organization portals could be used to help clerkship directors achieve the COMSEP curriculum competencies. Future explorations of the outcomes of these strategies, especially after adoption of the next version of the COMSEP curriculum will add to the findings of the current study. SUPPLEMENTAL MATERIAL Supplemental data for this article can be accessed on the publisher’s website at http://dx.doi.org/10.1080/10401334. 2014.945392.
REFERENCES 1. Liaison Committee on Medical Education. Available at: http://www. lcme.org/. Accessed May 15, 2012. 2. White CB, Waller JL, Freed G, Levine DA, Moore RS, Sharkey A, et al. The state of undergraduate pediatric medical education in North America: The COMSEP survey. Teaching and Learning in Medicine 2007;19:264–70. 3. Council on Medical Student Education in Pediatrics. Available at: http://www.comsep.org/educationalresources/curindex.cfm Accessed March 12, 2013. 4. Stokes ME, Davis CS, Koch GG. Categorical data analysis using the SAS system. Cary, NC: SAS Institute, 1995. 5. Goldenring JM, Cohen E. Getting into adolescent heads. Contemporary Pediatrics 1998;5:75. 6. Li ST, Smith S, Gigante JA. National survey of Pediatric Clerkship Directors’ approaches to meeting the LCME ED-2 requirement for quantified patient criteria for medical students. Teaching and Learning in Medicine 2007;19;4352–6. 7. Kind T, Greysen R, Chretien K. Pediatric clerkship directors’ social networking use and perceptions of online professionalism. Academic Pediatrics 2012;12:142–8. 8. Greenberg L, Sahler OJ, Siegel B, Sarkin R, Sharkey SA. The pediatric clerkship director: Support systems, professional development, and academic credentials. Archives of Pediatric & Adolescent Medicine 1995;149:916–20. 9. Institute for Innovative Technology in Medical Education. Computerassisted Learning in Pediatrics Project (CLIPP). Available at: http://www.med-u.org/virtual patient cases/clipp. Accessed May 15, 2012. 10. Kim S. The future of e-learning in medical education: Current trend and future opportunity. Journal of Educational Evaluation for Health Professions 2006;3:3. 11. Soares NS, Langkamp DL. Telehealth in developmental-behavioral pediatrics. Journal of Developmental and Behavioral Pediatrics. 2012;338:656–65. 12. Glascoe FP. PEDS: Parents’ Evaluation of Developmental Status. Nolensville, TN: PEDS Test.com, LLC. R 13. Squires J, Bricker D. Ages & Stages Questionnaires , Third edition (ASQ-3TM): A Parent-Completed Child-Monitoring System. Baltimore, MD: Brookes. 14. Frankenburg WK, Dodds J, Archer P. Denver–II Developmental Materials, Inc. Denver, CO, 1992. 15. Limbos MM, Joyce DP. Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. Journal of Developmental and Behavioral Pediatrics 2011;32:499–511. 16. Radecki L, Sand-Loud N, O’Connor KG, Sharp S, Olson LM. Trends in the use of standardized tools for developmental screening in early childhood: 2002–2009. Pediatrics 2011;128:14–9. 17. Horwitz SM, Kelleher KJ, Stein REK, Storfer-Isser A, Youngstrom EA, Park ER, et al. Barriers to the identification and management of psychosocial issues in children and maternal depression. Pediatrics 2007;119:e208–18. 18. Weitzman CC, Leventhal JM. Screening for behavioral health problems in primary care. Current Opinion in Pediatrics 2006;18:641–8. 19. Olson AL. Implementation of the new General Pediatrics Core Curriculum: Possible barriers and strategies to overcome them. Available at: http://comsep.org/educationalresources/viewStrategy.cfm?id=2. Accessed March 10, 2013. 20. Stites S, Vansaghi L, Pingleton S, Cox G, Paolo A. Aligning compensation with education: Design and implementation of the Educational Value Unit (EVU) system in an academic internal medicine department. Academic Medicine 2005;80:1100–6. 21. Association of American Medical Colleges. MedEd Portal. Available at: http://www.mededportal.org/. Accessed March 10, 2013.
RESEARCH BASIC TO TEACHING AND LEARNING Developmental-Behavioral Pediatric Teaching of Medical Students: A National COMSEP Survey Neelkamal Soares Department of Pediatrics, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California, USA
Qishan Wu Department of Biostatistics, University of Kentucky, Lexington, Kentucky, USA
Shibani Kanungo UCLA Intercampus Genetics Training program, Los Angeles, California, USA
Background: The Council on Medical Student Education in Pediatrics (COMSEP) pediatric clerkship curriculum is widely followed. To date, there are no known studies on clerkship instruction related to developmental-behavioral pediatric (DBP) curricular elements. Purposes: The goals of this study are to examine pediatric clerkships’ current DBP teaching methods and to identify barriers and solutions to recommended curriculum implementation. Methods: Electronic survey was conducted with COMSEPmember pediatric clerkship directors. Descriptive statistics and qualitative data analysis was conducted. Results: Response rate was 66%. General Pediatricians (87.1%) were mostly responsible for clerkship DBP teaching. Around 18% of directors reported not assessing DBP competencies. Most clerkship directors report time constraints (61.8%) as a barrier to implementing the curriculum, along with faculty availability and resources. Suggested solutions included DBP faculty collaboration and resources. Conclusions: General pediatricians should collaborate with DBP faculty for instructional content creation, and community-based observational opportunities and web-based shared resources could help clerkship directors achieve the COMSEP DBP curriculum competencies. Keywords
developmental-behavioral pediatrics, medical student education, curriculum
BACKGROUND The Liaison Committee on Medical Education (LCME) is the nationally recognized accrediting authority for medical
We thank the COMSEP Annual Survey Committee for inclusion of our questionnaire in their annual survey. We acknowledge Sam Zinner, Carol Weitzman, and Franklin Trimm for their inspiration, and Linda Nield and Sherilyn Smith for the manuscript review. Correspondence may be sent to Neelkamal Soares, 120 Hamm Drive, Suite 2A, Lewisburg, PA 17837, USA. E-mail: nssoares@ geisinger.edu
education programs at the U.S. and Canadian medical schools represented by the Association of American Medical Colleges.1 LCME recommends clinical experiences (clerkships) in pediatrics, family medicine, internal medicine, obstetrics and gynecology, psychiatry, and surgery, which usually occur in the 3rd year of undergraduate medical education. LCME also wants clerkship directors to actively manage their curricula, develop competency-based learning objectives, and monitor students’ progress in the clerkship.2 The Council on Medical Student Education in Pediatrics (COMSEP) is composed of pediatric clerkship directors and others interested in medical student education in pediatrics. In association with the Academic Pediatric Association (formerly the Ambulatory Pediatric Association), COMSEP developed a national pediatric clerkship curriculum in 1995, which has been revised most recently in 2005.3 The curriculum, which has been adopted by more than 90% of pediatric clerkships, aims to provide an educational roadmap for medical student education in pediatrics, not just in the clerkship year but throughout the entire medical school experience. Foundational knowledge of typical and atypical child development and behavior and of psychosocial frameworks are important to understanding pathological processes in childhood growth, learning, and general health. Ultimately this prepares future clinicians to engage families with children as partners for effective healthcare management. In contrast to postgraduate pediatric education, training during the undergraduate pediatric clerkship has no mandatory developmental-behavioral pediatric (DBP) component. Pediatric clerkships provide opportunities for students to participate in clinic-based and community-based DBP learning opportunities. To date, the extent and nature of these opportunities have not been published. The COMSEP curriculum delineates the expected competencies in DBP topics upon completion of the pediatrics clerkship, with suggested
366
367
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
strategies in instruction and in evaluating student performance.3 To our knowledge, there are no studies that determine if, and how, programs are implementing the COMSEP curriculum to achieve the delineated competencies and barriers these programs face in the process.
PURPOSE The goals of this study are to examine pediatric clerkships’ DBP teaching methods and assessment of suggested DBP curricular competencies, to identify potential barriers to implementing the recommended curriculum, and to explore strategies to address the barriers.
METHODS In the fall of 2010, COMSEP conducted a survey of its membership, consisting of leaders of pediatric education for medical students, including clerkship directors, site directors, academic pediatric deans, and other pediatric faculty interested in medical student education. The confidential electronic survey consisted of 60 questions from five investigator teams (including ours). The first 20 questions elicited general demographic information. The other investigators were studying social networking use, professionalism remediation, expectations about oral case presentations, and educational impact of family-centered rounds. COMSEP members were electronically invited to complete the survey, and nonresponders were sent a maximum of three reminders at 4-week intervals from September 2010 to December 2010. Survey participants’ deidentified responses were saved in a confidential database, and no incentives were provided to participate. Approval was obtained from both Institutional Review Boards at the University of Kentucky and the University of Washington, where the final survey was compiled. Our DBP component included 17 multiple-choice questions and elicited binomial and qualitative responses about methods of instruction, assessment, respondent perception of students’ competencies, and perceived barriers to implementing the DBP curriculum (see the supplementary file). All statistical analyses were done using SAS 9.3 (SAS Institute Inc, Cary, NC). Descriptive statistical methods were used to compute all frequency tables for categorical variables and mean, standard deviation, minimum, and maximum for continuous variables. McNemar’s test4 was used to analyze if developmental competencies and behavioral competencies have different patterns of attainment achievement (excellent, above average, average, below average, and poor). Qualitative analysis of barriers and suggested resources was performed independently by two authors (NS and SK) with absolute agreement on the categories. Descriptive statistics with frequency counts (number and percentages) were conducted.
RESULTS Respondent Characteristics Although the survey was disseminated to all COMSEP members, our study focused only on pediatric clerkship directors. Response rate for the clerkship directors was 66% (95/144), 53% (n = 50) were female. The predominant respondents were M.D./D.O. board certified in pediatrics, and the majority were general pediatricians, at either Associate or Assistant Professor rank, almost half of whom had been directing the clerkship for more than 5 years (Table 1). Clerkship Characteristics Among respondents, 93.7% (n = 89) indicated that their clerkships followed the COMSEP curriculum. There were an average of 20 medical students on the pediatric clerkship at a time (M = 20.30, SD = 8.46, minimum = four students, maximum = 50 students). The most common clerkship durations were 6 weeks (40.4%) and 8 weeks (43.6%; M = 6.88, SD = 1.13, minimum = 4 weeks, maximum = 9 weeks).
TABLE 1 Survey respondent characteristics Respondent Characteristics M.D./D.O. Degree Female Academic Rank Instructor Assistant Professor Associate Professor Full Professor Professor Emeritus/Other Board Certified in Pediatrics General Pediatricians Certified in a Subspecialty Area Years Directing the Pediatric Clerkship 10 Years Hours Devoted to the Administrative Responsibility of the Clerkship per Month 20 Hours Respondent Unable to Compute a
N = 95.
Clerkship Directorsa 95 (100%) 50 (52.63%) 0 37 (38.95%) 39 (41.05%) 19 (20.0%) 0 92 (97.87%) 66 (69.47%) 29 (30.53) 7 (7.37%) 19 (20%) 22 (23.16%) 24 (25.26%) 23 (24.21%)
13 (13.68%) 19 (20.0%) 4 (4.21%) 0 57 (60.0%) 2 (2.11%)
368
N. SOARES, Q. WU, S. KANUNGO
TABLE 2 How are developmental-behavioral topics taught in the clerkship?
Clinical Experiences Didactic Lectures Book Chapter/Article Readings Case Readings (CLIPP, etc.) Small-Group Discussion Other (Highlights Listed Below) Developed a Baby 411 Jeopardy! Game and Use COMSEP Problem Sets Online Interactive Video Students Perform Ages and Stages Evaluations in a Clinical Setting Clinical Skills Workshop Practice Developed History With a Standardized Patient Tour of a School for Developmentally Challenged Students a
Frequency
%a
83 71 21 71 40 9 1
89.25% 76.34% 22.58% 76.34% 43.01% 9.68%
1 1
1 1 1
N = 93.
Predominantly, general pediatricians 87.1% (n = 81) were responsible for DBP topics in the clerkship, whereas 52.7% (n = 49) reported that developmental-behavioral pediatricians or neurodevelopmental disabilities physicians participated in the teaching of DBP topics. Others involved were child psychiatrists, child neurologists, and psychologists. Clinical experiences, case readings, and didactic lectures were the primary teaching choices for developmental-behavioral topics (Table 2). However, some innovative approaches were also reported like “Baby 411” Jeopardy! game, online videos, and clinical skills workshop among them. Of the various activities that students were reported to engage in, the most frequent was adolescent HEADSS (Home, Education/Employment, Activities, Drugs, Sexuality, and Suicide/depression) 5 interview (76.7%), followed by a neonatal neuromotor examination (65.5%). Of the various activities that students were reported to observe, the most frequent was counseling on common behavior problems (55.6%), followed by administration of a parent report screening tool (40.7%). Most clerkships did not participate in naturalistic observations in community settings (61%; Table 3). Respondents were asked how they felt their instruction enabled students to achieve the DBP curricula competencies (Table 4). Respondents were more likely to indicate excellent, above average, or average for competencies in medical and developmental activities compared to behavioral and psychosocial activities (McNemar’s test = 3.2667, p = .11), though the difference was not statistically significant. About 18.3% (17/93) of respondents reported not assessing competencies with two
TABLE 3 Student clerkship activities n (%) Denver Development Screening Testa Observe 26 (28.89%) Conduct 24 (26.67%) Don’t Participate 40 (44.44%) Parent Report Screening Toolb Observe 35 (40.70%) Conduct 29 (33.72%) Don’t Participate 22 (25.58%) Neonatal Neuromotor Examc Observe 18 (20.69%) Conduct 57 (65.52%) Don’t Participate 12 (13.79%) Adolescent HEADSS Interviewa Observe 16 (17.78%) Conduct 69 (76.67%) Don’t Participate 5 (5.56%) Preschool/School Evaluationd Observe 23 (26.14%) Conduct 42 (47.73%) Don’t Participate 23 (26.14%) Naturalistic Observations in Community Settingsc Observe 22 (25.29%) Conduct 12 (13.79%) Don’t Participate 53 (60.92%) Counseling on Common Behavior Problemsa Observe 50 (55.56%) Conduct 33 (36.67%) Don’t Participate 7 (7.78%) Note: HEADSS = Home, Education/Employment, Activities, Drugs, Sexuality, and Suicide/depression. a N = 90. bN = 86. cN = 87. dN = 88.
nonrespondents. Posttests and feedback from faculty supervisors were the most commonly reported forms of assessment (Table 5). Of interest, 96% (16/17) of respondents who did not assess competencies indicated that their clerkships followed the COMSEP curriculum. Barriers and Solutions Clerkship directors most frequently report time constraints (61.8%, duration of the clerkship primarily) as the barrier to implementing the COMSEP DBP curriculum, with availability of faculty (27.6%), resources (7.9%, money, materials), and location of clerkship experiences (7.9%) as the other barriers (Table 6). Proposed solutions included resources (48.1%), faculty involvement (27.7%), and curricular elements (20.4%) (see Table 7). Although few reported lack of funds as a barrier, many more suggested it as a solution to alleviate the perceived barriers.
369
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
TABLE 4 How well do you feel your clerkship helps the students achieve these competencies? Competency Developmental Describe the four developmental domains of childhooda Describe how abnormal findings on screening would suggest a diagnosis of developmental delayb Identify normal pattern of behaviors in the developing childa Demonstrate an ability to assess development in pediatric patients using interview and screening toolsc Behavioral and Psychosocial Describe typical presentation of common behavioral problems in different age groupsb Describe the emotional or medical conditions that may affect school performance/peer/family relationshipsb Describe how somatic complaints may represent psychosocial problemsb Describe the types of situations where family pathology contributes to childhood behavior problemsb Identify behavioral and psychosocial problems of childhood using medical history and physical examinationc a
Excellent n (%)
Above Average n (%)
Average n (%)
Below Average n (%)
Poor (%)
16 (17.39%)
30 (32.61%)
45 (48.91%)
1 (1.09%)
0 (0%)
5 (5.49%)
24 (26.37%)
60 (65.93%)
2 (2.20%)
0 (0%)
5 (5.43%)
32 (34.78%)
53 (57.61%)
2 (2.17%)
0 (0%)
5 (5.56%)
23 (25.56%)
48 (53.33%)
13 (14.44%)
4 (4.4%)
19 (20.88%)
59 (64.84%)
9 (9.89%)
0 (0%)
5 (5.49%)
24 (26.37%)
51 (56.04%)
11 (12.09%)
0 (0%)
5 (5.49%)
17 (18.68%)
56 (61.54%)
13 (14.29%)
0 (0%)
7 (7.69%)
18 (19.78%)
52 (57.14%)
14 (15.38%)
0 (0%)
3 (3.33%)
31 (34.44%)
51 (56.67%)
5 (5.56%)
0 (0%)
1 (1.11%)
N = 92. bN = 91. cN = 90.
DISCUSSION This study describes prevailing methods of teaching DBP topics during Pediatric clerkship for medical students. A revision TABLE 5 Assessment of developmental and behavioral competencies of students
Portfolio Entries Case Log Observing Demonstrated Skill (Screening, Exam, Interview) Assessment of Knowledge (Posttest) Feedback From Faculty Supervisors Do Not Assess Competencies a
Frequency
%a
3 39 26
3.23% 41.94% 27.96%
40
43.01%
41
44.09%
17
18.26%
Multiple responses permitted, so totals are greater than 100%.
to the COMSEP curriculum is currently under way; however, it is unclear how much of the current content will change. We intend conducting a follow-up study at least a year after the revision to determine how the clerkship directors are responding to the changes. Until that time, the current study could serve as a baseline. The 66% response rate for this study compares favorably with previously published surveys of clerkship directors.2,6,7 Clerkship directors are predominantly female (continuing an increasing national trend from previous studies).2,8 Increasing number of directors are general pediatricians (69.47% in our study vs. 65% in 2004), and fewer subspecialists (30.53% in our study vs. 35% in 2004). Almost 95% of clerkships report following the COMSEP curriculum, increased from the 2004 study of 84%. The clerkships have approximately the same number of students and duration as in the previous study. Only 50% of DBP faculty are reportedly involved in DBP teaching, whereas general pediatric faculty appear to have most of the responsibility of the clerkship DBP teaching. All medical schools with associated pediatric residency programs are
370
N. SOARES, Q. WU, S. KANUNGO
TABLE 6 Themes of barriers to implementing curriculum Theme Time 61.8% (47/76)
Subthemes
Representative Responses
Clerkship time Faculty time
Faculty Availability & Training 27.6% (21 /76)
General faculty Specialty faculty
Resources/Tools 7.9% (6/76)
Resources
Location/Site 7.9% (6/76)
Funding Amount
Variability Curricular Elements 2.6% (2/76)
Lack of awareness
• Short clerkship length • Time during our 6-week rotation • Time constraints • Takes time to do it • Time limited • Not enough experienced clinicians available to teach • Paucity of faculty • We lack a neurodevelopmental specialist that dedicates time to students within the clinics • Small DBP department and difficult to ensure exposure of all students to clinical behavior issues • Lack of resources • Assessment tools • Lack of funding • Multiple outpatient sites which is where most of the appropriate clinical exposure would take place • Lack of primary care site • Diversity of experiences at distant sites • Variations in screening practices across sites • I am not aware of this curriculum • Need more information regarding it
Note: N = 76. Multiple themes identified for some respondents, totals more than 100%. DBP = developmental and behavioral pediatric.
required to have at least one faculty member with subspecialty DBP training, board certification, and expertise in DBP topics for residency accreditation. Engaging DBP faculty in medical student education is reported as a potential barrier, possibly due to time constraints and pediatric resident mandatory DBP teaching responsibilities. The Society for Developmental and Behavioral Pediatrics is placing more emphasis on DBP physician engagement in undergraduate medical education. As expected, didactics, case discussions and clinical encounters were the primary ways to instruct, though innovative practices such as web portals and interactive activities were also reported. There are 32 Computer-assisted Learning in Pediatrics Program9 cases that are used in instruction, but only two cases cover developmental (#28: an 18-month-old with developmental delay) and behavioral (#4: an 8-year-old with attention deficit hyperactivity disorder) topics. Clinicians may enhance engagement in DBP medical student education through collaboration in the development of Computer-assisted Learning in Pediatrics Program cases that cover common developmental-behavioral topics like temper tantrums and elimination problems. E-learning10 promises a delivery system for DBP teaching, and modalities such as webcasts allow for archival of materials for later access. Video conferencing (tele-education) can be conducted through “live” streamed lectures or clinical observations of patients through a telehealth modality.11 These strategies enable students at off-site locations for clerkships to avail of the same
learning opportunities as those at primary sites. We note that parent report tools such as Parents’ Evaluation of Developmental Status12 and Ages and Stages Questionnaire13 are being observed and conducted more than traditional Denver Developmental Screening Test.14 This could be due to the fact that the parent report tools have reasonable validity and reliability15 and are more convenient to administer than the Denver Developmental Screening Test. However, there is no study to quantify or validate this shift in practice or to explore and confirm the reasons. For this study, we chose to focus on only those developmental and behavioral competencies regarded as “Universal” and “Core Pediatric,” which are relevant throughout the medical school curriculum and essential for the pediatric clerkship, respectively.3 Most respondents reported that their students were at least average or better in acquiring the competencies related to developmental topics, but less so for behavioral topics. One explanation could be the increasing trend of pediatricians’ use of developmental screening tools16 and the comfort level of academicians with teaching developmental milestones. Another explanation could be the perceived barriers to identification and management of psychosocial issues in pediatrics particularly by general pediatricians.17 Some other explanations could be limited use of standardized instruments, lack of behavioral evaluation training, limited time, and poor reimbursement for behavioral screening.18
DEVELOPMENTAL-BEHAVIORAL PEDIATRICS IN CLERKSHIP
371
TABLE 7 Themes of resources to address barriers Theme Time 22.2% (12/54)
Subthemes
Representative Responses
Increased overall time
Redistribution of time
Faculty Involvement 27.7% (15/54)
General faculty
Specialty faculty
Resources 48.1% (26/54)
Materials
Monetary
Curricular Elements 20.4% (11/54)
Clinical curricular elements
Other curricular elements
• Increase the length of the clerkship • Adding 2 more weeks to the rotation • More time • More general pediatrics time • Increase the amount of time for medical students on outpatient • Experienced faculty • Encourage our general pediatrics to teach in the outpatient setting • Faculty participation • Getting community ambulatory preceptors to actively ensure this content is covered • Willingness of DBP faculty (or ancillary staff) to shift efforts from residents to med student core clerkship • A DBP training program • A full time developmental pediatrician • Self-study modules • CLIPP cases, simulations • Online learning modules • Funds for community preceptors • Money • Funding for CLIPP • Protected time for teachers • Fun ways to introduce DB issues on different areas of the rotation • Dependable clinical experiences • More integration of development into general pediatrics • Specific strategies for implementing this that have been successful at other institutions • Short educational package of information developed for students • Robust online curriculum
Note: N = 54. Multiple themes identified for some respondents, totals are more than 100%. DBP = developmental and behavioral pediatric; CLIPP = Computer-assisted Learning in Pediatrics Program.
Assessment of competencies is a key component of the COMSEP curriculum and strategies for assessment are outlined in the curriculum.3 Almost 18% of respondents did not assess DBP competencies, and most of these reported following the COMSEP curriculum, but this study was not designed to identify barriers to assessment. After introduction of the current curriculum in 2005, suggestions were made regarding possible solutions to anticipated implementation barriers.19 In our study, majority of clerkship directors felt that inadequate time (both the duration of the clerkship and time for faculty to provide instruction) was a barrier to implementing the DBP curriculum. Although it is hard to increase clerkship time relative to the
other clinical experiences that medical students need, “time” could be a proxy for faculty availability and time protected for educational activities. In many institutions, billable clinical activities seem to take precedence over clerkship didactic instruction, especially in the current healthcare economic environment. Strategies like “educational value units” could be used to fairly measure clinician-educators teaching activities.20 Without that, faculty may not be inclined to teach as it compromises their clinical productivity, which is often linked to their compensation. Apart from protecting time, faculty development may also be necessary for both general and DBP clinicians to deliver DBP content to medical students, which necessitates a focus
372
N. SOARES, Q. WU, S. KANUNGO
on typical development and basic variant processes, compared to the teaching needs of pediatric residents and DBP fellows. Most DBP teaching appears to occur in outpatient and inpatient medical settings; however, naturalistic observations of children occur best in community settings like schools and child care locations. A shift in paradigm with emphasis on these sites for student learning experiences will be needed. Directors also cited lack of materials and instructional resources, both for teaching and assessment. National organizations like COMSEP and Society for Developmental and Behavioral Pediatrics could serve as portals for sharing information both at conferences and on the Internet. The Association of American Medical Colleges’ MedEd Portal21 is a suggested venue that facilitates the open exchange of peer-reviewed teaching and assessment resources. Limitations We chose to focus only on the primary clerkship director for each of the COMSEP member programs, so our findings may not reflect the views of nondirectors and others involved in undergraduate pediatric medical education. It is possible that nondirectors’ perspectives could provide valuable information on barriers to implementing the curriculum. Also, there may be a response bias when asking directors to reflect on their students’ abilities to achieve the COMSEP competencies. Perhaps a concurrent survey of the medical students and directors/educators at individual medical schools eliciting responses on competency achievement could have mitigated this bias. However, this was beyond the scope of the current study. SUMMARY This study provides the first glimpse into how developmental and behavioral pediatric topics are taught and assessed during the medical student pediatric clerkship. With general pediatricians mostly involved in clerkship DBP teaching, collaboration with DBP clinicians for content creation, faculty development, and utilization of electronic resources will be necessary. Utilizing community-based naturalistic opportunities for learning normal child development and behavior, and instructional and assessment strategies shared through organization portals could be used to help clerkship directors achieve the COMSEP curriculum competencies. Future explorations of the outcomes of these strategies, especially after adoption of the next version of the COMSEP curriculum will add to the findings of the current study. SUPPLEMENTAL MATERIAL Supplemental data for this article can be accessed on the publisher’s website at http://dx.doi.org/10.1080/10401334. 2014.945392.
REFERENCES 1. Liaison Committee on Medical Education. Available at: http://www. lcme.org/. Accessed May 15, 2012. 2. White CB, Waller JL, Freed G, Levine DA, Moore RS, Sharkey A, et al. The state of undergraduate pediatric medical education in North America: The COMSEP survey. Teaching and Learning in Medicine 2007;19:264–70. 3. Council on Medical Student Education in Pediatrics. Available at: http://www.comsep.org/educationalresources/curindex.cfm Accessed March 12, 2013. 4. Stokes ME, Davis CS, Koch GG. Categorical data analysis using the SAS system. Cary, NC: SAS Institute, 1995. 5. Goldenring JM, Cohen E. Getting into adolescent heads. Contemporary Pediatrics 1998;5:75. 6. Li ST, Smith S, Gigante JA. National survey of Pediatric Clerkship Directors’ approaches to meeting the LCME ED-2 requirement for quantified patient criteria for medical students. Teaching and Learning in Medicine 2007;19;4352–6. 7. Kind T, Greysen R, Chretien K. Pediatric clerkship directors’ social networking use and perceptions of online professionalism. Academic Pediatrics 2012;12:142–8. 8. Greenberg L, Sahler OJ, Siegel B, Sarkin R, Sharkey SA. The pediatric clerkship director: Support systems, professional development, and academic credentials. Archives of Pediatric & Adolescent Medicine 1995;149:916–20. 9. Institute for Innovative Technology in Medical Education. Computerassisted Learning in Pediatrics Project (CLIPP). Available at: http://www.med-u.org/virtual patient cases/clipp. Accessed May 15, 2012. 10. Kim S. The future of e-learning in medical education: Current trend and future opportunity. Journal of Educational Evaluation for Health Professions 2006;3:3. 11. Soares NS, Langkamp DL. Telehealth in developmental-behavioral pediatrics. Journal of Developmental and Behavioral Pediatrics. 2012;338:656–65. 12. Glascoe FP. PEDS: Parents’ Evaluation of Developmental Status. Nolensville, TN: PEDS Test.com, LLC. R 13. Squires J, Bricker D. Ages & Stages Questionnaires , Third edition (ASQ-3TM): A Parent-Completed Child-Monitoring System. Baltimore, MD: Brookes. 14. Frankenburg WK, Dodds J, Archer P. Denver–II Developmental Materials, Inc. Denver, CO, 1992. 15. Limbos MM, Joyce DP. Comparison of the ASQ and PEDS in screening for developmental delay in children presenting for primary care. Journal of Developmental and Behavioral Pediatrics 2011;32:499–511. 16. Radecki L, Sand-Loud N, O’Connor KG, Sharp S, Olson LM. Trends in the use of standardized tools for developmental screening in early childhood: 2002–2009. Pediatrics 2011;128:14–9. 17. Horwitz SM, Kelleher KJ, Stein REK, Storfer-Isser A, Youngstrom EA, Park ER, et al. Barriers to the identification and management of psychosocial issues in children and maternal depression. Pediatrics 2007;119:e208–18. 18. Weitzman CC, Leventhal JM. Screening for behavioral health problems in primary care. Current Opinion in Pediatrics 2006;18:641–8. 19. Olson AL. Implementation of the new General Pediatrics Core Curriculum: Possible barriers and strategies to overcome them. Available at: http://comsep.org/educationalresources/viewStrategy.cfm?id=2. Accessed March 10, 2013. 20. Stites S, Vansaghi L, Pingleton S, Cox G, Paolo A. Aligning compensation with education: Design and implementation of the Educational Value Unit (EVU) system in an academic internal medicine department. Academic Medicine 2005;80:1100–6. 21. Association of American Medical Colleges. MedEd Portal. Available at: http://www.mededportal.org/. Accessed March 10, 2013.
