Editorial
How Will the Next Accreditation System Affect Ophthalmology Residency Training? Steven J. Gedde, MD - Miami, Florida Susan H. Day, MD - San Francisco, California George B. Bartley, MD - Rochester, Minnesota Years ago, imagine a co-resident who demonstrated suboptimal surgical skills early in residency. In some programs, this person would have received extra attention to bring his or her performance up to a passable level of competence. In others, the resident would have been labeled (and treated) as a future medical ophthalmologist. No longer is such variability in education allowed. How has this change occurred? In the United States, the Accreditation Council for Graduate Medical Education (ACGME) has been accorded the right and responsibilities of overseeing the quality of residency training programs. If a program is accredited, this in essence is a signal to applicants that the education they receive will provide what the trainee needs. The ability to “sit” for a certifying board examination, administered by the American Board of Ophthalmology in our specialty, is linked to completion of residency training in either an ACGME-accredited program or a Canadian program accredited by its system. In the eyes of the ACGME, each trainee deserves an equivalent educational experience; hence, an accredited program cannot “label” a resident as being competent only for a restricted practice mode. And yet, this process-based model (i.e., spend 36 months in various rotations) lacked the assurance that all graduates had accumulated the proper skills and outcomes. Simply doing a minimum number of cases is not the same as demonstrating to educators that outcomes are satisfactory. In part, the shift from a process-based educational model to an outcomes- and competency-based model has been fueled by public expectations. Emphasis on outcomes, patient safety, and quality improvement pervade much of societal demand for changes across all of medicine. These values have been carefully assessed and increasingly integrated into graduate medical education, including residency training in ophthalmology. The initial foray into outcomes measurement was the Outcomes Project, introduced in 1999. Evaluation of resident performance was required within a framework of 6 general domains of competence (patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice). Although it catalyzed many improvements, the Outcomes Project fell short of its goal of creating a robust assessment system that would allow accreditation of training programs based on outcomes.1 In response, in 2009 the ACGME and specialty boards launched the Milestones Project, which has become an integral part of an extensive revision of graduate medical education termed the Next Accreditation System (NAS).
2364
Ó 2013 by the American Academy of Ophthalmology Published by Elsevier Inc.
“Milestones” are observable developmental steps that describe a trajectory of progress from novice (entering resident) to proficient (graduating resident) organized under the 6 competency areas. For example, milestones are arranged into numbered levels for the patient examination, which exists within the competency domain of patient care. In level 1 (expected entry-level performance), the resident describes components of the complete ophthalmic examination, and he or she performs the basic parts of a screening or bedside eye examination without special equipment. In level 2, the resident performs and documents a complete ophthalmic examination targeted to a patient’s ocular complaints and medical condition, and he or she distinguishes between normal and abnormal findings. In level 3, the resident performs a problem-focused examination and documents pertinent positive and negative findings, and he or she consistently identifies common abnormalities and may identify subtle findings. In level 4 (expected performance at graduation), the resident identifies subtle or uncommon findings of common entities and typical or common findings of rarer entities. In level 5 (aspirational performance), the resident incorporates into clinical practice new literature about examination techniques. Each residency program director is responsible to assess and document that a resident is capable of achieving a particular goaldrather than assuming that simply by having completed a rotation, the resident has accomplished that goal. The NAS moves the ACGME from an episodic “biopsy” model, in which compliance is assessed with an ACGME staff site visit with subsequent Residency Review Committee evaluation every 3 to 5 years for most programs, to a continuous accreditation model with annual review of data from each program.2 Most of the data that will be collected in the NAS are already part of the accreditation process. In essence, performance will be reported in “real time” and includes the annual ACGME resident-fellow survey, case log and clinical experience, and the program’s first time pass rate on the American Board of Ophthalmology examination by its graduates. New components for the NAS include a faculty survey and scholarly activity reports by core clinical faculty and residents, as well as educational milestones data from the semiannual evaluation of residents. The other major change in the NAS is implementation of the Clinical Learning Environment Review (CLER) Program. Site visits to sponsoring institutions are planned to occur every 12 to 18 months, with a predominant focus on group meetings with institutional leadership and “walk arounds” to interact with house staff and other members of the medical center. The goal of these visits is to ascertain whether the medical center fosters an educational environment that ISSN 0161-6420/13/$ - see front matter http://dx.doi.org/10.1016/j.ophtha.2013.08.003
Editorial addresses patient safety and quality improvement. In large part, these visits are intended to ensure that individuals within the institution are not only supportive of such an environment, but also are willing to provide resources to achieve such goals. Additionally, the inclusion of residents in patient safety and quality improvement efforts will be assessed, with the inference that involvement during residency results in lifelong participation in such activities. The CLER site visits will only be linked with accreditation of individual programs in the event an egregious issue is identified. The Residency Review Committee will no longer assign a cycle length to programs. All programs, with the exception of newly accredited programs, will have a scheduled selfstudy visit every 10 years. In addition, programs may have focused or diagnostic site visits if the annual data submission suggests a potential problem or if a significant weakness is identified during the CLER site visit. In July 2013, the milestone portion of the NAS was implemented by 7 of 26 core specialties (emergency medicine, internal medicine, neurologic surgery, orthopedic surgery, pediatrics, diagnostic radiology, and urology). Ophthalmology and the other specialties will begin milestones implementation in July 2014. An Ophthalmology Milestone Group led by Anthony Arnold, MD, has developed specialty-specific achievements that residents are expected to demonstrate at established intervals throughout their training. These milestones may be accessed at: http:// www.acgme-nas.org/milestones.html (accessed June 9, 2013). Current residency program directors, as well as peers from the ophthalmology Residency Review Committee and directors from the American Board of Ophthalmology, have had input into their development. Other portions of the NAS have already been initiated, including annual data review and CLER site visits. A pilot study of the milestones process involving >10 ophthalmology residency programs was completed in June 2013. The NAS offers several theoretical advantages over the current ACGME accreditation system. The milestones should ensure that each resident demonstrates readiness for independent practice, and the visits to sponsoring institutions will emphasize that trainees are functioning within a positive and healthy learning environment.2 A primary goal of the NAS is to reduce the current administrative burden involved with the accreditation process, and to allow more time for program
directors and other faculty to interact with residents. The new system will provide programs with national comparative data to judge the progress of their residents. The NAS is designed to facilitate the early identification of deficiencies in both residents and training programs, so that remediation can be swift and effective. It is expected that by relaxing detailed program requirements, the NAS will provide flexibility for programs with high-quality outcomes to innovate. As with any major change, implementation of the NAS will present several challenges. Each program director will need to appoint a clinical competency committee to assist in evaluating each resident’s success in meeting the milestones and his or her progress toward unsupervised practice. Educators must collaborate to develop a portfolio of meaningful assessment tools within the milestones framework at the national and programmatic level. Faculty development in performing milestone assessment will be required, which may represent a significant departure from previous methods of resident evaluation. Continued revision and refinement of the milestones is expected over time. Research is needed to validate the effect of the NAS on training programs and their graduates. All phases of medical training, including medical student education and continuing medical education, will ultimately need to align with the focus on outcomes. Questions exist about how the NAS will affect the training of residents and the daily work of program directors and other faculty.3 In the setting of anticipated cuts in graduate medical education funding and increased demands for patient care by clinical faculty, most training programs will be challenged. The NAS has potential to provide administrative relief from the arduous task of accreditation and allow a greater focus on learning, innovation, and quality care. References 1. Swing SR, Beeson MS, Carraccio C, et al. Educational milestone development in the first 7 specialties to enter the Next Accreditation System. J Grad Med Educ 2013;5:98–106. 2. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation systemdrationale and benefits. N Engl J Med 2012;366:1051–6. 3. Philibert I, Nasca TJ. The next accreditation system: stakeholder expectations and dialogue with the community. J Grad Med Educ 2012;4:276–8.
2365
How Will the Next Accreditation System Affect Ophthalmology Residency Training? Steven J. Gedde, MD - Miami, Florida Susan H. Day, MD - San Francisco, California George B. Bartley, MD - Rochester, Minnesota Years ago, imagine a co-resident who demonstrated suboptimal surgical skills early in residency. In some programs, this person would have received extra attention to bring his or her performance up to a passable level of competence. In others, the resident would have been labeled (and treated) as a future medical ophthalmologist. No longer is such variability in education allowed. How has this change occurred? In the United States, the Accreditation Council for Graduate Medical Education (ACGME) has been accorded the right and responsibilities of overseeing the quality of residency training programs. If a program is accredited, this in essence is a signal to applicants that the education they receive will provide what the trainee needs. The ability to “sit” for a certifying board examination, administered by the American Board of Ophthalmology in our specialty, is linked to completion of residency training in either an ACGME-accredited program or a Canadian program accredited by its system. In the eyes of the ACGME, each trainee deserves an equivalent educational experience; hence, an accredited program cannot “label” a resident as being competent only for a restricted practice mode. And yet, this process-based model (i.e., spend 36 months in various rotations) lacked the assurance that all graduates had accumulated the proper skills and outcomes. Simply doing a minimum number of cases is not the same as demonstrating to educators that outcomes are satisfactory. In part, the shift from a process-based educational model to an outcomes- and competency-based model has been fueled by public expectations. Emphasis on outcomes, patient safety, and quality improvement pervade much of societal demand for changes across all of medicine. These values have been carefully assessed and increasingly integrated into graduate medical education, including residency training in ophthalmology. The initial foray into outcomes measurement was the Outcomes Project, introduced in 1999. Evaluation of resident performance was required within a framework of 6 general domains of competence (patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice). Although it catalyzed many improvements, the Outcomes Project fell short of its goal of creating a robust assessment system that would allow accreditation of training programs based on outcomes.1 In response, in 2009 the ACGME and specialty boards launched the Milestones Project, which has become an integral part of an extensive revision of graduate medical education termed the Next Accreditation System (NAS).
2364
Ó 2013 by the American Academy of Ophthalmology Published by Elsevier Inc.
“Milestones” are observable developmental steps that describe a trajectory of progress from novice (entering resident) to proficient (graduating resident) organized under the 6 competency areas. For example, milestones are arranged into numbered levels for the patient examination, which exists within the competency domain of patient care. In level 1 (expected entry-level performance), the resident describes components of the complete ophthalmic examination, and he or she performs the basic parts of a screening or bedside eye examination without special equipment. In level 2, the resident performs and documents a complete ophthalmic examination targeted to a patient’s ocular complaints and medical condition, and he or she distinguishes between normal and abnormal findings. In level 3, the resident performs a problem-focused examination and documents pertinent positive and negative findings, and he or she consistently identifies common abnormalities and may identify subtle findings. In level 4 (expected performance at graduation), the resident identifies subtle or uncommon findings of common entities and typical or common findings of rarer entities. In level 5 (aspirational performance), the resident incorporates into clinical practice new literature about examination techniques. Each residency program director is responsible to assess and document that a resident is capable of achieving a particular goaldrather than assuming that simply by having completed a rotation, the resident has accomplished that goal. The NAS moves the ACGME from an episodic “biopsy” model, in which compliance is assessed with an ACGME staff site visit with subsequent Residency Review Committee evaluation every 3 to 5 years for most programs, to a continuous accreditation model with annual review of data from each program.2 Most of the data that will be collected in the NAS are already part of the accreditation process. In essence, performance will be reported in “real time” and includes the annual ACGME resident-fellow survey, case log and clinical experience, and the program’s first time pass rate on the American Board of Ophthalmology examination by its graduates. New components for the NAS include a faculty survey and scholarly activity reports by core clinical faculty and residents, as well as educational milestones data from the semiannual evaluation of residents. The other major change in the NAS is implementation of the Clinical Learning Environment Review (CLER) Program. Site visits to sponsoring institutions are planned to occur every 12 to 18 months, with a predominant focus on group meetings with institutional leadership and “walk arounds” to interact with house staff and other members of the medical center. The goal of these visits is to ascertain whether the medical center fosters an educational environment that ISSN 0161-6420/13/$ - see front matter http://dx.doi.org/10.1016/j.ophtha.2013.08.003
Editorial addresses patient safety and quality improvement. In large part, these visits are intended to ensure that individuals within the institution are not only supportive of such an environment, but also are willing to provide resources to achieve such goals. Additionally, the inclusion of residents in patient safety and quality improvement efforts will be assessed, with the inference that involvement during residency results in lifelong participation in such activities. The CLER site visits will only be linked with accreditation of individual programs in the event an egregious issue is identified. The Residency Review Committee will no longer assign a cycle length to programs. All programs, with the exception of newly accredited programs, will have a scheduled selfstudy visit every 10 years. In addition, programs may have focused or diagnostic site visits if the annual data submission suggests a potential problem or if a significant weakness is identified during the CLER site visit. In July 2013, the milestone portion of the NAS was implemented by 7 of 26 core specialties (emergency medicine, internal medicine, neurologic surgery, orthopedic surgery, pediatrics, diagnostic radiology, and urology). Ophthalmology and the other specialties will begin milestones implementation in July 2014. An Ophthalmology Milestone Group led by Anthony Arnold, MD, has developed specialty-specific achievements that residents are expected to demonstrate at established intervals throughout their training. These milestones may be accessed at: http:// www.acgme-nas.org/milestones.html (accessed June 9, 2013). Current residency program directors, as well as peers from the ophthalmology Residency Review Committee and directors from the American Board of Ophthalmology, have had input into their development. Other portions of the NAS have already been initiated, including annual data review and CLER site visits. A pilot study of the milestones process involving >10 ophthalmology residency programs was completed in June 2013. The NAS offers several theoretical advantages over the current ACGME accreditation system. The milestones should ensure that each resident demonstrates readiness for independent practice, and the visits to sponsoring institutions will emphasize that trainees are functioning within a positive and healthy learning environment.2 A primary goal of the NAS is to reduce the current administrative burden involved with the accreditation process, and to allow more time for program
directors and other faculty to interact with residents. The new system will provide programs with national comparative data to judge the progress of their residents. The NAS is designed to facilitate the early identification of deficiencies in both residents and training programs, so that remediation can be swift and effective. It is expected that by relaxing detailed program requirements, the NAS will provide flexibility for programs with high-quality outcomes to innovate. As with any major change, implementation of the NAS will present several challenges. Each program director will need to appoint a clinical competency committee to assist in evaluating each resident’s success in meeting the milestones and his or her progress toward unsupervised practice. Educators must collaborate to develop a portfolio of meaningful assessment tools within the milestones framework at the national and programmatic level. Faculty development in performing milestone assessment will be required, which may represent a significant departure from previous methods of resident evaluation. Continued revision and refinement of the milestones is expected over time. Research is needed to validate the effect of the NAS on training programs and their graduates. All phases of medical training, including medical student education and continuing medical education, will ultimately need to align with the focus on outcomes. Questions exist about how the NAS will affect the training of residents and the daily work of program directors and other faculty.3 In the setting of anticipated cuts in graduate medical education funding and increased demands for patient care by clinical faculty, most training programs will be challenged. The NAS has potential to provide administrative relief from the arduous task of accreditation and allow a greater focus on learning, innovation, and quality care. References 1. Swing SR, Beeson MS, Carraccio C, et al. Educational milestone development in the first 7 specialties to enter the Next Accreditation System. J Grad Med Educ 2013;5:98–106. 2. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation systemdrationale and benefits. N Engl J Med 2012;366:1051–6. 3. Philibert I, Nasca TJ. The next accreditation system: stakeholder expectations and dialogue with the community. J Grad Med Educ 2012;4:276–8.
2365
