Original Article
599
Obstetric Ultrasound Quality Improvement Initiative—Utilization of a Quality Assurance Process and Standardized Checklists Barbora Mrazek-Pugh, RDMS, RVT1
Yair J. Blumenfeld, MD2
1 Lucile Packard Children’s Hospital Stanford, Palo Alto, California 2 Department of Obstetrics and Gynecology, Stanford University
School of Medicine, Stanford, California
Henry C. Lee, MD3
Jane Chueh, MD2
Address for correspondence Barbora Mrazek-Pugh, RDMS, RVT, 725 Welch Road, Suite HF306, Palo Alto, CA 94043 (e-mail: [email protected]).
3 Department of Pediatrics, Stanford University School of Medicine,
Stanford, California
Abstract
Keywords
► ► ► ► ►
ultrasound quality checklists obstetric improvement
Objective Our aim was to assess whether mandated completion of an electronic checklist and a quality assurance (QA) process improved obstetric (OB) ultrasound image documentation. Study Design A checklist of mandated images based on the American Institute of Ultrasound in Medicine guidelines was created. A baseline QA assessment was performed with a lead senior sonographer reviewing eight random OB examinations for each sonographer. An electronic checklist was then instituted for all OB examinations on each ultrasound machine. It was mandated that each anatomical structure be checked off during real-time image acquisition. A repeat QA assessment of each sonographer was then performed quarterly. Results Baseline assessments were performed between September 2011 and November 2011. Out of the 110 examinations analyzed, only 49% were deemed “complete” with none of the sonographers having a 100% complete examination rate. Following institution of the mandated electronic checklist, a repeat assessment revealed an 81% complete examination rate for the next quarter, and 90% were complete at the end of a year. All sonographers improved their image acquisition regardless of baseline skill level at the initial QA. Conclusion A QA process and a mandated standardized electronic checklist improved the image documentation.
Utilization of obstetric (OB) ultrasound is rising, with pregnancies in the United States receiving four to five ultrasound examinations per pregnancy.1 Simultaneously, ultrasound machine technology and complexity have advanced, and requirements for meeting appropriate image acquisition and documentation of OB examinations have increased as well.2–7 Therefore, busy prenatal diagnostic centers find themselves in the precarious position of needing to maximize efficiency while performing ultrasound examinations that may contain over 30 required elements per study.2 Not surprisingly, lawsuit
cases involving OB ultrasound examinations are the most common in all of medical ultrasound.8 Since patient safety is of primary concern, and ultrasound procedures need to be both cost-effective and clinically effective, maintaining assurance of quality in OB ultrasound examinations is warranted. The American Institute of Ultrasound in Medicine (AIUM) “Standards and Guidelines for the Accreditation of Ultrasound Practices” states that “adequate documentation is essential for high-quality patient care. There should be a permanent record of the ultrasound examination and its interpretation.
received December 11, 2014 accepted after revision December 31, 2014 published online March 2, 2015
Copyright © 2015 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.
DOI http://dx.doi.org/ 10.1055/s-0035-1545667. ISSN 0735-1631.
Downloaded by: UPSTATE Medical University. Copyrighted material.
Am J Perinatol 2015;32:599–604.
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist Images of all appropriate areas, both normal and abnormal, should be recorded. Retention of the ultrasound examination should be consistent both with clinical needs and with relevant legal and local health care facility requirements.2 According to the American College of Obstetricians and Gynecologists (ACOG), quality control for ultrasonography in pregnancy should be accomplished through careful record keeping of OB ultrasound examination results, archiving reports and images, and correlation with clinical outcome.9 Several quality assurance (QA) processes have been developed for OB ultrasound examinatiios.10–13 For example, nuchal translucency (NT) measurements require rigorous education, training, and certification, with an ongoing review of image quality and audit of measurement ranges.11–13 Unfortunately, a similar process does not exist for other types of OB ultrasound examinations such as fetal anatomy examinations. The AIUM and other international ultrasound societies have published checklists outlining optimal fetal and maternal anatomical structures required to be visualized and documented during NT examinations, second trimester fetal anatomical surveys, and third trimester examinations.2 In addition the AIUM certification process requires that the fetal biometry be measured on certain standardized planes, and fetal anatomy images be obtained in certain views.2 However, there is currently no standard QA method for OB image quality and documentation for many OB ultrasound examinations, and the utility of AIUM checklists as a means of improving ultrasound documentation is unknown. The aim of our study was to assess whether a QA process and mandated completion of an electronic checklist of required OB (both fetal and maternal) anatomical structures and views, during real-time acquisition, would improve image acquisition and documentation.
Materials and Methods Checklists of mandated images for first trimester, second trimester, and third trimester examinations for fetal assessments based on AIUM guidelines were created (►Table 1).2 The checklists were then instituted in electronic format for the different examination types on each ultrasound machine (General Electric Voluson E8 Expert; GE Healthcare Austria GmbH & Co OG, Zipf, Austria) used at the Lucile Packard Children’s Hospital Stanford (LPCH) Perinatal Diagnostic Center (PDC) and affiliated sites (10 ultrasound machines at 5 sites) (►Fig. 1). This was done using the “scan assistant” feature that was available to program on each ultrasound machine. The use of an electronic checklist, as compared with a paper checklist, was chosen to allow images to be checked during real-time acquisition. Before implementation of routine use of the checklist, a baseline QA assessment of each sonographer employed by the LPCH–PDC and affiliated sites (13 sonographers) was performed. Each sonographer assessed has at least 3 years’ experience performing OB ultrasound examinations, and at most 27 years. All sonographers are NT-certified through either the Nuchal Translucency Quality Review Program or the Fetal Medicine Foundation, American Registry for Diagnostic Medical Sonography registered in OB, and all sites are AIUM accredited.2,13 In American Journal of Perinatology
Vol. 32
No. 6/2015
Mrazek-Pugh et al.
addition, sonographers participate in monthly ultrasound conferences conducted by members of the Division of MaternalFetal Medicine at Stanford University School of Medicine. They also attend two different bimonthly image review multidisciplinary case conferences. For the baseline assessment, a single lead senior sonographer (B. M. P.) reviewed eight random ultrasound examinations (two first-trimester examinations, four second-trimester fetal anatomical surveys, which included one anatomy survey of multiple gestations, and two third-trimester ultrasound examinations) for each sonographer. Images from the entire examination were reviewed and compared with the printed ultrasound report (ClickView Corporation, San Francisco, CA). For the purpose of the QA assessment, an examination was considered “complete” if all images reported as seen in the written report were indeed acquired by the sonographer and stored on the LPCH Picture Archiving and Communication System (PACS). Image quality for the individual fetal or maternal structures was also assessed, specifically for the accurate scanning plane and comprehensiveness of the image. Sonographers were not aware of the initial QA review program and the results of the baseline assessment were reviewed with each sonographer in private. Inadequacies were discussed along with recommendations for improvement. The sonographers were then notified about the institution of the mandated electronic checklist. Following the installation of the mandated electronic checklist on each ultrasound machine, it was required that each anatomical structure listed be checked during real-time acquisition, and not after the completion of the scan. When the examination was completed by the sonographer and before the physician’s review, a “scan assistant” report was printed for presentation to the physician. This report allows the physician to quickly assess which images were not acquired during the examination. Limitations prohibiting adequate image acquisition included technical factors, such as suboptimal fetal position or maternal factors such as obesity, prior cesareans, or other abdominal surgery. Follow-up QA assessments for each sonographer were performed quarterly, and the results were shared with each sonographer and the attending physicians. To assess whether there was a difference over time, we compared the percentage examinations correct across each time period (baseline and Q1–Q4 postchecklist) using Wilcoxon matched-pairs signed-ranks, comparing each time period to all other time periods. The percent correct was determined for each sonographer and compared at different time points, accounting for each sonographer’s previous score. To assess whether there was a general trend of progress over time, we computed the correlation coefficient for percentage examinations correct across time. The research was deemed institutional review board exempt by the Institutional Review Board of Stanford University School of Medicine.
Results The list of mandated fetal and maternal structures/views based on AIUM guidelines is presented in ►Table 1. The initial, baseline QA review was performed between September 2011 and
Downloaded by: UPSTATE Medical University. Copyrighted material.
600
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
601
First-trimester examination (transabdominal view)
Fetal anatomy examination
Fetal anatomy examination (multiple gestations)
Third-trimester examination
Cervical length (transabdominal views)
Lower uterine segment
Lower uterine segment
Lower uterine segment
Uterus in sagittal views
Cervical length (transabdominal views)
Chorionicity determination (membrane thickness/peak sign)
Placental location
Uterus in transverse views
Placental location
Cervical length (transabdominal views)
Lateral ventricle measurement
Placental location
Placental cord insertion
Placental location
Distal spine
Maternal right adnexa
Maternal right adnexa
Placental cord insertion
Bladder
Maternal left adnexa
Maternal left adnexa
Maternal right adnexa
Kidneys
Profile
Cavum septum pellucidum
Maternal left adnexa
Diaphragm
Cranium
Lateral ventricle
Cavum septum pellucidum
Four chamber view of heart
Choroid plexus
Posterior fossa with measurement of cerebellum, cisterna magna, and nuchal fold
Lateral ventricle
Amniotic fluid assessment
Fetal cord insertion
Spine in sagittal views
Posterior fossa with measurement of cerebellum, cisterna magna, and nuchal fold
Umbilical artery Doppler assessment if AFI less than 8 cm and/or EFW < 10%
Spine
Spine in transverse views
Spine in sagittal views
Kidneys
Bladder
Spine in transverse views
Stomach
Three vessel cord
Bladder
Bladder
Fetal cord insertion
Three vessel cord
Upper extremities
Bowel
Fetal cord insertion
Lower extremities
Kidneys
Bowel
Three crown rump lengths
Diaphragm
Kidneys
Documented fetal heart rate
Stomach
Diaphragm
Situs
Stomach
Four chamber view of heart
Situs
Left ventricular outflow tract
Four chamber view of heart
Right ventricular outflow tract
Left ventricular outflow tract
Three vessel view
Right ventricular outflow tract
Criss-cross of great arteries
Three vessel view
Upper extremities including hands
Criss-cross of great arteries
Lower extremities including feet
Upper extremities including hands
Orbits
Lower extremities including feet
Lips
Orbits
Palate
Lips (Continued)
American Journal of Perinatology
Vol. 32
No. 6/2015
Downloaded by: UPSTATE Medical University. Copyrighted material.
Table 1 Maternal and fetal anatomical structures incorporated into the electronic checklist for each type of obstetric ultrasound examination
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
Table 1 (Continued) First-trimester examination (transabdominal view)
Fetal anatomy examination
Fetal anatomy examination (multiple gestations)
Profile
Palate
Third-trimester examination
Profile Abbreviations: AFI, amniotic fluid index; EFW, estimated fetal weight.
Fig. 1 Image of electronic checklist on GE Voluson E8.
November 2011. A total of 110 ultrasound examinations (2 NT examinations, 3 second-trimester fetal anatomy surveys, 1 fetal anatomy survey from multiple gestations, and 2 third-trimester fetal assessments for each sonographer) were reviewed, with only 49% of examinations deemed “complete” (►Table 2). None of the sonographers had a 100% complete examination rate. There were a variety of reasons examinations were deemed incomplete, including missed anatomical structures, discrepancy between the anatomical structured marked as acquired on the printed ultrasound report versus the actual image captured on the PACS, poor documentation of the image acquired, and poor image quality. “Incomplete” examinations were found lacking in as few as one anatomical structure to as many as 20 structures. The same sonographers tended to make similar errors across multiple examinations. Following institution of the mandated electronic checklist, a repeat assessment of eight random ultrasound studies for all sonographers was performed per quarter (►Table 2). Sonographers were made aware of the ongoing QA assessment process. Results from the first quarter following institution of the checklist revealed that 81% of studies were “complete,” and 90% were complete by year-end assessment. All sonographers improved their image acquisition and documentation regardless of baseline skill level at the initial QA. The percentage examinations correct for all four postchecklist time periods were significantly increased from baseline (all p values < 0.01). The percentage examinations correct was also significantly increased at postchecklist Q4 versus Q1 (mean 97 vs. 81%, p ¼ 0.022). The percentage examinations American Journal of Perinatology
Vol. 32
No. 6/2015
correct were similar across Q2, Q3, and Q4 (p > 0.05). There was a general increase of the scores over time (correlation coefficient ¼ 0.57, p < 0.0001).
Comment Checklists have long been identified in the medical field as a means of improving patient safety and reducing medical errors.10 Checklists have also been published by leading obstetrical ultrasound societies, although data describing the implementation of these checklists are limited.2,6,9 The goal of our QA process and mandated electronic checklist based on AIUM guidelines was to ensure that all anatomical structures were appropriately acquired and documented by our sonographers during OB ultrasound examinations. Based on our internal QA process, the mandated electronic checklist resulted in an improvement in image acquisition, image documentation, and image quality for all OB examination types. All sonographers, even obstetrical sonographers with over 10 years of experience, showed improvement. Moreover, the improvements seemed to persist, as suggested by our QA results 2 years later. Although the focus of the mandated electronic checklist was to improve image acquisition and documentation for individual examinations, several unforeseen benefits were also recognized following its implementation. For the sonographers, introducing the checklists allowed for automatic labeling of images, thereby reducing typing, arm fatigue, and errors in documentation such as spelling mistakes. For
Downloaded by: UPSTATE Medical University. Copyrighted material.
602
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
603
Sonographer
Baseline QA (percentage examinations correct, %)
Q1 postchecklist (percentage examinations correct, %)
Q2 postchecklist (percentage examinations correct, %)
Q3 postchecklist (percentage examinations correct, %)
Q4 postchecklist (percentage examinations correct, %)
1
75
100
100
100
100
2
88
63
100
100
100
3
75
100
75
100
100
4
75
100
100
100
88
5
0
38
88
100
100
6
57
75
63
88
100
7
50
88
63
88
88
8
88
100
100
100
100
9
75
75
75
75
100
10
13
100
100
63
100
11
50
75
100
100
100
12
38
88
100
63
100
13
0
50
100
100
88
Average Percentage
49
81
90
91
97
0.0060
0.0020
0.0016
0.0014
p a
a
p: Difference from baseline quality assurance.
the unit as a whole, the checklist assisted in creating a template for image acquisition across all ultrasound sites and thereby ensuring uniformity in standardized workflow. Once all sonographers at all sites performed identical examinations, it was clear that certain sonographers required additional education and refinement of their scanning skills. In addition, as new ultrasound protocols were created during the study period, additional images or views were incorporated directly into the electronic checklist, thereby ensuring immediate implementation of the protocol in standardized fashion across all sites. The introduction of the electronic list also improved communication between sonographers and physicians. A scan assistant report could now be printed following the examination, allowing physicians to identify the anatomical structures that could not be obtained by the sonographer. Also, the scan assistant introduced a suggested order of image acquisition, allowing the physicians to view images in an expected, ordered, and organized fashion, resulting in a more efficient review. Our study is not without limitations. Although the checklist was introduced as an electronic list placed directly on each ultrasound machine, it is possible that our improved documentation and acquisition could be obtained by a printed list as well. That being said, we feel that an electronic format directly on the ultrasound machine allowed for consistent image acquisition across all ultrasound sites and improved labeling efficacy. That being said, it is unclear whether the checklist improved accurate image acquisition or labeling, as the reasoning behind any deficient examination was not recorded during the QA process. Also, since sonographers
were made aware of the ongoing QA process, it is unclear whether the improved image acquisition and documentation were a direct result of the checklist or of the QA process itself. An additional limitation is that we did not collect information related to sonographer experience during the QA process and whether they felt that the electronic checklist was of benefit. Likewise, the length of individual ultrasound examinations was not tracked and we are unable to determine whether the electronic checklist resulted in increased time of examination. That being said, we did not change our ultrasound examination schedule to accommodate the checklists, and the total number of examinations performed at all sites did not change over the course of the study period. Finally, we have not yet determined whether the electronic checklist improved the prenatal detection of fetal anomalies, but we plan to address this question in future studies involving our ongoing QA program. Despite the limitations, our study is strengthened by the fact that we were able to assess the effects of the electronic checklist at multiple university-affiliated OB ultrasound sites, and improvements were seen in all sonographers, irrespective of baseline skill level or years of experience. A single lead sonographer reviewed all images, and led the ongoing QA process, thereby reducing any bias in the grading of ultrasound examinations. There are few quality monitoring programs in OB ultrasound such as the NT program, which consists of rigorous QA and ongoing quality control. Not only qualified sonographers must provide images of NT measurements that meet a certain standard, but there is ongoing QA through qualitative review of images and quantitative analysis of NT measurements.12,13 Currently, the American Journal of Perinatology
Vol. 32
No. 6/2015
Downloaded by: UPSTATE Medical University. Copyrighted material.
Table 2 Individual sonographer performance at baseline and in the first four quarters following introduction of the quality assurance process
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist AIUM provides certification of prenatal imaging centers through qualitative assessment of selective images. However, there is no method by which quantitative data on documentation compliance can be tracked. Our electronic checklist serves as a vehicle by which image quality and examination completeness can be tracked for the purposes of internal QA and program improvement.
Note The article was presented in poster format at the 34th Annual Meeting of the Society for Maternal-Fetal Medicine; February 2014; New Orleans, LA.
Funding The authors have no financial interests to disclose related to this article.
4
5
6
7
8 9
10
References 1 O’Keeffe DF, Abuhamad A. Obstetric ultrasound utilization in the
United States: data from various health plans. Semin Perinatol 2013;37(5):292–294 2 American Institute of Ultrasound in Medicine. AIUM practice guideline for the performance of obstetric ultrasound examinations. J Ultrasound Med 2013;32(6):1083–1101 3 Salomon LJ, Alfirevic Z, Berghella V, et al; ISUOG Clinical Standards Committee. Practice guidelines for performance of the routine
American Journal of Perinatology
Vol. 32
No. 6/2015
11
12 13
Mrazek-Pugh et al.
mid-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol 2011;37(1):116–126 International Society of Ultrasound in Obstetrics & Gynecology Education Committee. Sonographic examination of the fetal central nervous system: guidelines for performing the ‘basic examination’ and the ‘fetal neurosonogram’. Ultrasound Obstet Gynecol 2007;29(1):109–116 Carvalho JS, Allan LD, Chaoui R, et al; International Society of Ultrasound in Obstetrics and Gynecology. ISUOG Practice Guidelines (updated): sonographic screening examination of the fetal heart. Ultrasound Obstet Gynecol 2013;41(3):348–359 Salomon LJ, Alfirevic Z, Bilardo CM, et al. ISUOG practice guidelines: performance of first-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol 2013;41(1):102–113 Bhide A, Acharya G, Bilardo CM, et al. ISUOG practice guidelines: use of Doppler ultrasonography in obstetrics. Ultrasound Obstet Gynecol 2013;41(2):233–239 Chervenak FA, Chervenak JL. Medical legal issues in obstetric ultrasound. Clin Perinatol 2007;34(2):299–308, vi American College of Obstetricians and Gynecologists. ACOG Practice Bulletin No. 101: Ultrasonography in pregnancy. Obstet Gynecol 2009;113(2 Pt 1):451–461 D’Alton ME, Fuchs KM, Abuhammad A, et al; Nuchal Translucency Quality Review Program. Implementation of a national nuchal translucency education and quality monitoring program. Obstet Gynecol 2014;123(1):149–154 Abuhamad AZ, Benacerraf BR, Woletz P, Burke BL. The accreditation of ultrasound practices: impact on compliance with minimum performance guidelines. J Ultrasound Med 2004;23(8):1023–1029 The Fetal Medicine Foundation. Available at: www.fetalmedicineusa.com/accreditation.php. Accessed December 17, 2014 Nuchal Translucency Quality Review. Available at: www.NQTR.org. Accessed December 17, 2014
Downloaded by: UPSTATE Medical University. Copyrighted material.
604
599
Obstetric Ultrasound Quality Improvement Initiative—Utilization of a Quality Assurance Process and Standardized Checklists Barbora Mrazek-Pugh, RDMS, RVT1
Yair J. Blumenfeld, MD2
1 Lucile Packard Children’s Hospital Stanford, Palo Alto, California 2 Department of Obstetrics and Gynecology, Stanford University
School of Medicine, Stanford, California
Henry C. Lee, MD3
Jane Chueh, MD2
Address for correspondence Barbora Mrazek-Pugh, RDMS, RVT, 725 Welch Road, Suite HF306, Palo Alto, CA 94043 (e-mail: [email protected]).
3 Department of Pediatrics, Stanford University School of Medicine,
Stanford, California
Abstract
Keywords
► ► ► ► ►
ultrasound quality checklists obstetric improvement
Objective Our aim was to assess whether mandated completion of an electronic checklist and a quality assurance (QA) process improved obstetric (OB) ultrasound image documentation. Study Design A checklist of mandated images based on the American Institute of Ultrasound in Medicine guidelines was created. A baseline QA assessment was performed with a lead senior sonographer reviewing eight random OB examinations for each sonographer. An electronic checklist was then instituted for all OB examinations on each ultrasound machine. It was mandated that each anatomical structure be checked off during real-time image acquisition. A repeat QA assessment of each sonographer was then performed quarterly. Results Baseline assessments were performed between September 2011 and November 2011. Out of the 110 examinations analyzed, only 49% were deemed “complete” with none of the sonographers having a 100% complete examination rate. Following institution of the mandated electronic checklist, a repeat assessment revealed an 81% complete examination rate for the next quarter, and 90% were complete at the end of a year. All sonographers improved their image acquisition regardless of baseline skill level at the initial QA. Conclusion A QA process and a mandated standardized electronic checklist improved the image documentation.
Utilization of obstetric (OB) ultrasound is rising, with pregnancies in the United States receiving four to five ultrasound examinations per pregnancy.1 Simultaneously, ultrasound machine technology and complexity have advanced, and requirements for meeting appropriate image acquisition and documentation of OB examinations have increased as well.2–7 Therefore, busy prenatal diagnostic centers find themselves in the precarious position of needing to maximize efficiency while performing ultrasound examinations that may contain over 30 required elements per study.2 Not surprisingly, lawsuit
cases involving OB ultrasound examinations are the most common in all of medical ultrasound.8 Since patient safety is of primary concern, and ultrasound procedures need to be both cost-effective and clinically effective, maintaining assurance of quality in OB ultrasound examinations is warranted. The American Institute of Ultrasound in Medicine (AIUM) “Standards and Guidelines for the Accreditation of Ultrasound Practices” states that “adequate documentation is essential for high-quality patient care. There should be a permanent record of the ultrasound examination and its interpretation.
received December 11, 2014 accepted after revision December 31, 2014 published online March 2, 2015
Copyright © 2015 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.
DOI http://dx.doi.org/ 10.1055/s-0035-1545667. ISSN 0735-1631.
Downloaded by: UPSTATE Medical University. Copyrighted material.
Am J Perinatol 2015;32:599–604.
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist Images of all appropriate areas, both normal and abnormal, should be recorded. Retention of the ultrasound examination should be consistent both with clinical needs and with relevant legal and local health care facility requirements.2 According to the American College of Obstetricians and Gynecologists (ACOG), quality control for ultrasonography in pregnancy should be accomplished through careful record keeping of OB ultrasound examination results, archiving reports and images, and correlation with clinical outcome.9 Several quality assurance (QA) processes have been developed for OB ultrasound examinatiios.10–13 For example, nuchal translucency (NT) measurements require rigorous education, training, and certification, with an ongoing review of image quality and audit of measurement ranges.11–13 Unfortunately, a similar process does not exist for other types of OB ultrasound examinations such as fetal anatomy examinations. The AIUM and other international ultrasound societies have published checklists outlining optimal fetal and maternal anatomical structures required to be visualized and documented during NT examinations, second trimester fetal anatomical surveys, and third trimester examinations.2 In addition the AIUM certification process requires that the fetal biometry be measured on certain standardized planes, and fetal anatomy images be obtained in certain views.2 However, there is currently no standard QA method for OB image quality and documentation for many OB ultrasound examinations, and the utility of AIUM checklists as a means of improving ultrasound documentation is unknown. The aim of our study was to assess whether a QA process and mandated completion of an electronic checklist of required OB (both fetal and maternal) anatomical structures and views, during real-time acquisition, would improve image acquisition and documentation.
Materials and Methods Checklists of mandated images for first trimester, second trimester, and third trimester examinations for fetal assessments based on AIUM guidelines were created (►Table 1).2 The checklists were then instituted in electronic format for the different examination types on each ultrasound machine (General Electric Voluson E8 Expert; GE Healthcare Austria GmbH & Co OG, Zipf, Austria) used at the Lucile Packard Children’s Hospital Stanford (LPCH) Perinatal Diagnostic Center (PDC) and affiliated sites (10 ultrasound machines at 5 sites) (►Fig. 1). This was done using the “scan assistant” feature that was available to program on each ultrasound machine. The use of an electronic checklist, as compared with a paper checklist, was chosen to allow images to be checked during real-time acquisition. Before implementation of routine use of the checklist, a baseline QA assessment of each sonographer employed by the LPCH–PDC and affiliated sites (13 sonographers) was performed. Each sonographer assessed has at least 3 years’ experience performing OB ultrasound examinations, and at most 27 years. All sonographers are NT-certified through either the Nuchal Translucency Quality Review Program or the Fetal Medicine Foundation, American Registry for Diagnostic Medical Sonography registered in OB, and all sites are AIUM accredited.2,13 In American Journal of Perinatology
Vol. 32
No. 6/2015
Mrazek-Pugh et al.
addition, sonographers participate in monthly ultrasound conferences conducted by members of the Division of MaternalFetal Medicine at Stanford University School of Medicine. They also attend two different bimonthly image review multidisciplinary case conferences. For the baseline assessment, a single lead senior sonographer (B. M. P.) reviewed eight random ultrasound examinations (two first-trimester examinations, four second-trimester fetal anatomical surveys, which included one anatomy survey of multiple gestations, and two third-trimester ultrasound examinations) for each sonographer. Images from the entire examination were reviewed and compared with the printed ultrasound report (ClickView Corporation, San Francisco, CA). For the purpose of the QA assessment, an examination was considered “complete” if all images reported as seen in the written report were indeed acquired by the sonographer and stored on the LPCH Picture Archiving and Communication System (PACS). Image quality for the individual fetal or maternal structures was also assessed, specifically for the accurate scanning plane and comprehensiveness of the image. Sonographers were not aware of the initial QA review program and the results of the baseline assessment were reviewed with each sonographer in private. Inadequacies were discussed along with recommendations for improvement. The sonographers were then notified about the institution of the mandated electronic checklist. Following the installation of the mandated electronic checklist on each ultrasound machine, it was required that each anatomical structure listed be checked during real-time acquisition, and not after the completion of the scan. When the examination was completed by the sonographer and before the physician’s review, a “scan assistant” report was printed for presentation to the physician. This report allows the physician to quickly assess which images were not acquired during the examination. Limitations prohibiting adequate image acquisition included technical factors, such as suboptimal fetal position or maternal factors such as obesity, prior cesareans, or other abdominal surgery. Follow-up QA assessments for each sonographer were performed quarterly, and the results were shared with each sonographer and the attending physicians. To assess whether there was a difference over time, we compared the percentage examinations correct across each time period (baseline and Q1–Q4 postchecklist) using Wilcoxon matched-pairs signed-ranks, comparing each time period to all other time periods. The percent correct was determined for each sonographer and compared at different time points, accounting for each sonographer’s previous score. To assess whether there was a general trend of progress over time, we computed the correlation coefficient for percentage examinations correct across time. The research was deemed institutional review board exempt by the Institutional Review Board of Stanford University School of Medicine.
Results The list of mandated fetal and maternal structures/views based on AIUM guidelines is presented in ►Table 1. The initial, baseline QA review was performed between September 2011 and
Downloaded by: UPSTATE Medical University. Copyrighted material.
600
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
601
First-trimester examination (transabdominal view)
Fetal anatomy examination
Fetal anatomy examination (multiple gestations)
Third-trimester examination
Cervical length (transabdominal views)
Lower uterine segment
Lower uterine segment
Lower uterine segment
Uterus in sagittal views
Cervical length (transabdominal views)
Chorionicity determination (membrane thickness/peak sign)
Placental location
Uterus in transverse views
Placental location
Cervical length (transabdominal views)
Lateral ventricle measurement
Placental location
Placental cord insertion
Placental location
Distal spine
Maternal right adnexa
Maternal right adnexa
Placental cord insertion
Bladder
Maternal left adnexa
Maternal left adnexa
Maternal right adnexa
Kidneys
Profile
Cavum septum pellucidum
Maternal left adnexa
Diaphragm
Cranium
Lateral ventricle
Cavum septum pellucidum
Four chamber view of heart
Choroid plexus
Posterior fossa with measurement of cerebellum, cisterna magna, and nuchal fold
Lateral ventricle
Amniotic fluid assessment
Fetal cord insertion
Spine in sagittal views
Posterior fossa with measurement of cerebellum, cisterna magna, and nuchal fold
Umbilical artery Doppler assessment if AFI less than 8 cm and/or EFW < 10%
Spine
Spine in transverse views
Spine in sagittal views
Kidneys
Bladder
Spine in transverse views
Stomach
Three vessel cord
Bladder
Bladder
Fetal cord insertion
Three vessel cord
Upper extremities
Bowel
Fetal cord insertion
Lower extremities
Kidneys
Bowel
Three crown rump lengths
Diaphragm
Kidneys
Documented fetal heart rate
Stomach
Diaphragm
Situs
Stomach
Four chamber view of heart
Situs
Left ventricular outflow tract
Four chamber view of heart
Right ventricular outflow tract
Left ventricular outflow tract
Three vessel view
Right ventricular outflow tract
Criss-cross of great arteries
Three vessel view
Upper extremities including hands
Criss-cross of great arteries
Lower extremities including feet
Upper extremities including hands
Orbits
Lower extremities including feet
Lips
Orbits
Palate
Lips (Continued)
American Journal of Perinatology
Vol. 32
No. 6/2015
Downloaded by: UPSTATE Medical University. Copyrighted material.
Table 1 Maternal and fetal anatomical structures incorporated into the electronic checklist for each type of obstetric ultrasound examination
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
Table 1 (Continued) First-trimester examination (transabdominal view)
Fetal anatomy examination
Fetal anatomy examination (multiple gestations)
Profile
Palate
Third-trimester examination
Profile Abbreviations: AFI, amniotic fluid index; EFW, estimated fetal weight.
Fig. 1 Image of electronic checklist on GE Voluson E8.
November 2011. A total of 110 ultrasound examinations (2 NT examinations, 3 second-trimester fetal anatomy surveys, 1 fetal anatomy survey from multiple gestations, and 2 third-trimester fetal assessments for each sonographer) were reviewed, with only 49% of examinations deemed “complete” (►Table 2). None of the sonographers had a 100% complete examination rate. There were a variety of reasons examinations were deemed incomplete, including missed anatomical structures, discrepancy between the anatomical structured marked as acquired on the printed ultrasound report versus the actual image captured on the PACS, poor documentation of the image acquired, and poor image quality. “Incomplete” examinations were found lacking in as few as one anatomical structure to as many as 20 structures. The same sonographers tended to make similar errors across multiple examinations. Following institution of the mandated electronic checklist, a repeat assessment of eight random ultrasound studies for all sonographers was performed per quarter (►Table 2). Sonographers were made aware of the ongoing QA assessment process. Results from the first quarter following institution of the checklist revealed that 81% of studies were “complete,” and 90% were complete by year-end assessment. All sonographers improved their image acquisition and documentation regardless of baseline skill level at the initial QA. The percentage examinations correct for all four postchecklist time periods were significantly increased from baseline (all p values < 0.01). The percentage examinations correct was also significantly increased at postchecklist Q4 versus Q1 (mean 97 vs. 81%, p ¼ 0.022). The percentage examinations American Journal of Perinatology
Vol. 32
No. 6/2015
correct were similar across Q2, Q3, and Q4 (p > 0.05). There was a general increase of the scores over time (correlation coefficient ¼ 0.57, p < 0.0001).
Comment Checklists have long been identified in the medical field as a means of improving patient safety and reducing medical errors.10 Checklists have also been published by leading obstetrical ultrasound societies, although data describing the implementation of these checklists are limited.2,6,9 The goal of our QA process and mandated electronic checklist based on AIUM guidelines was to ensure that all anatomical structures were appropriately acquired and documented by our sonographers during OB ultrasound examinations. Based on our internal QA process, the mandated electronic checklist resulted in an improvement in image acquisition, image documentation, and image quality for all OB examination types. All sonographers, even obstetrical sonographers with over 10 years of experience, showed improvement. Moreover, the improvements seemed to persist, as suggested by our QA results 2 years later. Although the focus of the mandated electronic checklist was to improve image acquisition and documentation for individual examinations, several unforeseen benefits were also recognized following its implementation. For the sonographers, introducing the checklists allowed for automatic labeling of images, thereby reducing typing, arm fatigue, and errors in documentation such as spelling mistakes. For
Downloaded by: UPSTATE Medical University. Copyrighted material.
602
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist
Mrazek-Pugh et al.
603
Sonographer
Baseline QA (percentage examinations correct, %)
Q1 postchecklist (percentage examinations correct, %)
Q2 postchecklist (percentage examinations correct, %)
Q3 postchecklist (percentage examinations correct, %)
Q4 postchecklist (percentage examinations correct, %)
1
75
100
100
100
100
2
88
63
100
100
100
3
75
100
75
100
100
4
75
100
100
100
88
5
0
38
88
100
100
6
57
75
63
88
100
7
50
88
63
88
88
8
88
100
100
100
100
9
75
75
75
75
100
10
13
100
100
63
100
11
50
75
100
100
100
12
38
88
100
63
100
13
0
50
100
100
88
Average Percentage
49
81
90
91
97
0.0060
0.0020
0.0016
0.0014
p a
a
p: Difference from baseline quality assurance.
the unit as a whole, the checklist assisted in creating a template for image acquisition across all ultrasound sites and thereby ensuring uniformity in standardized workflow. Once all sonographers at all sites performed identical examinations, it was clear that certain sonographers required additional education and refinement of their scanning skills. In addition, as new ultrasound protocols were created during the study period, additional images or views were incorporated directly into the electronic checklist, thereby ensuring immediate implementation of the protocol in standardized fashion across all sites. The introduction of the electronic list also improved communication between sonographers and physicians. A scan assistant report could now be printed following the examination, allowing physicians to identify the anatomical structures that could not be obtained by the sonographer. Also, the scan assistant introduced a suggested order of image acquisition, allowing the physicians to view images in an expected, ordered, and organized fashion, resulting in a more efficient review. Our study is not without limitations. Although the checklist was introduced as an electronic list placed directly on each ultrasound machine, it is possible that our improved documentation and acquisition could be obtained by a printed list as well. That being said, we feel that an electronic format directly on the ultrasound machine allowed for consistent image acquisition across all ultrasound sites and improved labeling efficacy. That being said, it is unclear whether the checklist improved accurate image acquisition or labeling, as the reasoning behind any deficient examination was not recorded during the QA process. Also, since sonographers
were made aware of the ongoing QA process, it is unclear whether the improved image acquisition and documentation were a direct result of the checklist or of the QA process itself. An additional limitation is that we did not collect information related to sonographer experience during the QA process and whether they felt that the electronic checklist was of benefit. Likewise, the length of individual ultrasound examinations was not tracked and we are unable to determine whether the electronic checklist resulted in increased time of examination. That being said, we did not change our ultrasound examination schedule to accommodate the checklists, and the total number of examinations performed at all sites did not change over the course of the study period. Finally, we have not yet determined whether the electronic checklist improved the prenatal detection of fetal anomalies, but we plan to address this question in future studies involving our ongoing QA program. Despite the limitations, our study is strengthened by the fact that we were able to assess the effects of the electronic checklist at multiple university-affiliated OB ultrasound sites, and improvements were seen in all sonographers, irrespective of baseline skill level or years of experience. A single lead sonographer reviewed all images, and led the ongoing QA process, thereby reducing any bias in the grading of ultrasound examinations. There are few quality monitoring programs in OB ultrasound such as the NT program, which consists of rigorous QA and ongoing quality control. Not only qualified sonographers must provide images of NT measurements that meet a certain standard, but there is ongoing QA through qualitative review of images and quantitative analysis of NT measurements.12,13 Currently, the American Journal of Perinatology
Vol. 32
No. 6/2015
Downloaded by: UPSTATE Medical University. Copyrighted material.
Table 2 Individual sonographer performance at baseline and in the first four quarters following introduction of the quality assurance process
Obstetric Ultrasound Quality Improvement Initiative Using Electronic Checklist AIUM provides certification of prenatal imaging centers through qualitative assessment of selective images. However, there is no method by which quantitative data on documentation compliance can be tracked. Our electronic checklist serves as a vehicle by which image quality and examination completeness can be tracked for the purposes of internal QA and program improvement.
Note The article was presented in poster format at the 34th Annual Meeting of the Society for Maternal-Fetal Medicine; February 2014; New Orleans, LA.
Funding The authors have no financial interests to disclose related to this article.
4
5
6
7
8 9
10
References 1 O’Keeffe DF, Abuhamad A. Obstetric ultrasound utilization in the
United States: data from various health plans. Semin Perinatol 2013;37(5):292–294 2 American Institute of Ultrasound in Medicine. AIUM practice guideline for the performance of obstetric ultrasound examinations. J Ultrasound Med 2013;32(6):1083–1101 3 Salomon LJ, Alfirevic Z, Berghella V, et al; ISUOG Clinical Standards Committee. Practice guidelines for performance of the routine
American Journal of Perinatology
Vol. 32
No. 6/2015
11
12 13
Mrazek-Pugh et al.
mid-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol 2011;37(1):116–126 International Society of Ultrasound in Obstetrics & Gynecology Education Committee. Sonographic examination of the fetal central nervous system: guidelines for performing the ‘basic examination’ and the ‘fetal neurosonogram’. Ultrasound Obstet Gynecol 2007;29(1):109–116 Carvalho JS, Allan LD, Chaoui R, et al; International Society of Ultrasound in Obstetrics and Gynecology. ISUOG Practice Guidelines (updated): sonographic screening examination of the fetal heart. Ultrasound Obstet Gynecol 2013;41(3):348–359 Salomon LJ, Alfirevic Z, Bilardo CM, et al. ISUOG practice guidelines: performance of first-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol 2013;41(1):102–113 Bhide A, Acharya G, Bilardo CM, et al. ISUOG practice guidelines: use of Doppler ultrasonography in obstetrics. Ultrasound Obstet Gynecol 2013;41(2):233–239 Chervenak FA, Chervenak JL. Medical legal issues in obstetric ultrasound. Clin Perinatol 2007;34(2):299–308, vi American College of Obstetricians and Gynecologists. ACOG Practice Bulletin No. 101: Ultrasonography in pregnancy. Obstet Gynecol 2009;113(2 Pt 1):451–461 D’Alton ME, Fuchs KM, Abuhammad A, et al; Nuchal Translucency Quality Review Program. Implementation of a national nuchal translucency education and quality monitoring program. Obstet Gynecol 2014;123(1):149–154 Abuhamad AZ, Benacerraf BR, Woletz P, Burke BL. The accreditation of ultrasound practices: impact on compliance with minimum performance guidelines. J Ultrasound Med 2004;23(8):1023–1029 The Fetal Medicine Foundation. Available at: www.fetalmedicineusa.com/accreditation.php. Accessed December 17, 2014 Nuchal Translucency Quality Review. Available at: www.NQTR.org. Accessed December 17, 2014
Downloaded by: UPSTATE Medical University. Copyrighted material.
604
