European Heart Journal – Cardiovascular Imaging (2014) 15, 747–752 doi:10.1093/ehjci/jet280
Development of a National Echocardiography Quality Improvement Programme: insights into feasibility, uptake, and clinical utility Sanjeev Bhattacharyya1,2*, Rachael James 2,3, Helen Rimington 2,4, Jane Allen 2,5, Lorraine Lee 3, Kevin Fox 1,2, Navroz D Masani 2,6, and Guy Lloyd 7, On behalf of the British Society of Echocardiography 1
Department of Cardiology, Hammersmith Hospital, Imperial College Healthcare NHS Trust, Du Cane Road, London W12 OHS, UK; 2British Society of Echocardiography, London, UK; Sussex Cardiac Centre, Brighton & Sussex University Hospitals Trust, Brighton, UK; 4Department of Echocardiography, Guy’s & St Thomas’ NHS Trust, London, UK; 5Department of Echocardiography, York Teaching Hospitals NHS Trust, York, UK; 6Department of Cardiology, University Hospital of UK, Cardiff, UK; and 7Department of Cardiology, East Sussex NHS Trust, Eastbourne, UK 3
Received 4 October 2013; accepted after revision 13 December 2013; online publish-ahead-of-print 9 January 2014
Aims
----------------------------------------------------------------------------------------------------------------------------------------------------------Keywords
cardiovascular imaging † quality † echocardiography
Introduction The concept of quality improvement in cardiovascular imaging encompasses laboratory systems and policies to ensure appropriate use of imaging, accurate acquisition and interpretation in order to optimize patient outcomes.1 Individual certification schemes have been developed to ensure that practitioners are competent at acquiring and reporting studies. Laboratory accreditation aims to ensure that adequate facilities, personnel, policies, and procedures are in place
to facilitate a high quality clinical service, including the need for an established in-house quality assurance (QA) programme.2,3 Appropriateness criteria for echocardiography have also recently been published. Several investigators have identified the clinical utility and prognostic value of implementation of these criteria.4,5,6 There is much less clarity over what represents a robust QA system, although the American Society of Echocardiography and European Association of Cardiovascular Imaging have recently published recommendations3,7
* Corresponding author. Tel: + 44 208 383 1000; Fax: + 44 208 383 4392. Email: [email protected] Published on behalf of the European Society of Cardiology. All rights reserved. & The Author 2014. For permissions please email: [email protected]
Downloaded from by guest on June 6, 2016
There are minimal data on methods to measure and improve quality in image interpretation when reporting cardiovascular imaging studies. We sought to identify the feasibility and clinical utility of a nationally implemented quality assurance (QA) process. ..................................................................................................................................................................................... Methods A web-based platform was developed to administer a national QA module to 27 echocardiography departments within the UK. Three QA modules were delivered from 2011. The proportion of units using the QA module increased from 14 and results (52.2%) in the first module to 22 (81.5%) in the third module. There was no significant change in the proportion of correct answers between the first module (88.9%) and the third module (82.8%), P ¼ 0.3. The number of echocardiographers with at least one incorrect answer increased from 16 (21.6%) in the first module to 54 (34%) in the third module, P ¼ 0.03. Overall, in valvular heart disease cases there were 36 (10.6%) incorrect responses where qualitative assessment of the severity of valve dysfunction was tested compared with 4 (3.6%) incorrect responses where quantitative assessment of valve dysfunction was tested, P ¼ 0.04. In chamber quantification cases, there were 36 (6.8%) incorrect responses where qualitative assessment of chamber function was tested compared with 3 (2.1%) incorrect responses where quantitative assessment of chamber function was tested, P ¼ 0.04. ..................................................................................................................................................................................... Conclusion The incorporation of national QA programme is feasible with rapid uptake. The platform allows comparison of an individual’s interpretation skills against a reference standard which can be used as a method to identify inter-observer variability and as a training tool.
748 Reduction of physician/sonographer inter-observer variability is important both for accuracy, and for long-term follow-up of chronic disease in particular valvular heart disease to determine timing of surgery8 and patients require serial cardiac monitoring of left ventricular function (such as those undergoing trastuzumab therapy). Inter-observer variability of 14% for determination of left ventricular (LV) ejection fraction and concordance of between 58 and 92% for determination of diastolic class have been reported.9,10 The Intersocietal Accreditation Commission has assessment of inter-observer variability within its accreditation scheme.11 The British Society of Echocardiography (BSE) introduced a dual component QA process for accredited echocardiography departments in 2011. The objective of this initiative was to provide an arena for formative assessment of echocardiographers’ interpretative skills and use this assessment to define areas for further professional development. The second aspect of the QA process was to provide a central electronic repository to promote and act as a stimulus to further on-going locally organized QA and audit activity. Currently, there are no data which describe the organization, feasibility or clinical utility of a QA process in cardiovascular imaging.
Methods The BSE QA module is an electronic, web-based platform. All individuals within accredited departments are invited to participate in a module twice a year. Each module consists of a series of five multiple choice questions based on echocardiographic images randomly allocated from the central BSE database (Figure 1). Each of the possible answers to the multiple choice question is graded as green, yellow, or red. The green answer is correct, amber is correct but could be refined and red is incorrect. The reason for the amber classification is that in echocardiography, particularly with visual interpretation of structures or valves, there may be some debate even amongst expert readers about the exact severity or nature of pathology. There is a time limit of 10 min for each question.
Question database A series of 78 questions based on the BSE core curriculum was formulated.12 Questions incorporated echocardiographic images (still and cine) covering cardiac structures [LV, right ventricle (RV), atria, pericardium] and valves/shunts (mitral, aortic, tricuspid, pulmonary, atrial septal defect, ventricular septal defect). Both qualitative and quantitative assessment of cardiac function and structure was performed. Qualitative questions utilized cine images where individuals were required to identify LV wall motion abnormalities, visually estimate LV size and ejection fraction, RV size and function, and estimate grades of valve regurgitation. Quantitative questions included m-mode, 2D, or Doppler-derived measurements where individuals were required to use the data to quantify LV and RV systolic function, LV diastolic function, cardiac chamber size and severity of valvular regurgitation, stenosis, and intra-cardiac shunts. The questions utilized a multiple choice format. Two expert readers formulated and designed the multiple choice questions. Another two expert readers re-read all the questions and associated images to ensure agreement with the answers. A consensus opinion was reached for all questions.
Individual echocardiographer participation Access to the QA module is available to accredited departments for 1 month. Individual echocardiographers are e-mailed to inform them of
the module opening and closing dates. The scores are made available to a designated QA lead in each department. The QA lead is responsible for identifying echocardiographers who have incorrectly answered questions and formulating an action plan which is uploaded onto the QA module. The plan utilized the module to identify areas of echocardiography where an individual echocardiographer needs development and further continuing professional education.
Statistics Data are reported as median and inter-quartile range or number and percentage. The x2 test was used to compare categorical variables. When the number of categorical variables was less than five the Fisher exact test was used. All tests of significance were two sided. A probability value (P) of ,0.05 was considered significant. Statistical analysis was performed using StatsDirect Version 2.5.7 (StatsDirect, UK).
Results A total of 27 echocardiography departments have BSE laboratory accreditation in the UK. A total of 193 echocardiographers [median of 8 (inter-quartile range 6 –11] per department) were registered to participate in the QA programme. A total of three QA modules have been delivered since 2011.
Temporal trends in utilization of the QA module The proportion of BSE accredited departments using the QA module has increased from 14 (52.2%) in the first module to 22 (81.5%) in the third module. The number of echocardiographers registered to participate in the QA modules increased from 77 in the first module to 193 in the third module (Table 1). The number of registered echocardiographers who actually participate increased from 74 in the first module to 159 in the third module. However, the proportion of registered users to those actually taking the module dropped from 96.1% in the first module to 82.4% in the third module.
QA module results Overall, 1745 (99.1%) questions out of a possible 1760 questions were answered over the 3 QA modules. There was no significant change in the proportion of incorrect answers (6%) between the first module and the last module (8.2%), P ¼ 0.8 (Table 2). There was no significant change in the proportion of correct answers between the first module (88.9%) and the third module (82.8%), P ¼ 0.3. On an individual basis, the number of echocardiographers with at least one incorrect answer increased from 16 (21.6%) in the first module to 54 (34%) in the third module, P ¼ 0.03.
Temporal trends in departmental performance There was no significant change in the proportion of echocardiographers per department who participated in the QA modules [(mean 80.8, 79.4 and 78.1% in the first, second, and third modules, respectively), P ¼ 0.36]. There was a significant increase in the proportion of echocardiographers per department with at least one incorrect answer [(mean 16.6, 24.5, 35.4% in the first, second, and third modules, respectively), P ¼ 0.04].
Downloaded from by guest on June 6, 2016
QA module
S. Bhattacharyya et al.
Development of a national echocardiography quality improvement programme
Figure 1 A sample question from the quality assurance module. A still image or a movie clip is displayed with the question below. The echocardiographer answers the multiple choice question. Correct answer is highlighted in green
749
Downloaded from by guest on June 6, 2016
750
Table 1
S. Bhattacharyya et al.
Trends in the usage of the QA modules
QA module
Number (%) of accredited echocardiography departments using module
Number of users eligible to participate
Number (%) of users who participated in module
............................................................................................................................................................................... November 2011
14 (51.2)
77
74 (96.1)
May 2012
21 (77.8)
134
119 (88.8)
November 2012
25 (92.6)
193
159 (82.4)
Table 2
Overall responses to the QA module questions
QA module
Green (correct) answers, n (%)
Yellow answers, n (%)
Red (incorrect) answers, n (%)
Total answers
............................................................................................................................................................................... November 2011
311 (88.9)
18 (5.1)
21 (6)
350
May 2012
495 (83.6)
50 (8.4)
47 (7.9)
592
November 2012
665 (82.8)
72 (9)
66 (8.2)
803
Pathology
Green answer, n (%)
Yellow answer, n (%)
Red answer, n (%)
287 (84.4) 92 (82.9) 464 (88.3) 113 (78.5)
17 (5) 15 (13.5) 25 (4.8) 27 (18.8)
36 (10.6) 4 (3.6) 36 (6.8) 3 (2.1)
............................................................................................................................................................................... Valve dysfunction Chambers function
Qualitative Quantitative Qualitative Quantitative
Quantitative versus qualitative assessment Overall, in valvular heart disease cases there were 36 (10.6%) incorrect responses where qualitative assessment of the severity of valve dysfunction was tested compared with 4 (3.6%) incorrect responses where quantitative assessment of valve dysfunction was tested, P ¼ 0.04 (Table 3). In chamber quantification there were 36 (6.8) incorrect responses where qualitative assessment of chamber function was tested compared with 3 (2.1%) incorrect responses where quantitative assessment of chamber function was tested, P ¼ 0.04 (Table 4).
Professional development/action plans The proportion of echocardiographers with at least one incorrect answer who had an action plan set up initially increased from 8 (50%) in the first module to 34 (97%) in the second module. This decreased in the third module to 25 (46%) (Table 4). The proportion of echocardiographers with an action plan who completed the plans learning objectives increased from 6 (75%) in the first module to 24 (96%) in the third module.
Local departmental QA Nine (33.3%) departments had used the electronic repository to document their local QA activity [median 4 episodes per department
(inter-quartile range 1 –11]. The majority of these were clinical review meetings.
Discussion This is the first study to examine the feasibility and clinical value of incorporating a nationally led quality improvement programme into echocardiography departments which focussed on image interpretation and variability. Our results demonstrate that the incorporation of the programme into clinical practice of an echocardiography can be achieved rapidly. In less than a year the proportion of eligible departments utilizing this voluntary programme increased from 52 to 82%. Furthermore, there was a steady increase in the number of echocardiographers using the QA modules. There are a range measures which are recognized as key principles in providing a high quality and consistent echocardiography practice. These include individual and laboratory accreditation and use of appropriateness criteria.1 Individual and laboratory accreditation was introduced by the BSE in 1994 and 2005, respectively. More recently, the American College of Cardiology together with respective imaging societies has published appropriateness criteria for echocardiography.4 QA has become embedded in other branches of diagnostic medicine,13,14 but has not yet been well developed in
Downloaded from by guest on June 6, 2016
Table 3 Comparison of responses to the QA modules between questions requiring qualitative interpretation and those with quantitative data
751
Development of a national echocardiography quality improvement programme
Table 4
Impact of the QA module on development of professional development plans Number (%) of echocardiographers who participated with at least one incorrect answer
Number (%) of echocardiographers with an incorrect answer who developed an action plan
Number (%) of echocardiographers where action plan has been completed
November 2011 May 2012
16 (21.6) 35 (29.4)
8 (50) 34 (97)
6 (75) 28 (82.4)
November 2012
54 (34)
25 (46)
24 (96)
QA module
...............................................................................................................................................................................
attendance have not been critically evaluated but are likely to all form a part of the improvement process.11 Only a third of centres used electronic repository to document their local QA activity. This is disappointingly low. Further investigation should be directed to examine the range and quality of local QA undertaken. Potential future applications of the QA programme include identifying inter-departmental variability and tracking changes over time. This provides an infrastructure for departments to benchmark their performance against national averages. We propose that quality standards for laboratory accreditation should include QA benchmarks. These could include the proportion of echocardiographers per department who participate in the QA modules, echocardiographic interpretation variability markers, and the proportion of echocardiographers who completed personal development plans. A limitation of this study is the effect of quality programme on an individual subsequent performance in the interpretation of echocardiographic images, has not been examined. Future studies with longitudinal follow-up of an individual’s scores will be possible once the programme has completed several more cycles. The QA programme was only made available to echocardiography departments with laboratory accreditation and therefore only 27 departments were involved. However, the clinical value is likely to be similar in other laboratories.
Conclusion The incorporation of a national cardiovascular imaging QA programme is feasible with relatively rapid uptake. The platform allows comparison of an individual’s interpretation skills against a reference standard which can be used as a training tool, a method to identify inter-observer variability and as an aid to allow the identification of further learning objectives. It also permits the documentation of on-going local departmental QA activity.
Funding British Heart Foundation Grant for development of a web-based quality assurance module. Conflicts of interest: none
References 1. Douglas PS, Chen J, Gillam L, Hendel R, Hundley WG, Masoudi F et al. Achieving quality in cardiovascular imaging II: proceedings from the Second American College of Cardiology—Duke University Medical Center Think Tank on Quality in Cardiovascular Imaging. J Am Coll Cardiol Img 2009;2:231 –40. 2. Popescu BA, Andrade MJ, Badano LP, Fox KF, Flachskampf FA, Lancellotti P et al. European Association of Echocardiography recommendations for training,
Downloaded from by guest on June 6, 2016
echocardiography. This may be because precise standards and recommendations have until recently not been published.3,7 There is general agreement that a QA programme in echocardiography should contain the following elements: regular echocardiography review meetings, systematic re-reading of echocardiographic reports for accuracy, departmental benchmarking exercises for common pathology such as a valvular regurgitation and ventricular function, and a method of feedback and remediation. The BSE QA module is not designed to supplant existing departmental processes, but rather to complement them. The outcome of approaches to quality improvement has not been described. One of the strengths of our programme is the ability for individual echocardiographers to quantify valve dysfunction or chamber function against standardized images. The modules used both images requiring only qualitative interpretation and images with quantitative data included. We identified a significant difference in the performance of echocardiographers when comparing the two. Correct quantification of both valve dysfunction and chamber function were significantly improved when quantitative data were supplied compared with when only qualitative interpretation were used. Indeed, recent data have shown that qualitative interpretation of LV ejection fraction by visual estimate has a high inter-observer variability (14%) even in an academic centre.9 A tutorial review of reference cases reduced the inter-observer variability to 8%. When quantitative methods were used the inter-observer variability was significantly less (6%). Teaching sessions reduced the variability of interpretation of these studies to 3%. Therefore, we propose our QA programme as a method to assess variability in interpretation against a reference standard and use the questions/answers as teaching materials to reduce variability. In our study, approximately a third of echocardiographers had at least one incorrect answer when compared with expert opinion. This highlights the value of the programme in identifying areas of practice where an individual echocardiographer requires further development. The number of echocardiographers with an incorrect answer increased during the study. Although purely speculative, it is possible the first cohort of centres who joined the QA scheme may have already had internal QA processes in place, whereas centres who joined the scheme subsequently did not. Reassuringly, a large proportion of these echocardiographers went on to formulate and complete an action plan to improve the skills and knowledge in the area identified. The optimal method of improving echocardiographic interpretation skills is not well defined. As previously mentioned, a teaching course with standardized images can reduce inter-observer variability.9 The effect of online casebased learning, guidelines, journal review articles, and conference
752
3.
4.
5.
6.
competence, and qualityimprovement in echocardiography. Eur J Echocardiogr 2009; 10:893–905. Picard MH, Adams D, Bierig SM, Dent JM, Douglas PS, Gillam LD et al. American Society of Echocardiography. American Society of Echocardiography recommendations for quality echocardiography laboratory operations. J Am Soc Echocardiogr 2011;24:1 –10. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR et al., American College of Cardiology Foundation Appropriate Use Criteria Task Force; American Society of Echocardiography; American Heart Association; American Society of Nuclear Cardiology; Heart Failure Society of America; Heart Rhythm Society; Society for Cardiovascular Angiography and Interventions; Society of Critical Care Medicine; Society of Cardiovascular Computed Tomography; Society for Cardiovascular Magnetic Resonance. ACCF/ASE/AHA/ASNC/ HFSA/HRS/SCAI/ SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Society of Echocardiography, American Heart Association, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular Magnetic Resonance, endorsed by the American College of Chest Physicians. J Am Coll Cardiol 2011;57:1126 –66. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr 2012;25:1162 –9. Bhattacharyya S, Kamperidis V, Shah BN, Roussin I, Chahal N, Li W et al. Clinical utility and prognostic value of appropriateness criteria in stress echocardiography for evaluation of valvular heart disease. J Am Coll Cardiol Img 2013;6:987 –92.
S. Bhattacharyya et al.
7. Cosyns B, Garbi M, Separovic J, Pasquet A, Lancellotti P. Education Committee of the European Association of Cardiovascular Imaging Association (EACVI): update of the echocardiography core syllabus of the European Association of Cardiovascular Imaging (EACVI). Eur Heart J Cardiovasc Imaging 2013;14:837 –9. 8. Lancellotti P, Tribouilloy C, Hagendorff A, Popescu BA, Edvardsen T, Pierard LA et al. Scientific Document Committee of the European Association of Cardiovascular Imaging. Recommendations for the echocardiographic assessment of native valvular regurgitation: an executive summary from the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging 2013;14: 611 –44. 9. Johri AM, Picard MH, Newell J, Marshall JE, King ME, Hung J. Can a teaching intervention reduce interobserver variability in LVEF assessment: a quality control exercise in the echocardiography lab. J Am Coll Cardiol Img 2011;4:821 –9. 10. Unzek S, Popovic ZB, Marwick TH. Diastolic Guidelines Concordance Investigators. Effect of recommendations on interobserver consistency of diastolic function evaluation. J Am Coll Cardiol Img 2011;4:460 –7. 11. 2013 IAC Standards and Guidelines for Accreditation Echocardiography Accreditation. http://www.intersocietal.org/echo/main/echo_standards.htm (25 April 2013, date last accessed). 12. British Society of Echocardiography Training Syllabus. http://www.bsecho.org/ media/103571/accreditation_pack_tte_-october_2013.pdf (10 November 2013, date last accessed). 13. Arbyn M, Anttila A, Jordan J, Ronco G, Schenck U, Segnan N et al. European Guidelines for Quality Assurance in Cervical Cancer Screening. Second edition—summary document. Ann Oncol 2010;21:448 – 58. 14. Perry N, Broeders M, de Wolf C, To¨rnberg S, Holland R, von Karsa L. European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition—summary document. Ann Oncol 2008;19:614 –22.
IMAGE FOCUS
.............................................................................................................................................................................
Left main trunk connecting to superior vena cava via aneurysmal coronary artery fistula Pritish Bagul*, Milind Phadke, Charan Lanjewar, Ashish Nabar, and Prafulla Kerkar Department of Cardiology, King Edward Memorial Hospital, CVTC Building, E. Borges Road, Parel, Mumbai, Maharashtra 422012, India
* Corresponding author. Tel: +91 8983108838, Email: [email protected]
A 3-year-male child presented with an incidentally detected continuous murmur during evaluation for respiratory tract infection. Two-dimensional echocardiography (Panel A) showed an aneurysmally dilated left main coronary artery (CA) with the presence of large coronary cameral fistula originating from the left main CA. Coronary computed tomography (Panels B and C) showed CA fistula extending from the left main CA, and distally opening into right-sided superior vena cava (SVC), proximal to superior vena cava-right atrium (SVC-RA) junction. The fistulous communication is seen as a tortuous vessel coursing posterior to aorta in the interatrial groove. The maximum diameter was 12 mm. CA fistulas are reported in 0.1 –0.2% of all patients undergoing selective coronary angiography. The major sites of origin of fistulae are right coronary artery (55%), left coronary artery (35%), and both coronary arteries. The major sites of terminations are right ventricle (40%), right atrium (26%), pulmonary artery (17%) less frequently SVC, CS and least in left atrium and left ventricle (2%). We report this case as proximal fistulae arising from the left main CA are rare and so also their opening in SVC is a rare entity1. Supplementary data are available at European Heart Journal – Cardiovascular Imaging online. Reference 1. Ekonomou CK et al. Coronary fistula from left main stem to main pulmonary artery. J Invasive Cardiol 2003;15 [Epub ahead of print]. Published on behalf of the European Society of Cardiology. All rights reserved. & The Author 2014. For permissions please email: [email protected]
Downloaded from by guest on June 6, 2016
doi:10.1093/ehjci/jet281 Online publish-ahead-of-print 20 January 2014
Development of a National Echocardiography Quality Improvement Programme: insights into feasibility, uptake, and clinical utility Sanjeev Bhattacharyya1,2*, Rachael James 2,3, Helen Rimington 2,4, Jane Allen 2,5, Lorraine Lee 3, Kevin Fox 1,2, Navroz D Masani 2,6, and Guy Lloyd 7, On behalf of the British Society of Echocardiography 1
Department of Cardiology, Hammersmith Hospital, Imperial College Healthcare NHS Trust, Du Cane Road, London W12 OHS, UK; 2British Society of Echocardiography, London, UK; Sussex Cardiac Centre, Brighton & Sussex University Hospitals Trust, Brighton, UK; 4Department of Echocardiography, Guy’s & St Thomas’ NHS Trust, London, UK; 5Department of Echocardiography, York Teaching Hospitals NHS Trust, York, UK; 6Department of Cardiology, University Hospital of UK, Cardiff, UK; and 7Department of Cardiology, East Sussex NHS Trust, Eastbourne, UK 3
Received 4 October 2013; accepted after revision 13 December 2013; online publish-ahead-of-print 9 January 2014
Aims
----------------------------------------------------------------------------------------------------------------------------------------------------------Keywords
cardiovascular imaging † quality † echocardiography
Introduction The concept of quality improvement in cardiovascular imaging encompasses laboratory systems and policies to ensure appropriate use of imaging, accurate acquisition and interpretation in order to optimize patient outcomes.1 Individual certification schemes have been developed to ensure that practitioners are competent at acquiring and reporting studies. Laboratory accreditation aims to ensure that adequate facilities, personnel, policies, and procedures are in place
to facilitate a high quality clinical service, including the need for an established in-house quality assurance (QA) programme.2,3 Appropriateness criteria for echocardiography have also recently been published. Several investigators have identified the clinical utility and prognostic value of implementation of these criteria.4,5,6 There is much less clarity over what represents a robust QA system, although the American Society of Echocardiography and European Association of Cardiovascular Imaging have recently published recommendations3,7
* Corresponding author. Tel: + 44 208 383 1000; Fax: + 44 208 383 4392. Email: [email protected] Published on behalf of the European Society of Cardiology. All rights reserved. & The Author 2014. For permissions please email: [email protected]
Downloaded from by guest on June 6, 2016
There are minimal data on methods to measure and improve quality in image interpretation when reporting cardiovascular imaging studies. We sought to identify the feasibility and clinical utility of a nationally implemented quality assurance (QA) process. ..................................................................................................................................................................................... Methods A web-based platform was developed to administer a national QA module to 27 echocardiography departments within the UK. Three QA modules were delivered from 2011. The proportion of units using the QA module increased from 14 and results (52.2%) in the first module to 22 (81.5%) in the third module. There was no significant change in the proportion of correct answers between the first module (88.9%) and the third module (82.8%), P ¼ 0.3. The number of echocardiographers with at least one incorrect answer increased from 16 (21.6%) in the first module to 54 (34%) in the third module, P ¼ 0.03. Overall, in valvular heart disease cases there were 36 (10.6%) incorrect responses where qualitative assessment of the severity of valve dysfunction was tested compared with 4 (3.6%) incorrect responses where quantitative assessment of valve dysfunction was tested, P ¼ 0.04. In chamber quantification cases, there were 36 (6.8%) incorrect responses where qualitative assessment of chamber function was tested compared with 3 (2.1%) incorrect responses where quantitative assessment of chamber function was tested, P ¼ 0.04. ..................................................................................................................................................................................... Conclusion The incorporation of national QA programme is feasible with rapid uptake. The platform allows comparison of an individual’s interpretation skills against a reference standard which can be used as a method to identify inter-observer variability and as a training tool.
748 Reduction of physician/sonographer inter-observer variability is important both for accuracy, and for long-term follow-up of chronic disease in particular valvular heart disease to determine timing of surgery8 and patients require serial cardiac monitoring of left ventricular function (such as those undergoing trastuzumab therapy). Inter-observer variability of 14% for determination of left ventricular (LV) ejection fraction and concordance of between 58 and 92% for determination of diastolic class have been reported.9,10 The Intersocietal Accreditation Commission has assessment of inter-observer variability within its accreditation scheme.11 The British Society of Echocardiography (BSE) introduced a dual component QA process for accredited echocardiography departments in 2011. The objective of this initiative was to provide an arena for formative assessment of echocardiographers’ interpretative skills and use this assessment to define areas for further professional development. The second aspect of the QA process was to provide a central electronic repository to promote and act as a stimulus to further on-going locally organized QA and audit activity. Currently, there are no data which describe the organization, feasibility or clinical utility of a QA process in cardiovascular imaging.
Methods The BSE QA module is an electronic, web-based platform. All individuals within accredited departments are invited to participate in a module twice a year. Each module consists of a series of five multiple choice questions based on echocardiographic images randomly allocated from the central BSE database (Figure 1). Each of the possible answers to the multiple choice question is graded as green, yellow, or red. The green answer is correct, amber is correct but could be refined and red is incorrect. The reason for the amber classification is that in echocardiography, particularly with visual interpretation of structures or valves, there may be some debate even amongst expert readers about the exact severity or nature of pathology. There is a time limit of 10 min for each question.
Question database A series of 78 questions based on the BSE core curriculum was formulated.12 Questions incorporated echocardiographic images (still and cine) covering cardiac structures [LV, right ventricle (RV), atria, pericardium] and valves/shunts (mitral, aortic, tricuspid, pulmonary, atrial septal defect, ventricular septal defect). Both qualitative and quantitative assessment of cardiac function and structure was performed. Qualitative questions utilized cine images where individuals were required to identify LV wall motion abnormalities, visually estimate LV size and ejection fraction, RV size and function, and estimate grades of valve regurgitation. Quantitative questions included m-mode, 2D, or Doppler-derived measurements where individuals were required to use the data to quantify LV and RV systolic function, LV diastolic function, cardiac chamber size and severity of valvular regurgitation, stenosis, and intra-cardiac shunts. The questions utilized a multiple choice format. Two expert readers formulated and designed the multiple choice questions. Another two expert readers re-read all the questions and associated images to ensure agreement with the answers. A consensus opinion was reached for all questions.
Individual echocardiographer participation Access to the QA module is available to accredited departments for 1 month. Individual echocardiographers are e-mailed to inform them of
the module opening and closing dates. The scores are made available to a designated QA lead in each department. The QA lead is responsible for identifying echocardiographers who have incorrectly answered questions and formulating an action plan which is uploaded onto the QA module. The plan utilized the module to identify areas of echocardiography where an individual echocardiographer needs development and further continuing professional education.
Statistics Data are reported as median and inter-quartile range or number and percentage. The x2 test was used to compare categorical variables. When the number of categorical variables was less than five the Fisher exact test was used. All tests of significance were two sided. A probability value (P) of ,0.05 was considered significant. Statistical analysis was performed using StatsDirect Version 2.5.7 (StatsDirect, UK).
Results A total of 27 echocardiography departments have BSE laboratory accreditation in the UK. A total of 193 echocardiographers [median of 8 (inter-quartile range 6 –11] per department) were registered to participate in the QA programme. A total of three QA modules have been delivered since 2011.
Temporal trends in utilization of the QA module The proportion of BSE accredited departments using the QA module has increased from 14 (52.2%) in the first module to 22 (81.5%) in the third module. The number of echocardiographers registered to participate in the QA modules increased from 77 in the first module to 193 in the third module (Table 1). The number of registered echocardiographers who actually participate increased from 74 in the first module to 159 in the third module. However, the proportion of registered users to those actually taking the module dropped from 96.1% in the first module to 82.4% in the third module.
QA module results Overall, 1745 (99.1%) questions out of a possible 1760 questions were answered over the 3 QA modules. There was no significant change in the proportion of incorrect answers (6%) between the first module and the last module (8.2%), P ¼ 0.8 (Table 2). There was no significant change in the proportion of correct answers between the first module (88.9%) and the third module (82.8%), P ¼ 0.3. On an individual basis, the number of echocardiographers with at least one incorrect answer increased from 16 (21.6%) in the first module to 54 (34%) in the third module, P ¼ 0.03.
Temporal trends in departmental performance There was no significant change in the proportion of echocardiographers per department who participated in the QA modules [(mean 80.8, 79.4 and 78.1% in the first, second, and third modules, respectively), P ¼ 0.36]. There was a significant increase in the proportion of echocardiographers per department with at least one incorrect answer [(mean 16.6, 24.5, 35.4% in the first, second, and third modules, respectively), P ¼ 0.04].
Downloaded from by guest on June 6, 2016
QA module
S. Bhattacharyya et al.
Development of a national echocardiography quality improvement programme
Figure 1 A sample question from the quality assurance module. A still image or a movie clip is displayed with the question below. The echocardiographer answers the multiple choice question. Correct answer is highlighted in green
749
Downloaded from by guest on June 6, 2016
750
Table 1
S. Bhattacharyya et al.
Trends in the usage of the QA modules
QA module
Number (%) of accredited echocardiography departments using module
Number of users eligible to participate
Number (%) of users who participated in module
............................................................................................................................................................................... November 2011
14 (51.2)
77
74 (96.1)
May 2012
21 (77.8)
134
119 (88.8)
November 2012
25 (92.6)
193
159 (82.4)
Table 2
Overall responses to the QA module questions
QA module
Green (correct) answers, n (%)
Yellow answers, n (%)
Red (incorrect) answers, n (%)
Total answers
............................................................................................................................................................................... November 2011
311 (88.9)
18 (5.1)
21 (6)
350
May 2012
495 (83.6)
50 (8.4)
47 (7.9)
592
November 2012
665 (82.8)
72 (9)
66 (8.2)
803
Pathology
Green answer, n (%)
Yellow answer, n (%)
Red answer, n (%)
287 (84.4) 92 (82.9) 464 (88.3) 113 (78.5)
17 (5) 15 (13.5) 25 (4.8) 27 (18.8)
36 (10.6) 4 (3.6) 36 (6.8) 3 (2.1)
............................................................................................................................................................................... Valve dysfunction Chambers function
Qualitative Quantitative Qualitative Quantitative
Quantitative versus qualitative assessment Overall, in valvular heart disease cases there were 36 (10.6%) incorrect responses where qualitative assessment of the severity of valve dysfunction was tested compared with 4 (3.6%) incorrect responses where quantitative assessment of valve dysfunction was tested, P ¼ 0.04 (Table 3). In chamber quantification there were 36 (6.8) incorrect responses where qualitative assessment of chamber function was tested compared with 3 (2.1%) incorrect responses where quantitative assessment of chamber function was tested, P ¼ 0.04 (Table 4).
Professional development/action plans The proportion of echocardiographers with at least one incorrect answer who had an action plan set up initially increased from 8 (50%) in the first module to 34 (97%) in the second module. This decreased in the third module to 25 (46%) (Table 4). The proportion of echocardiographers with an action plan who completed the plans learning objectives increased from 6 (75%) in the first module to 24 (96%) in the third module.
Local departmental QA Nine (33.3%) departments had used the electronic repository to document their local QA activity [median 4 episodes per department
(inter-quartile range 1 –11]. The majority of these were clinical review meetings.
Discussion This is the first study to examine the feasibility and clinical value of incorporating a nationally led quality improvement programme into echocardiography departments which focussed on image interpretation and variability. Our results demonstrate that the incorporation of the programme into clinical practice of an echocardiography can be achieved rapidly. In less than a year the proportion of eligible departments utilizing this voluntary programme increased from 52 to 82%. Furthermore, there was a steady increase in the number of echocardiographers using the QA modules. There are a range measures which are recognized as key principles in providing a high quality and consistent echocardiography practice. These include individual and laboratory accreditation and use of appropriateness criteria.1 Individual and laboratory accreditation was introduced by the BSE in 1994 and 2005, respectively. More recently, the American College of Cardiology together with respective imaging societies has published appropriateness criteria for echocardiography.4 QA has become embedded in other branches of diagnostic medicine,13,14 but has not yet been well developed in
Downloaded from by guest on June 6, 2016
Table 3 Comparison of responses to the QA modules between questions requiring qualitative interpretation and those with quantitative data
751
Development of a national echocardiography quality improvement programme
Table 4
Impact of the QA module on development of professional development plans Number (%) of echocardiographers who participated with at least one incorrect answer
Number (%) of echocardiographers with an incorrect answer who developed an action plan
Number (%) of echocardiographers where action plan has been completed
November 2011 May 2012
16 (21.6) 35 (29.4)
8 (50) 34 (97)
6 (75) 28 (82.4)
November 2012
54 (34)
25 (46)
24 (96)
QA module
...............................................................................................................................................................................
attendance have not been critically evaluated but are likely to all form a part of the improvement process.11 Only a third of centres used electronic repository to document their local QA activity. This is disappointingly low. Further investigation should be directed to examine the range and quality of local QA undertaken. Potential future applications of the QA programme include identifying inter-departmental variability and tracking changes over time. This provides an infrastructure for departments to benchmark their performance against national averages. We propose that quality standards for laboratory accreditation should include QA benchmarks. These could include the proportion of echocardiographers per department who participate in the QA modules, echocardiographic interpretation variability markers, and the proportion of echocardiographers who completed personal development plans. A limitation of this study is the effect of quality programme on an individual subsequent performance in the interpretation of echocardiographic images, has not been examined. Future studies with longitudinal follow-up of an individual’s scores will be possible once the programme has completed several more cycles. The QA programme was only made available to echocardiography departments with laboratory accreditation and therefore only 27 departments were involved. However, the clinical value is likely to be similar in other laboratories.
Conclusion The incorporation of a national cardiovascular imaging QA programme is feasible with relatively rapid uptake. The platform allows comparison of an individual’s interpretation skills against a reference standard which can be used as a training tool, a method to identify inter-observer variability and as an aid to allow the identification of further learning objectives. It also permits the documentation of on-going local departmental QA activity.
Funding British Heart Foundation Grant for development of a web-based quality assurance module. Conflicts of interest: none
References 1. Douglas PS, Chen J, Gillam L, Hendel R, Hundley WG, Masoudi F et al. Achieving quality in cardiovascular imaging II: proceedings from the Second American College of Cardiology—Duke University Medical Center Think Tank on Quality in Cardiovascular Imaging. J Am Coll Cardiol Img 2009;2:231 –40. 2. Popescu BA, Andrade MJ, Badano LP, Fox KF, Flachskampf FA, Lancellotti P et al. European Association of Echocardiography recommendations for training,
Downloaded from by guest on June 6, 2016
echocardiography. This may be because precise standards and recommendations have until recently not been published.3,7 There is general agreement that a QA programme in echocardiography should contain the following elements: regular echocardiography review meetings, systematic re-reading of echocardiographic reports for accuracy, departmental benchmarking exercises for common pathology such as a valvular regurgitation and ventricular function, and a method of feedback and remediation. The BSE QA module is not designed to supplant existing departmental processes, but rather to complement them. The outcome of approaches to quality improvement has not been described. One of the strengths of our programme is the ability for individual echocardiographers to quantify valve dysfunction or chamber function against standardized images. The modules used both images requiring only qualitative interpretation and images with quantitative data included. We identified a significant difference in the performance of echocardiographers when comparing the two. Correct quantification of both valve dysfunction and chamber function were significantly improved when quantitative data were supplied compared with when only qualitative interpretation were used. Indeed, recent data have shown that qualitative interpretation of LV ejection fraction by visual estimate has a high inter-observer variability (14%) even in an academic centre.9 A tutorial review of reference cases reduced the inter-observer variability to 8%. When quantitative methods were used the inter-observer variability was significantly less (6%). Teaching sessions reduced the variability of interpretation of these studies to 3%. Therefore, we propose our QA programme as a method to assess variability in interpretation against a reference standard and use the questions/answers as teaching materials to reduce variability. In our study, approximately a third of echocardiographers had at least one incorrect answer when compared with expert opinion. This highlights the value of the programme in identifying areas of practice where an individual echocardiographer requires further development. The number of echocardiographers with an incorrect answer increased during the study. Although purely speculative, it is possible the first cohort of centres who joined the QA scheme may have already had internal QA processes in place, whereas centres who joined the scheme subsequently did not. Reassuringly, a large proportion of these echocardiographers went on to formulate and complete an action plan to improve the skills and knowledge in the area identified. The optimal method of improving echocardiographic interpretation skills is not well defined. As previously mentioned, a teaching course with standardized images can reduce inter-observer variability.9 The effect of online casebased learning, guidelines, journal review articles, and conference
752
3.
4.
5.
6.
competence, and qualityimprovement in echocardiography. Eur J Echocardiogr 2009; 10:893–905. Picard MH, Adams D, Bierig SM, Dent JM, Douglas PS, Gillam LD et al. American Society of Echocardiography. American Society of Echocardiography recommendations for quality echocardiography laboratory operations. J Am Soc Echocardiogr 2011;24:1 –10. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR et al., American College of Cardiology Foundation Appropriate Use Criteria Task Force; American Society of Echocardiography; American Heart Association; American Society of Nuclear Cardiology; Heart Failure Society of America; Heart Rhythm Society; Society for Cardiovascular Angiography and Interventions; Society of Critical Care Medicine; Society of Cardiovascular Computed Tomography; Society for Cardiovascular Magnetic Resonance. ACCF/ASE/AHA/ASNC/ HFSA/HRS/SCAI/ SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Society of Echocardiography, American Heart Association, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular Magnetic Resonance, endorsed by the American College of Chest Physicians. J Am Coll Cardiol 2011;57:1126 –66. Bhatia RS, Carne DM, Picard MH, Weiner RB. Comparison of the 2007 and 2011 appropriate use criteria for transthoracic echocardiography in various clinical settings. J Am Soc Echocardiogr 2012;25:1162 –9. Bhattacharyya S, Kamperidis V, Shah BN, Roussin I, Chahal N, Li W et al. Clinical utility and prognostic value of appropriateness criteria in stress echocardiography for evaluation of valvular heart disease. J Am Coll Cardiol Img 2013;6:987 –92.
S. Bhattacharyya et al.
7. Cosyns B, Garbi M, Separovic J, Pasquet A, Lancellotti P. Education Committee of the European Association of Cardiovascular Imaging Association (EACVI): update of the echocardiography core syllabus of the European Association of Cardiovascular Imaging (EACVI). Eur Heart J Cardiovasc Imaging 2013;14:837 –9. 8. Lancellotti P, Tribouilloy C, Hagendorff A, Popescu BA, Edvardsen T, Pierard LA et al. Scientific Document Committee of the European Association of Cardiovascular Imaging. Recommendations for the echocardiographic assessment of native valvular regurgitation: an executive summary from the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging 2013;14: 611 –44. 9. Johri AM, Picard MH, Newell J, Marshall JE, King ME, Hung J. Can a teaching intervention reduce interobserver variability in LVEF assessment: a quality control exercise in the echocardiography lab. J Am Coll Cardiol Img 2011;4:821 –9. 10. Unzek S, Popovic ZB, Marwick TH. Diastolic Guidelines Concordance Investigators. Effect of recommendations on interobserver consistency of diastolic function evaluation. J Am Coll Cardiol Img 2011;4:460 –7. 11. 2013 IAC Standards and Guidelines for Accreditation Echocardiography Accreditation. http://www.intersocietal.org/echo/main/echo_standards.htm (25 April 2013, date last accessed). 12. British Society of Echocardiography Training Syllabus. http://www.bsecho.org/ media/103571/accreditation_pack_tte_-october_2013.pdf (10 November 2013, date last accessed). 13. Arbyn M, Anttila A, Jordan J, Ronco G, Schenck U, Segnan N et al. European Guidelines for Quality Assurance in Cervical Cancer Screening. Second edition—summary document. Ann Oncol 2010;21:448 – 58. 14. Perry N, Broeders M, de Wolf C, To¨rnberg S, Holland R, von Karsa L. European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition—summary document. Ann Oncol 2008;19:614 –22.
IMAGE FOCUS
.............................................................................................................................................................................
Left main trunk connecting to superior vena cava via aneurysmal coronary artery fistula Pritish Bagul*, Milind Phadke, Charan Lanjewar, Ashish Nabar, and Prafulla Kerkar Department of Cardiology, King Edward Memorial Hospital, CVTC Building, E. Borges Road, Parel, Mumbai, Maharashtra 422012, India
* Corresponding author. Tel: +91 8983108838, Email: [email protected]
A 3-year-male child presented with an incidentally detected continuous murmur during evaluation for respiratory tract infection. Two-dimensional echocardiography (Panel A) showed an aneurysmally dilated left main coronary artery (CA) with the presence of large coronary cameral fistula originating from the left main CA. Coronary computed tomography (Panels B and C) showed CA fistula extending from the left main CA, and distally opening into right-sided superior vena cava (SVC), proximal to superior vena cava-right atrium (SVC-RA) junction. The fistulous communication is seen as a tortuous vessel coursing posterior to aorta in the interatrial groove. The maximum diameter was 12 mm. CA fistulas are reported in 0.1 –0.2% of all patients undergoing selective coronary angiography. The major sites of origin of fistulae are right coronary artery (55%), left coronary artery (35%), and both coronary arteries. The major sites of terminations are right ventricle (40%), right atrium (26%), pulmonary artery (17%) less frequently SVC, CS and least in left atrium and left ventricle (2%). We report this case as proximal fistulae arising from the left main CA are rare and so also their opening in SVC is a rare entity1. Supplementary data are available at European Heart Journal – Cardiovascular Imaging online. Reference 1. Ekonomou CK et al. Coronary fistula from left main stem to main pulmonary artery. J Invasive Cardiol 2003;15 [Epub ahead of print]. Published on behalf of the European Society of Cardiology. All rights reserved. & The Author 2014. For permissions please email: [email protected]
Downloaded from by guest on June 6, 2016
doi:10.1093/ehjci/jet281 Online publish-ahead-of-print 20 January 2014
