Leading article
Lessons learned from national surgical audits R. S. Raju and G. J. Maddern University of Adelaide, Discipline of Surgery, The Queen Elizabeth Hospital, Woodville, South Australia 5011, and Australian Safety and Efficacy Register of New Interventional Procedures-Surgical (ASERNIP-S), Royal Australasian College of Surgeons, Adelaide, South Australia 5006, Australia (e-mail: [email protected])
Published online 16 September 2014 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9660
Audits are an important part of a surgeon’s life. Log book audits, mortality and morbidity meetings, chart audits, hospital death reviews, resource utilization audits and assessments of costeffectiveness are all regular features of the surgeon’s workload. Specialist surgical societies often have their own audits, and many surgeons participate in outcome audits of specific interventions, operations or diagnoses1 . Audits are popular because of their multiple functions relating to surgical education, patient safety, comparison of outcomes, risk management, economic stewardship and evaluation of healthcare systems. Each audit, however, takes time away from the surgeon’s clinical duties, and if an audit is to be introduced, particularly at a national level, benefit for the time, effort and cost of such large-scale data collection needs to be demonstrated. National audits aimed at improving quality have been introduced into the USA, as in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) process adopted by many centres2,3 . Although this enables comparisons between and within centres, it continues to fall well short of universal coverage. Similar efforts in the UK resulted in the National Confidential Enquiry into Patient Outcome and Death (NCEPOD) initiative4 . Both initiatives have led to improved clinical outcomes (reduction in complication3 , morbidity5 and mortality6 rates, and improved quality of care3 ) at participating institutions. © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
In Australia, a number of national audits have proven to be of significant value, despite the effort and cost associated with them. The Australian and New Zealand Audit of Surgical Mortality (ANZASM)7 was modelled on the Scottish Audit of Surgical Mortality. It began in Western Australia in 2001, and over the following 9 years was extended to cover all of Australia. The basic premise of the national audit was peer review of every in-hospital death of patients in whose care a surgeon was significantly involved8 . Feedback was provided to the treating surgeon on alternative ways of managing patients. De-identified patient summaries are published annually as an educational tool, with seminars and other educational activities based on issues identified. The audit provides aggregated data to State Departments of Health, and trends can be evaluated and analysed as well as variances investigated at a national level. An important feature in the acceptance of the audit by surgeons was that patient data provided by the views of participants and reviewers were privileged and protected legally. Surgeons feel safe in disclosing patient details with the understanding that their reports will not be used against them. Assessors have been able to express their views honestly, knowing that they will not indirectly harm the surgeon being assessed. Even before participation became mandatory, more than two-thirds of surgeons had signed up for the audit, but the requirement of compliance in order to maintain a medical licence
within Australia helped encourage the recalcitrant. The Australian Orthopaedic Association established a National Joint Replacement Registry Audit funded through the Australian Government at approximately €1.4 million (AUD $2 million; exchange rate 31 August 2014) per year. This was based on the pioneering Swedish audit of joint replacement9,10 , adopted in both the USA11 and the UK12 . Such oversight of new technology and production is vital within all jurisdictions. Factors that influence the success of a device or procedure depend not only on the technology itself but also on the context into which it is placed, including factors such as the available medical infrastructure and associated training and skills sets13 . This data collection monitors all joint replacements being put in or removed from patients within the Australian healthcare system14 . Although participation is voluntary, more than 99 per cent of all procedures have been captured, with the publication of an annual report providing detailed information on the performance of various prosthetic devices. This information has been used to identify the poorly performing artificial surface replacement metalon-metal hip joint, initially largely ignored by industry. This ongoing audit, with the high participation rate of orthopaedic surgeons at a national level, is now being used by hospitals, consumers and regulatory authorities to provide objective quality feedback on devices being implanted in patients by orthopaedic surgeons. BJS 2014; 101: 1485–1487
1486
When considering the enormous cost of orthopaedic implant surgery in the developed world, this relatively inexpensive registry seems an essential safeguard for patients, as well as for industry. Although many would wish national audits on all surgical procedures and devices, the costs and difficulties associated with analysis of the results are substantial. In the context, however, of new technologies, the ability to collect a national data set, at least for the early period of introduction, would seem valuable and likely to be cost-effective. The Australian Government commissioned the Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S) to monitor the first 18 months of implantation of endoluminal aortic grafts at the time of their introduction into the Australian healthcare system15 . This audit examined implants over an 18-month period and followed them for 7 years, resulted in evidence supporting their long-term efficacy, and led to ongoing funding within the Australian healthcare system. The principal challenge remains comprehensively to collect outcomes from all centres. Reliance on well resourced, closely audited sites runs the risk of missing important safety and efficacy outcomes with more widespread adoption. Short-term safety data are collected relatively easily, but long-term safety and efficacy results are expensive and challenging to monitor, particularly in healthcare systems that do not reliably capture most significant episodes of patient care. The Audit of Surgical Mortality has now been running in Western Australia for more than 10 years, with a 30 per cent reduction in the surgical mortality rate over a decade3 . In the 4 years of its comprehensive coverage, ANZASM has highlighted major © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
R. S. Raju and G. J. Maddern
issues for intervention. Several areas of delay in investigation, diagnosis and treatment have been identified. Delay in treatment was shown to lead to increased postoperative complications and resource utilization. Without comprehensive national data it seems impossible to convince governments or surgeons that a significant problem exists within a particular healthcare system. Individual patient death or device failures are often put down to a local or patient issue, rather than being seen as a systemic or engineering problem. The need for national audits is important within each area of surgical practice in the context of that healthcare system. The clinical results from Australia may be transferable to other countries. Factors such as the training associated with the relevant surgery, the infrastructure in which it is being performed, and the primary healthcare system and rehabilitation services available to patients after discharge from hospital are all important in determining outcome. For this reason, alerts can arise from individual national audits globally, but a robust process is needed within each country. Finding the balance between the cost, safety and accurate measurable data is difficult, but if appropriate regard can be paid to successful audits, such as those in Australia, appropriate decisions can be made that are likely to be cost-effective and patient-focused.
3
4
5
6
7
8
9
Disclosure
The authors declare no conflict of interest. 10
References 1 Boult M, Maddern GJ. Clinical audits: why and for whom. ANZ J Surg 2007; 77: 572–578. 2 Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical
www.bjs.co.uk
11
care in the Department of Veterans Affairs. Arch Surg 2002; 137: 20–27. Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009; 250: 363–376. NCEPOD. National Confidential Enquiry into Patient Outcome and Death. http://www.ncepod.org.uk [accessed 9 January 2014]. Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA Jr et al.; Principal Investigators of the Patient Safety in Surgery Study. Successful implementation of the Department of Veterans Affairs’ National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery Study. Ann Surg 2008; 248: 329–336. MacKenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Frey KP, Egleston BL et al. A national evaluation of the effect of trauma-center care on mortality. N Engl J Med 2006; 354: 366–378. Raju RS, Guy GS, Majid AJ, Babidge W, Maddern GJ. The Australian and New Zealand Audit of Surgical Mortality (ANZASM) – Birth, Deaths, and Carriage. Ann Surg 2014; [Epub ahead of print]. Azzam DG, Neo CA, Itotoh FE, Aitken RJ. The Western Australian Audit of Surgical Mortality: outcomes from the first 10 years. Med J Aust 2013; 199: 539–542. Herberts P, Ahnfelt L, Malchau H, Strömberg C, Andersson GB. Multicentre clinical trials and their value in assessing total joint arthroplasty. Clin Orthop Relat Res 1989; 249: 48–55. Ahnfelt L, Herberts P, Malchau H, Andersson GB. Prognosis of total hip replacement. A Swedish multicentre study of 4664 revisions. Acta Orthop Scand Suppl 1990; 238: 1–26. Rankin EA. AJRR: becoming a National US Joint Registry. Orthopedics 2013; 36: 175–176.
BJS 2014; 101: 1485–1487
National surgical audits
12 Horan F. The National Joint Registry for England and Wales. J Bone Joint Surg Br 2003; 85: 1–2. 13 Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern GJ. Laparoscopic skills acquisition: a study of simulation and traditional training.
1487
ANZ J Surg 2013; [Epub ahead of print]. 14 Graves SE, Davidson D, Ingerson L, Ryan P, Griffith EC, McDermott BF et al. The Australian Orthopaedic Association National Joint Replacement Registry. Med J Aust
2004; 180(Suppl): S31–S34. 15 Boult M, Babidge W, Anderson J, Denton M, Fitridge R, Harris J et al. Australian audit for the endoluminal repair of abdominal aortic aneurysm – the first 12 months. ANZ J Surg 2002; 72: 190–195.
If you wish to comment on this, or any other article published in the BJS, please visit the on-line correspondence section of the website (www.bjs.co.uk).
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1485–1487
Lessons learned from national surgical audits R. S. Raju and G. J. Maddern University of Adelaide, Discipline of Surgery, The Queen Elizabeth Hospital, Woodville, South Australia 5011, and Australian Safety and Efficacy Register of New Interventional Procedures-Surgical (ASERNIP-S), Royal Australasian College of Surgeons, Adelaide, South Australia 5006, Australia (e-mail: [email protected])
Published online 16 September 2014 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9660
Audits are an important part of a surgeon’s life. Log book audits, mortality and morbidity meetings, chart audits, hospital death reviews, resource utilization audits and assessments of costeffectiveness are all regular features of the surgeon’s workload. Specialist surgical societies often have their own audits, and many surgeons participate in outcome audits of specific interventions, operations or diagnoses1 . Audits are popular because of their multiple functions relating to surgical education, patient safety, comparison of outcomes, risk management, economic stewardship and evaluation of healthcare systems. Each audit, however, takes time away from the surgeon’s clinical duties, and if an audit is to be introduced, particularly at a national level, benefit for the time, effort and cost of such large-scale data collection needs to be demonstrated. National audits aimed at improving quality have been introduced into the USA, as in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) process adopted by many centres2,3 . Although this enables comparisons between and within centres, it continues to fall well short of universal coverage. Similar efforts in the UK resulted in the National Confidential Enquiry into Patient Outcome and Death (NCEPOD) initiative4 . Both initiatives have led to improved clinical outcomes (reduction in complication3 , morbidity5 and mortality6 rates, and improved quality of care3 ) at participating institutions. © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
In Australia, a number of national audits have proven to be of significant value, despite the effort and cost associated with them. The Australian and New Zealand Audit of Surgical Mortality (ANZASM)7 was modelled on the Scottish Audit of Surgical Mortality. It began in Western Australia in 2001, and over the following 9 years was extended to cover all of Australia. The basic premise of the national audit was peer review of every in-hospital death of patients in whose care a surgeon was significantly involved8 . Feedback was provided to the treating surgeon on alternative ways of managing patients. De-identified patient summaries are published annually as an educational tool, with seminars and other educational activities based on issues identified. The audit provides aggregated data to State Departments of Health, and trends can be evaluated and analysed as well as variances investigated at a national level. An important feature in the acceptance of the audit by surgeons was that patient data provided by the views of participants and reviewers were privileged and protected legally. Surgeons feel safe in disclosing patient details with the understanding that their reports will not be used against them. Assessors have been able to express their views honestly, knowing that they will not indirectly harm the surgeon being assessed. Even before participation became mandatory, more than two-thirds of surgeons had signed up for the audit, but the requirement of compliance in order to maintain a medical licence
within Australia helped encourage the recalcitrant. The Australian Orthopaedic Association established a National Joint Replacement Registry Audit funded through the Australian Government at approximately €1.4 million (AUD $2 million; exchange rate 31 August 2014) per year. This was based on the pioneering Swedish audit of joint replacement9,10 , adopted in both the USA11 and the UK12 . Such oversight of new technology and production is vital within all jurisdictions. Factors that influence the success of a device or procedure depend not only on the technology itself but also on the context into which it is placed, including factors such as the available medical infrastructure and associated training and skills sets13 . This data collection monitors all joint replacements being put in or removed from patients within the Australian healthcare system14 . Although participation is voluntary, more than 99 per cent of all procedures have been captured, with the publication of an annual report providing detailed information on the performance of various prosthetic devices. This information has been used to identify the poorly performing artificial surface replacement metalon-metal hip joint, initially largely ignored by industry. This ongoing audit, with the high participation rate of orthopaedic surgeons at a national level, is now being used by hospitals, consumers and regulatory authorities to provide objective quality feedback on devices being implanted in patients by orthopaedic surgeons. BJS 2014; 101: 1485–1487
1486
When considering the enormous cost of orthopaedic implant surgery in the developed world, this relatively inexpensive registry seems an essential safeguard for patients, as well as for industry. Although many would wish national audits on all surgical procedures and devices, the costs and difficulties associated with analysis of the results are substantial. In the context, however, of new technologies, the ability to collect a national data set, at least for the early period of introduction, would seem valuable and likely to be cost-effective. The Australian Government commissioned the Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S) to monitor the first 18 months of implantation of endoluminal aortic grafts at the time of their introduction into the Australian healthcare system15 . This audit examined implants over an 18-month period and followed them for 7 years, resulted in evidence supporting their long-term efficacy, and led to ongoing funding within the Australian healthcare system. The principal challenge remains comprehensively to collect outcomes from all centres. Reliance on well resourced, closely audited sites runs the risk of missing important safety and efficacy outcomes with more widespread adoption. Short-term safety data are collected relatively easily, but long-term safety and efficacy results are expensive and challenging to monitor, particularly in healthcare systems that do not reliably capture most significant episodes of patient care. The Audit of Surgical Mortality has now been running in Western Australia for more than 10 years, with a 30 per cent reduction in the surgical mortality rate over a decade3 . In the 4 years of its comprehensive coverage, ANZASM has highlighted major © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
R. S. Raju and G. J. Maddern
issues for intervention. Several areas of delay in investigation, diagnosis and treatment have been identified. Delay in treatment was shown to lead to increased postoperative complications and resource utilization. Without comprehensive national data it seems impossible to convince governments or surgeons that a significant problem exists within a particular healthcare system. Individual patient death or device failures are often put down to a local or patient issue, rather than being seen as a systemic or engineering problem. The need for national audits is important within each area of surgical practice in the context of that healthcare system. The clinical results from Australia may be transferable to other countries. Factors such as the training associated with the relevant surgery, the infrastructure in which it is being performed, and the primary healthcare system and rehabilitation services available to patients after discharge from hospital are all important in determining outcome. For this reason, alerts can arise from individual national audits globally, but a robust process is needed within each country. Finding the balance between the cost, safety and accurate measurable data is difficult, but if appropriate regard can be paid to successful audits, such as those in Australia, appropriate decisions can be made that are likely to be cost-effective and patient-focused.
3
4
5
6
7
8
9
Disclosure
The authors declare no conflict of interest. 10
References 1 Boult M, Maddern GJ. Clinical audits: why and for whom. ANZ J Surg 2007; 77: 572–578. 2 Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical
www.bjs.co.uk
11
care in the Department of Veterans Affairs. Arch Surg 2002; 137: 20–27. Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009; 250: 363–376. NCEPOD. National Confidential Enquiry into Patient Outcome and Death. http://www.ncepod.org.uk [accessed 9 January 2014]. Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA Jr et al.; Principal Investigators of the Patient Safety in Surgery Study. Successful implementation of the Department of Veterans Affairs’ National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery Study. Ann Surg 2008; 248: 329–336. MacKenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Frey KP, Egleston BL et al. A national evaluation of the effect of trauma-center care on mortality. N Engl J Med 2006; 354: 366–378. Raju RS, Guy GS, Majid AJ, Babidge W, Maddern GJ. The Australian and New Zealand Audit of Surgical Mortality (ANZASM) – Birth, Deaths, and Carriage. Ann Surg 2014; [Epub ahead of print]. Azzam DG, Neo CA, Itotoh FE, Aitken RJ. The Western Australian Audit of Surgical Mortality: outcomes from the first 10 years. Med J Aust 2013; 199: 539–542. Herberts P, Ahnfelt L, Malchau H, Strömberg C, Andersson GB. Multicentre clinical trials and their value in assessing total joint arthroplasty. Clin Orthop Relat Res 1989; 249: 48–55. Ahnfelt L, Herberts P, Malchau H, Andersson GB. Prognosis of total hip replacement. A Swedish multicentre study of 4664 revisions. Acta Orthop Scand Suppl 1990; 238: 1–26. Rankin EA. AJRR: becoming a National US Joint Registry. Orthopedics 2013; 36: 175–176.
BJS 2014; 101: 1485–1487
National surgical audits
12 Horan F. The National Joint Registry for England and Wales. J Bone Joint Surg Br 2003; 85: 1–2. 13 Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern GJ. Laparoscopic skills acquisition: a study of simulation and traditional training.
1487
ANZ J Surg 2013; [Epub ahead of print]. 14 Graves SE, Davidson D, Ingerson L, Ryan P, Griffith EC, McDermott BF et al. The Australian Orthopaedic Association National Joint Replacement Registry. Med J Aust
2004; 180(Suppl): S31–S34. 15 Boult M, Babidge W, Anderson J, Denton M, Fitridge R, Harris J et al. Australian audit for the endoluminal repair of abdominal aortic aneurysm – the first 12 months. ANZ J Surg 2002; 72: 190–195.
If you wish to comment on this, or any other article published in the BJS, please visit the on-line correspondence section of the website (www.bjs.co.uk).
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1485–1487
