Practical Radiation Oncology (2013) xx, xxx–xxx

www.practicalradonc.org

Original Report

Prospective peer review quality assurance for outpatient radiation therapy Matthew T. Ballo MD ⁎, Gregory M. Chronowski MD, Pamela J. Schlembach MD, Elizabeth S. Bloom MD, Isadora Y. Arzu MD, PhD, Deborah A. Kuban MD Department of Radiation Oncology, Regional Care Centers, University of Texas MD Anderson Cancer Center, Houston, Texas Received 16 October 2013; revised 8 November 2013; accepted 12 November 2013

Abstract Purpose: We implemented a peer review program that required presentation of all nonpalliative cases to a weekly peer review conference. The purpose of this review is to document compliance and determine how this program impacted care. Methods and materials: A total of 2988 patients were eligible for peer review. Patient data were presented to a group of physicians, physicists, and dosimetrists, and the radiation therapy plan was reviewed. Details of changes made were documented within a quality assurance note dictated after discussion. Changes recommended by the peer review process were categorized as changes to radiation dose, target, or major changes. Results: Breast cancer accounted for 47.9% of all cases, followed in frequency by head-and-neck (14.8%), gastrointestinal (9.9%), genitourinary (9.3%), and thoracic (6.7%) malignancies. Of the 2988 eligible patients, 158 (5.3%) were not presented for peer review. The number of missed presentations decreased over time; 2007, 8.2%; 2008, 5.7%; 2009, 3.8%; and 2010, 2.7% (P b .001). The reason for a missed presentation was unknown but varied by disease site and physician. Of the 2830 cases presented for peer review, a change was recommended in 346 cases (12.2%) and categorized as a dose change in 28.3%, a target change in 69.1%, and a major treatment change in 2.6%. When examined by year of treatment the number of changes recommended decreased over time: 2007, 16.5%; 2008, 11.5%; 2009, 12.5%; and 2010, 7.8% (P b .001). The number of changes recommended varied by disease site and physician. The head-and-neck, gynecologic, and gastrointestinal malignancies accounted for the majority of changes made. Conclusions: Compliance with this weekly program was satisfactory and improved over time. The program resulted in decreased treatment plan changes over time reflecting a move toward treatment consensus. We recommend that peer review be considered for patients receiving radiation therapy as it creates a culture where guideline adherence and discussion are part of normal practice. © 2013 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.

Introduction

Conflicts of interest: None. ⁎ Corresponding author. E-mail address: [email protected] (M.T. Ballo).

The Department of Radiation Oncology at University of Texas MD Anderson Cancer Center has created a worldwide network of radiation oncology practices and manages all aspects of patient care through a quality

1879-8500/$ – see front matter © 2013 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.prro.2013.11.004

2

M.T. Ballo et al

Practical Radiation Oncology: Month 2013

management program. This program includes both technical quality assurance and clinical quality assurance. To assure technical standards we have adopted uniform machine specifications and prescribe to routine physics quality checks and dose monitoring. The clinical program includes written institutional guidelines that cover all aspects of radiation dose delivery from consultation and simulation to target volume contouring, treatment planning, and delivery. This portion of the quality management program includes a requirement that peer to peer review be performed on all nonpalliative cases. Consistent with this policy, our outpatient (ie, nonTexas Medical Center based campuses) radiation therapy centers implemented a peer review program in 2007 that required presentation of all nonpalliative cases to a weekly peer review conference that included outreach physicians, physicists and dosimetrists. Patient history, physical exam findings, and histological and radiological data were presented to the group via teleconferencing and the radiation therapy plan was reviewed via an electronic medical record. The planning contours and radiation dose distributions were reviewed as a group and compared with our institutional guidelines. Details of any changes made to patient care were documented within a quality assurance note dictated after discussion. The purpose of this review is to document compliance with the program and determine how it impacted patient care from the date of inception through a period of 4 years. The requirement that all definitive cases be reviewed was based upon the assumption that peer review results in improved guideline adherence and that all disease sites benefit equally. The primary goals of this analysis were to measure guideline adherence over time and determine the overall utility of peer review for different disease sites.

4 years of study. Physician characteristics are shown in Table 1. All 5 of the physicians trained at the University of Texas MD Anderson Cancer Center Radiation Oncology residency and all were in practice for 4 or more years before we initiated the outreach peer review program. Of the 2988 patients eligible for peer review and treated by these 5 physicians, breast cancer accounted for 47.9% of them, followed in frequency by head-and-neck (14.8%), gastrointestinal (9.9%), genitourinary (9.3%), and thoracic (6.7%) malignancies (Table 2). A twice-a-week peer review conference including physicians, physicists, and dosimetrists was held via teleconferencing and patient history, physical exam findings, and histologic and radiologic data were presented to the group using a common electronic medical record. The radiation therapy plan, including dose–volume histograms, contours, and planning objectives, was reviewed via the same electronic medical record. Each case was discussed as a group and details of the discussion and any changes made to patient care were documented within a quality assurance note dictated after discussion. For this analysis the quality assurance notes were retrospectively reviewed by 1 author (M.T.B.) and any changes recommended by the peer review process were categorized as changes to radiation dose, target, or major changes. Major changes referred to changes that changed treatment modality (eg, surgery rather than external beam radiation) or added modalities (eg, chemoradiation therapy rather than radiation alone). Any recommended change, even a minor change in dose or volume, was considered important as it reflected variation from our institutional guidelines. The significance of differences between proportions was tested with the χ 2 statistic and IBM SPSS statistics, version 19 (IBM Corp, Armonk, NY), was used for data analysis.

Methods and materials Between September of 2007 and August of 2011, 4113 nonpalliative patients were eligible for peer review within our outpatient radiation therapy practices. Of these, 2988 patients were treated by 5 physicians who were present from the date of program inception (9/1/2007) through all

Table 1

Physician characteristics

Physician Physician Physician Physician Physician Physician a b

1 2 3 4 5

Years in practice a

Patient volume b

7 4 11 4 5

749 352 366 702 819

At the time of program inception. Total over 4 years studied.

Results Of the 2988 analyzed patients, 158 (5.3%) were not presented for peer review and were more frequently of craniospinal or gynecologic disease sites (P b .001). The reason for a missed presentation was unknown. The total number of missed presentations for all 5 physicians decreased significantly over time: 2007, 8.2% missed; 2008, 5.7%; 2009, 3.8%; and 2010, 2.7% (P b .001). Two physicians (physicians number 2 and 4) were responsible for the majority of the missed cases (Table 3) and for one of the physicians the number not presented decreased significantly over time (P b .001). For the other physician the number of missed presentations numerically decreased over time, but this did not reach statistical significance. Of the 2830 cases presented for peer review, a change was recommended in 346 cases (12.2%) and categorized

Practical Radiation Oncology: Month 2013 Table 2

Prospective peer review

Number of cases and number missed by diagnosis

Diagnosis

Total no. of cases

Patients not presented (%)

Breast H/N GI GU THOR CNS HEM GYN MEL/SAR Total

1432 442 295 278 200 126 105 70 40 2988

64 (4.5) 21 (4.8) 8 (2.7) 11 (4) 19 (9.5) 16 (12.7) 8 (7.6) 9 (12.9) 2 (5) 158 (5.3)

CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; GYN, gynecologic; HEM, hematologic; H/N, head and neck; MEL/ SAR, melanoma or sarcoma; THOR, thoracic.

as a dose change in 28.3%, a target change in 69.1%, and a major treatment change in 2.6%. When examined by year of treatment the total number of changes recommended decreased over time: 2007, 16.5%; 2008, 11.5%; 2009, 12.5%; and 2010, 7.8% (P b .0001). The number of changes recommended also varied significantly according to physician and diagnosis. Three physicians (physicians number 1, 2, and 5) were responsible for the majority of the changes (Table 4, P b .001) and for one of the physicians the number of changes recommended after peer review decreased significantly over the 4-year period of time studied (P b .001). For the other 2 physicians the number of changes recommended over time decreased numerically, but this did not reach statistical significance. Peer review resulted in head-and-neck, gastrointestinal, and gynecologic cases requiring a plan change more than 15% of the time (Table 5, P b .001). For these same 3 disease sites the number of changes recommended decreased significantly over the 4-year period of time studied. The number of changes recommended also decreased significantly for breast cancer cases (P b .001).

Table 3 study

Missed presentations by physician over period of

Physician

Year of treatment Total

2007

2008

2009

2010

P value

% missed presentations Physician Physician Physician Physician Physician P value

1 2 3 4 5

4 8.2 1.1 8.7 4.2 b .001

NS, not significant.

4.6 9.4 1.6 22.8 5.5

4.0 12.6 2.0 7.4 4.7

4.1 5.8 0 3.8 3.9

3.3 4.1 0 3.5 1.7

NS NS NS b .001 NS

3

Discussion Radiation oncologists have a long history of commitment to the delivery of safe and effective care. Early Patterns of Care studies determined that outcome could be directly affected through a process of quality assessment, assurance, and control whereby standards of quality are established and adherence to published guidelines is measured. 1,2 Clinical trialists recognized that the results of multicenter phase 3 trials could be impacted by the quality of radiation therapy being delivered at individual centers. The European Organization for Research and Treatment of Cancer in particular led the development of a quality assurance program that standardized all aspects of protocol treatment from machine standards and department structure to physical parameters of the radiation beam. 3 Recently, Fairchild et al 4 performed a critical review of a centralized quality assurance process used in 17 multicenter clinical trials and 1 Patterns of Care study and correlated radiation therapy plan quality with outcome. Although the definition of a protocol deviation varied considerably between studies, retrospective review of patients' plans revealed significantly higher failure rates and worse overall and progression-free survival after delivery of radiation that did not adhere to the radiation specific protocol guidelines. While oncology oriented clinical guidelines are embraced by most radiation oncology practices, they tend to be very high level and focus on the benefit of adding radiation to a patient's care rather than focusing on the details of radiation delivery. While this level of oversight works to improve the overall quality of care for populations of patients by ensuring that radiation is being recommended appropriately, the delivery of radiation doses to specific target volumes within an individual patient requires greater scrutiny. Furthermore, while there is no evidence that the radiation oncology community would be opposed to this level of oversight it is simply impractical, and in some cases impossible, for some governing body to verify that every radiation dose distribution, dose prescription, or target volume contour conforms to some national guideline. These characteristics of treatment always involve a series of medical decisions that do not have a clear right or wrong answer and instead rely on medical judgment. This level of quality assurance can only be provided through prospective peer to peer review and discussion performed in the clinic. As part of a series of white papers addressing patient safety, the American Society for Radiation Oncology recently published a report entitled “Enhancing the Role of Case-Oriented Peer Review to Improve Quality and Safety in Radiation Oncology.” 5 The report divides the process of professional decision making into its component parts from radiation oncology consultation to radiation treatment delivery and suggests areas where peer review could improve the safety and quality of care by detecting errors

4

M.T. Ballo et al Table 4 of study

Practical Radiation Oncology: Month 2013

Changes recommended by physician over period

Physician

Year of treatment Total

2007

2008

2009

2010

P value

12.5 6.8 0 3.5 9.7

NS NS NS NS .02

% changes recommended Physician Physician Physician Physician Physician P value

1 2 3 4 5

15.7 10.8 3.3 6.1 18.7 b .001

18.3 8.5 4 9.7 24

13.6 6.9 4.9 5.4 17.9

15.9 17.4 2.7 4.4 17.4

NS, not significant.

before they occur. The authors provide a framework for thinking about peer review, but provide no data to guide the development of clinical guidelines or describe how it should be performed. Boxer et al 6 did describe the impact of a peer review audit meeting held every 2 weeks, but found that treatment prescription errors were rare and that only 3.8% of patients had a change recommended after peer review. Brundage et al 7 also reported on a real-time audit program and found that a recommendation to change the treatment plan was made in 7.7% of cases. Both studies suggested that routine peer review was feasible and resulted in potential patient benefits, but neither reported plan changes over time or changes according to physician, patient, or tumor characteristics. In the current analysis we have attempted to clarify the role of peer review and provide some insight into its specific clinical effect. We examined a group of patients treated by 5 physicians who were all trained within the University of Texas MD Anderson Cancer Center system

Table 5 of study

Changes recommended by disease site over period

Diagnosis

Year of treatment Total

2007

2008

2009

2010

P value

2.6 26.1 6 2.9 2.4 8.7 13 10 8.3

b .001 0.04 0.04 NS NS NS NS .05 NS

% changes recommended Breast H/N GI GU THOR CNS HEM GYN MEL/SAR P value

6.9 34.2 15.7 5.2 7.2 7.3 12.4 18 13.2 b .001

11.5 44.8 21.2 7.6 12.2 6.5 16.1 25 11.1

5.6 32.8 17.5 7.1 6.3 0 7.1 44 11.1

7.2 34.7 20 0 7.1 12.1 13.3 4.5 25

CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; GYN, gynecologic; HEM, hematologic; H/N, head and neck; MEL/ SAR, melanoma or sarcoma; NS, not significant; THOR, thoracic.

and practicing general radiation oncology. The expectation was that patients were to be treated according to departmental standards that were available both within the published literature and within a secure University of Texas MD Anderson Cancer Center intranet where specific treatment planning and delivery guidelines were available at all times. The analysis shows that initial compliance with the program was satisfactory and, although it certainly required some degree of time and effort on the part of the physicians, only 5.3% of all cases were not presented. Furthermore, the number of cases not presented decreased over time (8.2% to 2.7%) as the physicians either became accustomed to the requirement or started to see value in its performance. Regardless of whether the reason was mandate or personal acceptance the peer review requirement was satisfied, and over a period of 4 years a culture of quality assurance was created where professional decisions could be scrutinized and discussed. We documented that the quality assurance program resulted in decreased treatment plan changes over time reflecting a move toward radiation treatment consensus and consistency. During the first year of the program, certain physicians and diagnoses were associated with a higher number of treatment changes, but these decreased over time suggesting that the act of peer review had an educational effect and the system directly changed care by increasing adherence to institutional guidelines. This finding is consistent with the one reported by the Radiation Therapy Oncology Group where centralized plan review resulted in decreased plan change recommendations over time. 8 While their data suggested a long lasting educational benefit through clinical trial participation our data suggest that this same benefit is seen in the nonprotocol environment and the simple act of peer review can have direct and measurable effects on all patients receiving radiation. We also examined whether some disease sites required a different level of inspection or if routine peer review was even necessary for all patients. The analysis indicated that 3 disease sites that are often treated with intensity modulated radiation therapy (IMRT) were also the 3 disease sites that required the highest number of changes (ie, head-and-neck, gastrointestinal, and gynecologic malignancies). It is likely that changes were seen in these 3 disease sites because IMRT requires a large degree of normal nodal tissue contouring, an inherently subjective process. 9-12 Although the department has specific guidelines for target delineation for these 3 disease sites the analysis confirmed our suspicion that there are always physician-specific decisions that are only exposed during a process of peer review. This is in contrast to craniospinal malignancies or prostate cancer where our guidelines do not require any nodal contouring and fewer changes were recommended despite the use of IMRT. Similarly, although IMRT is used for almost all thoracic malignancies few changes were seen because policy does not require elective nodal coverage.

Practical Radiation Oncology: Month 2013

In our view, peer review has broad impact on clinical practice. The finding that peer review affected the care of patients with breast cancer suggests that peer review is important for all patients, even those where IMRT is never used within our system and radiation guidelines are well established. That said, after 4 years of peer review the number of changes made to a breast plan was only 2.6%, suggesting that after group consensus has been reached peer review might not be necessary on every breast cancer patient. This is in contrast to the large number of changes recommended to head-and-neck cancer treatment plans despite 4 years of peer review. These data suggest that peer review should be mandatory for this disease site and for practical reasons should be performed before treatment planning is initiated. Waldron et al 13 reviewed their quality assurance rounds for head-and-neck patients and noted deviations from treatment guidelines in 10.5% of cases. In the majority of these cases (70%) the deviation involved coverage of the target or normal tissue. Similarly, Rosenthal et al 14 reviewed their quality assurance rounds for head-and-neck patients and reported treatment plan changes in 66% of patients, although only 10% were considered major enough to affect clinical care. Our results confirm these findings and support the recommendation that head-and-neck peer review should be performed prior to the initiation of planning. If not, peer review may result in changes to the planning contours and replanning will be required. There are several important weaknesses to consider within the current analysis, but it is hoped that this type of report acts as “a catalyst for further investigation, development and study of the efficacy of peer review techniques… .” as recommended by Marks et al. 5 (p22, supplementary material) . One important weakness is the fact that we only report on adherence to institutional guidelines which may be evidence-based, but the strength of the evidence is subject to interpretation. We therefore are not able to comment upon an improvement in care quality, but we do believe that guideline adherence represents a surrogate for quality in many circumstances. We also maintain that guidelines do define a consistent and reproducible standard of care which is of great importance when managing care among multiple campuses, and it is likely that in institutions where routine peer review is part of the culture care quality is more apt to improve as more high level evidence accumulates. Lastly, we note that the number of changes recommended according to disease site may be dependent upon the tenure and level of expertise of the treating physician and it is not clear that our method of peer review is even practical for a solo practitioner without routine access to other radiation oncologists. The benefits of peer review along the career of a radiation oncologist and according to the spectrum of clinical practices are important questions for further investigation. It is reasonable policy to present every nonpalliative patient for peer review. However, patients with head-and-

Prospective peer review

5

neck, gastrointestinal, or gynecologic malignancies ideally should have their contours and normal tissue segmentation peer reviewed prior to dosimetric planning, especially when IMRT is used. For other disease sites (eg, breast cancer or prostate cancer) peer review should still be performed but it is reasonable to have cases reviewed after planning is complete, but within some departmentally agreed upon period of time. This balanced approach acknowledges the importance of peer review for all patients but also respects the work flow of the providers affected when peer review results in a change to a patient's care.

Conclusions Maintaining quality and consistency among multiple campuses is a challenge. Our data suggest that a simple twice-a-week peer review conference creates an environment where physician's clinical decisions can be discussed and changes to patient care recommended. Over a brief period of time group consensus and consistency is increased, as is guideline adherence. Some disease sites might benefit from earlier reviews because the peer review process results in frequent changes to planning contours. We recommend that peer review guidelines acknowledge the different needs of different disease sites so that peer review is not seen as a mandate that negatively affects patient flow, but instead as an integral part of patient care.

References 1. Hanks GE. Quality control and assurance. Am J Clin Oncol. 1988;11:411-414. 2. Levitt SH, Khan F. Quality assurance in radiation oncology. Cancer. 1994;74(Suppl 9):2642-2646. 3. Kouloulias VE, Poortmans PM, Bernier J, et al. The quality assurance programme of the Radiotherapy Group of the European Organization for Research and Treatment of Cancer (EORTC): A critical appraisal of 20 years of continuous efforts. Euro J Cancer. 2003;39:430-437. 4. Fairchild A, Straube W, Laurie F, Followill D. Does quality of radiation therapy predict outcomes of multicenter cooperative group trials? A literature review. Int J Radiat Oncol Biol Phys. 2013;87: 246-260. 5. Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of caseoriented peer review to improve quality and safety in radiation oncology: Executive summary. Pract Radiat Oncol. 2013;3:149-156. 6. Boxer M, Forstner D, Kneebone A, et al. Impact of a real-time peer review audit on patient management in a radiation oncology department. J Med Imaging Radiat Oncol. 2009;53:405-411. 7. Brundage MD, Dixon PF, Mackillop WJ, et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys. 1999;43:115-124. 8. Wallner PE, Lustig RA, Pajak TF, et al. Impact of initial quality control review on study outcome in lung and head/neck cancer studies—Review of the Radiation Therapy Oncology Group experience. Int J Radiat Oncol Biol Phys. 1989;17:893-900.

6

M.T. Ballo et al

9. Fiorino C, Reni M, Bolognesi A, Cattaneo GM, Calandrino R. Intraand inter-observer variability in contouring prostate and seminal vesicles: Implications for conformal treatment planning. Radiother Oncol. 1998;47:285-292. 10. Hurkmans CW, Borger JH, Pieters BR, Russell NS, Jansen EP, Mijnheer BJ. Variability in target volume delineation on CT scans of the breast. Int J Radiat Oncol Biol Phys. 2001;50:1366-1372. 11. Jeanneret-Sozzi W, Moeckli R, Valley J, et al. The reasons for discrepancies in target volume delineation: A SASRO study on headand-neck and prostate cancers. Strahlenther Onkol. 2006;182:450-457.

Practical Radiation Oncology: Month 2013 12. Logue JP, Sharrock CL, Cowan RA, Read G, Marrs J, Mott D. Clinical variability of target volume description in conformal radiotherapy planning. Int J Radiat Oncol Biol Phys. 1998;41: 929-931. 13. Waldron J, Rocca C, O'Sullivan B, et al. Quality assurance rounds for head and neck cancer patients managed with radiation therapy. Radiother Oncol. 2002;64(Suppl 1):S119. 14. Rosenthal DI, Asper JA, Barker Jr JL, et al. Importance of patient examination to clinical quality assurance in head and neck radiation oncology. Head Neck. 2006;28:967-973.

Prospective peer review quality assurance for outpatient radiation therapy.

We implemented a peer review program that required presentation of all nonpalliative cases to a weekly peer review conference. The purpose of this rev...
142KB Sizes 3 Downloads 4 Views