Leading article
Novel approaches to surgical trials and the assessment of new surgical technologies T. D. Pinkney and D. G. M. Morton Academic Department of Surgery, Division of Cancer Sciences, University of Birmingham, Birmingham B15 2TH, UK (e-mail: [email protected])
Published online in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9705
Clinical research activity in surgery in the UK is at an all-time high, reflecting engagement between the National Institute of Health Research, the Royal College of Surgeons of England and major charitable funders, to create a system for the development and delivery of multicentre clinical trials. This initiative, with support from surgical specialty groups, led to the development of national research leads in all major specialties and the creation of five dedicated surgical clinical trials units1 . In addition, trainee-led research collaboratives now exist across the country in both general and subspecialist branches of surgery. These groups have demonstrated the power of ground-level dissemination and rapid recruitment capabilities in several recently completed projects2 – 4 . There are few reasons why similar strategies could not be applied to other healthcare systems in developed countries. It now seems appropriate to look at ways of increasing the efficiency and expanding the boundaries of surgical trials. Efficient study designs
Most randomized clinical trials undertaken in surgery are traditional two-arm trials of a single intervention with a sample size calculated to answer the single primary research question. These trials are reassuringly familiar, and generally simple to design, run and analyse. They are, however, relatively inefficient. As results are not assessed until all patients have been © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
recruited and outcome data returned, there is no opportunity for modification of interventions until it is too late. They also force investigators to ‘put all of their eggs in one basket’ regarding the decision about which intervention to back. There is a move towards multiarm, multistage (MAMS) trials5,6 across medicine and these are also starting to emerge in surgery. These trials, which originated in oncology where various drug regimens can be tested in parallel, lend themselves to situations in which multiple similar interventions may exist and the primary outcome of interest is available soon after the intervention stage. This is often the case in trials of surgical or perioperative interventions, where short-term outcomes such as morbidity (for example postoperative pneumonia or surgical-site infection), length of stay or in-hospital death form the primary outcome. These trial designs are undoubtedly more efficient as they allow the rapid and parallel assessment of multiple similar interventions, and investigators can deliver effectively the equivalent of several traditional two-arm superiority trials simultaneously within one quotient of set-up and running costs. If one intervention is ineffective during the course of the trial, an alternative may be substituted7 . The efficiency gains extend further, as it has been shown that increasing the number of research arms in a trial will increase the probability of proving reliably that at least one of the new treatments is superior to the control6 . Finally, the addition
of adaptive randomization strategies, whereby arm sizes and recruitment proportions can be modified during the course of the trial, can further enhance the efficiency of a MAMS trial, by potentially decreasing the time until an efficacy decision can be made. These complex trial designs can be challenging to design, power and analyse, and an appropriately experienced trials unit and statistical team are paramount to their success. Early-phase and devices trials in surgery – a new pathway
Although the initial period of modern surgical practice was characterized by significant advances in anatomy, physiology, pathology and anaesthetics, most progress in the past two or three decades has been the result of technological innovation. The resulting advances in areas such as access, instrumentation and diagnostics have often been introduced in a haphazard manner with little formal testing of effectiveness. The recent development of the IDEAL (Idea, Development, Exploration, Assessment, Long-term study) framework for the assessment and reporting of new surgical innovations aims to improve the quality of research by emphasizing appropriate methods, transparency of data and rigorous reporting of outcomes8 . Despite this, only a handful of reports adhering to this framework have appeared to date. Part of the problem with devices trials is the convoluted processes BJS 2015; 102: e10–e11
Surgical trials and assessment of new surgical technologies
and multiple hurdles involved in bringing a new product from design, through regulation and into human trials. Although regulatory processes are clearly important, the development pipeline often fails to connect designers with clinical applications. A number of parties are looking at developing more integrated development pathways. These seem poorly developed across Europe. Organizations that represent surgeons, research funders and regulatory bodies need to work together to provide an integrated pathway that can address issues of patient acceptability, patient mapping and clinical development plans by facilitating the timely and coordinated involvement of expert clinicians and trials units into the process. Ideally, this would be through a single portal of access for new ideas. Developing such an integrated structure for the safe and efficient evaluation of new technologies has never been more important and should be a priority for many healthcare systems. Surgeons should play a central role in the delivery of such a programme. Shortening the development pipeline
A recent development that may complement the above is the realization that the traditional progression from phase I (first in humans) to phase II (safety, reproducibility, acceptability) and then phase III (clinical efficacy) trials of a new device or intervention can be shortened by eliminating gaps between the phases. Trial design
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
e11
can incorporate seamless advancement with minimal pauses. This method has been demonstrated recently in the ROCSS (Reinforcement of Closure of Stoma Site) trial, which explored the use of a biological mesh to prevent herniation at the site of stoma closure. The biological mesh itself had all relevant CE markings, but the placement and fixation of the mesh in this role was entirely novel. The surgical technique was developed in a single-centre cohort series (phase I trial; IDEAL framework stage 1–2a) and this technique disseminated via publication9 . A randomized internal pilot and feasibility study of 90 patients (phase II trial; IDEAL stage 2b) was then undertaken at five centres, which confirmed the safety and acceptability of this new technique. This rolled seamlessly into the full phase III trial (IDEAL stage 3) establishing the efficacy of the intervention, and currently 270 of the 560 needed have been recruited from 21 centres. The advances and initiatives described above should enhance the efficiency and quality of clinical surgical research, and ultimately result in improved outcomes for patients in the future. Disclosure
The authors declare no conflict of interest. References 1 McCall B. UK implements national programme for surgical trials. Lancet 2013; 382: 1083–1084.
www.bjs.co.uk
2 Pinkney TD, Calvert M, Bartlett DC, Gheorghe A, Redman V, Dowswell G et al.; West Midlands Research Collaborative; ROSSINI Trial Investigators. Impact of wound edge protection devices on surgical site infection after laparotomy: multicentre randomised controlled trial (ROSSINI Trial). BMJ 2013; 347: f4305. 3 National Surgical Research Collaborative. Multicentre observational study of performance variation in provision and outcome of emergency appendicectomy. Br J Surg 2013; 100: 1240–1252. 4 Bhangu A, Kolias AG, Pinkney T, Hall NJ, Fitzgerald JE. Surgical research collaboratives in the UK. Lancet 2013; 382: 1091–1092. 5 Freidlin B, Korn EL, Gray R, Martin A. Multi-arm clinical trials of new agents: some design considerations. Clin Cancer Res 2008; 14: 4368–4371. 6 Parmar MKB, Carpenter J, Sydes MR. More multiarm randomised trials of superiority are needed. Lancet 2014; 384: 283–284. 7 Sydes MR, Parmar MKB, Mason MD, Clarke NW, Amos C, Anderson J, et al. Flexible trial design in practice – stopping arms for lack-of-benefit and adding research arms mid-trial in STAMPEDE: a multi-arm multi-stage randomized controlled trial. Trials 2012; 13: 168. 8 McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009; 374: 1105–1112. 9 Bhangu A, Futaba K, Patel A, Pinkney T, Morton D. Reinforcement of closure of stoma site using a biological mesh. Tech Coloproctol 2014; 18: 305–308.
BJS 2015; 102: e10–e11
Copyright of British Journal of Surgery is the property of John Wiley & Sons, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Novel approaches to surgical trials and the assessment of new surgical technologies T. D. Pinkney and D. G. M. Morton Academic Department of Surgery, Division of Cancer Sciences, University of Birmingham, Birmingham B15 2TH, UK (e-mail: [email protected])
Published online in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9705
Clinical research activity in surgery in the UK is at an all-time high, reflecting engagement between the National Institute of Health Research, the Royal College of Surgeons of England and major charitable funders, to create a system for the development and delivery of multicentre clinical trials. This initiative, with support from surgical specialty groups, led to the development of national research leads in all major specialties and the creation of five dedicated surgical clinical trials units1 . In addition, trainee-led research collaboratives now exist across the country in both general and subspecialist branches of surgery. These groups have demonstrated the power of ground-level dissemination and rapid recruitment capabilities in several recently completed projects2 – 4 . There are few reasons why similar strategies could not be applied to other healthcare systems in developed countries. It now seems appropriate to look at ways of increasing the efficiency and expanding the boundaries of surgical trials. Efficient study designs
Most randomized clinical trials undertaken in surgery are traditional two-arm trials of a single intervention with a sample size calculated to answer the single primary research question. These trials are reassuringly familiar, and generally simple to design, run and analyse. They are, however, relatively inefficient. As results are not assessed until all patients have been © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
recruited and outcome data returned, there is no opportunity for modification of interventions until it is too late. They also force investigators to ‘put all of their eggs in one basket’ regarding the decision about which intervention to back. There is a move towards multiarm, multistage (MAMS) trials5,6 across medicine and these are also starting to emerge in surgery. These trials, which originated in oncology where various drug regimens can be tested in parallel, lend themselves to situations in which multiple similar interventions may exist and the primary outcome of interest is available soon after the intervention stage. This is often the case in trials of surgical or perioperative interventions, where short-term outcomes such as morbidity (for example postoperative pneumonia or surgical-site infection), length of stay or in-hospital death form the primary outcome. These trial designs are undoubtedly more efficient as they allow the rapid and parallel assessment of multiple similar interventions, and investigators can deliver effectively the equivalent of several traditional two-arm superiority trials simultaneously within one quotient of set-up and running costs. If one intervention is ineffective during the course of the trial, an alternative may be substituted7 . The efficiency gains extend further, as it has been shown that increasing the number of research arms in a trial will increase the probability of proving reliably that at least one of the new treatments is superior to the control6 . Finally, the addition
of adaptive randomization strategies, whereby arm sizes and recruitment proportions can be modified during the course of the trial, can further enhance the efficiency of a MAMS trial, by potentially decreasing the time until an efficacy decision can be made. These complex trial designs can be challenging to design, power and analyse, and an appropriately experienced trials unit and statistical team are paramount to their success. Early-phase and devices trials in surgery – a new pathway
Although the initial period of modern surgical practice was characterized by significant advances in anatomy, physiology, pathology and anaesthetics, most progress in the past two or three decades has been the result of technological innovation. The resulting advances in areas such as access, instrumentation and diagnostics have often been introduced in a haphazard manner with little formal testing of effectiveness. The recent development of the IDEAL (Idea, Development, Exploration, Assessment, Long-term study) framework for the assessment and reporting of new surgical innovations aims to improve the quality of research by emphasizing appropriate methods, transparency of data and rigorous reporting of outcomes8 . Despite this, only a handful of reports adhering to this framework have appeared to date. Part of the problem with devices trials is the convoluted processes BJS 2015; 102: e10–e11
Surgical trials and assessment of new surgical technologies
and multiple hurdles involved in bringing a new product from design, through regulation and into human trials. Although regulatory processes are clearly important, the development pipeline often fails to connect designers with clinical applications. A number of parties are looking at developing more integrated development pathways. These seem poorly developed across Europe. Organizations that represent surgeons, research funders and regulatory bodies need to work together to provide an integrated pathway that can address issues of patient acceptability, patient mapping and clinical development plans by facilitating the timely and coordinated involvement of expert clinicians and trials units into the process. Ideally, this would be through a single portal of access for new ideas. Developing such an integrated structure for the safe and efficient evaluation of new technologies has never been more important and should be a priority for many healthcare systems. Surgeons should play a central role in the delivery of such a programme. Shortening the development pipeline
A recent development that may complement the above is the realization that the traditional progression from phase I (first in humans) to phase II (safety, reproducibility, acceptability) and then phase III (clinical efficacy) trials of a new device or intervention can be shortened by eliminating gaps between the phases. Trial design
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
e11
can incorporate seamless advancement with minimal pauses. This method has been demonstrated recently in the ROCSS (Reinforcement of Closure of Stoma Site) trial, which explored the use of a biological mesh to prevent herniation at the site of stoma closure. The biological mesh itself had all relevant CE markings, but the placement and fixation of the mesh in this role was entirely novel. The surgical technique was developed in a single-centre cohort series (phase I trial; IDEAL framework stage 1–2a) and this technique disseminated via publication9 . A randomized internal pilot and feasibility study of 90 patients (phase II trial; IDEAL stage 2b) was then undertaken at five centres, which confirmed the safety and acceptability of this new technique. This rolled seamlessly into the full phase III trial (IDEAL stage 3) establishing the efficacy of the intervention, and currently 270 of the 560 needed have been recruited from 21 centres. The advances and initiatives described above should enhance the efficiency and quality of clinical surgical research, and ultimately result in improved outcomes for patients in the future. Disclosure
The authors declare no conflict of interest. References 1 McCall B. UK implements national programme for surgical trials. Lancet 2013; 382: 1083–1084.
www.bjs.co.uk
2 Pinkney TD, Calvert M, Bartlett DC, Gheorghe A, Redman V, Dowswell G et al.; West Midlands Research Collaborative; ROSSINI Trial Investigators. Impact of wound edge protection devices on surgical site infection after laparotomy: multicentre randomised controlled trial (ROSSINI Trial). BMJ 2013; 347: f4305. 3 National Surgical Research Collaborative. Multicentre observational study of performance variation in provision and outcome of emergency appendicectomy. Br J Surg 2013; 100: 1240–1252. 4 Bhangu A, Kolias AG, Pinkney T, Hall NJ, Fitzgerald JE. Surgical research collaboratives in the UK. Lancet 2013; 382: 1091–1092. 5 Freidlin B, Korn EL, Gray R, Martin A. Multi-arm clinical trials of new agents: some design considerations. Clin Cancer Res 2008; 14: 4368–4371. 6 Parmar MKB, Carpenter J, Sydes MR. More multiarm randomised trials of superiority are needed. Lancet 2014; 384: 283–284. 7 Sydes MR, Parmar MKB, Mason MD, Clarke NW, Amos C, Anderson J, et al. Flexible trial design in practice – stopping arms for lack-of-benefit and adding research arms mid-trial in STAMPEDE: a multi-arm multi-stage randomized controlled trial. Trials 2012; 13: 168. 8 McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009; 374: 1105–1112. 9 Bhangu A, Futaba K, Patel A, Pinkney T, Morton D. Reinforcement of closure of stoma site using a biological mesh. Tech Coloproctol 2014; 18: 305–308.
BJS 2015; 102: e10–e11
Copyright of British Journal of Surgery is the property of John Wiley & Sons, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
![[The patulous eustachian tube-novel surgical approaches].](/assets/img/hold.png)
