BJD
British Journal of Dermatology
E DI TO R IA L
The outstanding record of clinical trials in the British Journal of Dermatology
DOI: 10.1111/bjd.13011
Biggest publisher of dermatology trials In the year that the British Journal of Dermatology (BJD) celebrates its 125th year of production, it is timely to look back at the journal’s contribution to promoting and publishing randomized controlled clinical trials (RCTs). Along with systematic reviews, RCTs are the cornerstones of evidence-based decisions about treatment effectiveness for people with skin diseases.1 Since the first modern RCT of streptomycin for tuberculosis was published in 1948,2 RCTs and their principles of random allocation of participants and blinded assessment of outcomes have remained the study design most likely to minimize bias when assessing treatments. Most readers will be familiar with coming across trial reports in the BJD, but what may surprise many is that the BJD has published more trials than any other dermatology journal, whether one looks over the last 70 years or the last 10 years, as shown in Table 1. This is an outstanding record for a British journal.
But what about the quality? Although a large quantity of trials might sound impressive, it is important to consider the quality of those published trials. Although no recent head-to-head comparison of trial-reporting quality across the main dermatology journals is available, previous surveys of RCTs, including those conducted by the European Dermato-Epidemiology Network, suggest that the quality of reporting in the BJD is as good as, or better than, other dermatology journals depending on the items examined.3–7 Scrutiny of the topics of trials published in the BJD would suggest that they are not publishing the dregs that others journals reject. Although some high-impact dermatology trials are lost to the general medical journals,8 the BJD has published several influential trials such as the CHAMPION trial of adalimumab against methotrexate for psoriasis,9 the pivotal study of oral alitretinoin for chronic hand dermatitis10 and the demonstration of the superiority of photodynamic therapy over cryotherapy for Bowen disease,11 with a combined citation count of almost 500.
Consorting with CONSORT One important development in improving the quality of reporting of clinical trials in the BJD has been the adoption of the Consolidated Standards of Reporting Trials (CONSORT) © 2014 British Association of Dermatologists
Statement (http://www.consort-statement.org/), which the author introduced with the late Neil Cox in 2000.12 The CONSORT checklist is not just yet another annoying list of things to do after completing research, but a way of making the architecture and methods of a trial visible so that readers can judge the quality of the trial and make decisions about its utility. Just as one would not buy a second-hand car without seeing a service history record, evidence of roadworthiness, registration and tax – so readers should not accept a clinical trial without a clear description of basic features such as the method of generating and concealing the randomization sequence, blinding and whether all randomized items were included in the final analysis.13 As Table 2 shows, quality of reporting is not quite the same as study quality, but unless the key features of a trial are reported, it is difficult to distinguish between a poor-quality study and a sparkling diamond that has been poorly reported. It is also worth bearing in mind that clinical trials have a second life as they contribute to systematic summaries of RCTs such as those conducted by the Cochrane Skin Group. Time and time again, the risk of bias for individual clinical trials included in Cochrane reviews is simply unclear, as key items from constituent trials are completely missing.14 There is some evidence that adoption of the CONSORT checklist has resulted in improvement in quality of reporting of RCTs published in the BJD,6 but there is still room for improvement in implementing complete Table 1 Number of clinical trials published in leading dermatology journals from 1946 to 30 January 2014 and in the last 10 years
British Journal of Dermatology Journal of the American Academy of Dermatology Journal of Investigative Dermatology Archives/JAMA Dermatology
Medline (1946 to 30 January 2014)
Medline (2004 to 30 January 2014)
950
327
765
223
148
36
573
165
Articles were extracted from Medline using the terms randomized controlled trial.pt or controlled clinical trial.pt and the journal name. Data kindly provided by Ms Elizabeth Doney, trials search coordinator of the Cochrane Skin Group.
British Journal of Dermatology (2014) 170, pp761–763
761
762 Editorial Table 2 Trial-reporting and study quality may be related but are not necessarily the same. All poorly reported trials reside in the grey zone of uncertainty. From Williams13
Reporting quality
Study quality Good
Flawed
Clear
May be helpful for clinical practice
Poor
A sparkling diamond – but how do you know?
At least you can tell it is flawed and make a judgement on utility Difficult to distinguish from a good but poorly reported study
reporting – a task that is being addressed by the appointment of Dr Jonathan Batchelor to the journal. Indeed the author, who teaches critical appraisal on RCTs, is now finding it
increasingly difficult to find good examples of significantly flawed RCTs published in the BJD to teach on.15
New developments and challenges around the corner The publication in 2010 of a virtual issue of RCTs published over a 2-year period by the BJD,16 with direct links to the original articles, was an interesting idea that could be repeated every 5 years or so for those with an interest in reviewing the content and quality of published clinical trials. One area that the BJD could improve on is making sure that all published trials have been registered, along with their plan of investigation, in a publically accessible trial registry before recruitment takes place. The problem here is a simple one – that of researchers highlighting those outcomes that ‘look good’ rather than those primary outcomes on which the sample size and hypothesis testing were predicated upon – a bit like throwing a dart and then drawing a dartboard around where the dart lands (Fig. 1). Also known as selective reporting outcome bias, its pernicious effect in distorting the scientific record has been well documented in the general and dermatological literature.17,18 Although the BJD supported the notion of trial registration as far back in 2005 in an editorial by a section editor of the time,19 trial registration is still not compulsory with the BJD, even though it is mentioned as an item on the CONSORT checklist. The tension in imposing mandatory trial registration is that some trials might end up not being reported at all, and hence contribute to the arguably worse situation of not publishing trials or favouring those larger well-funded trials that produce ‘positive’ results (i.e. publication bias). Some sort of short amnesty period might therefore be needed for existing trials in the editorial process to be accepted without trial registration, but the journal should not sway from the challenge of trial registration that has been adopted by other leading dermatology journals.20 The BJD has a remarkable record for publishing a large number of clinical trials that inform patient care and for taking steps for continuous improvement of the quality of reporting of such trials. In this regard, the journal is a beacon for evidence-based dermatology. Long may the BJD continue to play a key role in influencing improved standards in the design, conduct and reporting of clinical trials in skin disease.
Acknowledgments The author wishes to thank Ms Elizabeth Doney (trials search coordinator of the Cochrane Skin Group) and Mr John Caulfield (Editorial Manager of the BJD) for the data on clinical trials shown in Table 1, and Ms Maria Khan (Editorial Director for Global Research at Wiley-Blackwell) for providing data on trial citations. Fig 1. Failure to declare the primary outcomes for a clinical trial up front, and just highlighting the ones that look impressive after analysis has taken place, is akin to throwing a dart and then drawing a dartboard around where it lands. British Journal of Dermatology (2014) 170, pp761–763
Conflicts of interest H.W. is a member of the BJD’s editorial advisory board and is clinical trials editor for the Journal of Investigative Dermatology. He is © 2014 British Association of Dermatologists
Editorial
an enthusiast of evidence-based medicine and chair of the U.K. Dermatology Clinical Trials Network. Centre of Evidence-Based Dermatology, King’s Meadow Campus, University of Nottingham, Lenton Lane, Nottingham NG7 2NR, U.K. E-mail: [email protected]
H. WILLIAMS
References 1 Williams HC, Dellavalle RP. The growth of clinical trials and systematic reviews in informing dermatological patient care. J Invest Dermatol 2012; 132:1008–17. 2 A Medical Research Council investigation. Streptomycin treatment of pulmonary tuberculosis. Br Med J 1948; 2:769–82. 3 Adetugbo K, Williams H. How well are randomized controlled trials reported in the dermatology literature? Arch Dermatol 2000; 136:381–5. 4 Naldi N, Svensson A, Diepgen T et al. Randomized clinical trials for psoriasis 1987–2000: the EDEN survey. J Invest Dermatol 2003; 120:738–41. 5 van Coevorden AM, Coenraads PJ, Svensson A et al. Overview of therapeutic studies of treatment for hand eczema – the EDEN hand eczema survey. Br J Dermatol 2004; 151:446–51. 6 Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol 2009; 161:1159–65. 7 Naldi L, Svensson A, Zenoni D et al. Comparators, study duration, outcome measures and sponsorship in therapeutic trials of psoriasis: update of the EDEN Psoriasis Survey 2001–2006. Br J Dermatol 2010; 162:384–9. 8 Thomas KS, Crook AM, Nunn AJ et al. Penicillin to prevent leg cellulitis recurrence. N Engl J Med 2013; 368:1695–703.
© 2014 British Association of Dermatologists
763
9 Saurat JH, Stingl G, Dubertret L et al. Efficacy and safety results from the randomized controlled comparative study of adalimumab vs. methotrexate vs. placebo in patients with psoriasis (CHAMPION). Br J Dermatol 2008; 158:558–66. 10 Ruzicka T, Lynde CW, Jemec GB et al. Efficacy and safety of oral alitretinoin (9-cis retinoic acid) in patients with severe chronic hand eczema refractory to topical corticosteroids: results of a randomized, double-blind, placebo-controlled, multicentre trial. Br J Dermatol 2008; 158:808–17. 11 Morton CA, Whitehurst C, Moseley H et al. Comparison of photodynamic therapy with cryotherapy in the treatment of Bowen’s disease. Br J Dermatol 1996; 135:766–71. 12 Cox N, Williams HC. Can you COPE with CONSORT? Br J Dermatol 2000; 142:1–7. 13 Williams HC. Cars, CONSORT 2010, and clinical practice. Trials 2010; 11:33. 14 Whitton ME, Pinart M, Batchelor J et al. Interventions for vitiligo. Cochrane Database Syst Rev 2010; 1:CD003263. 15 Rigopoulos D, Gregoriou S, Belyayeva E et al. Efficacy and safety of tacrolimus ointment 0.1% vs. betamethasone 17-valerate 0.1% in the treatment of chronic paronychia: an unblinded randomized study. Br J Dermatol 2009; 160:858–60. 16 Wiley Online Library. Virtual issues. Randomised controlled trials. Br J Dermatol. Available at: http://onlinelibrary.wiley.com/journal/ 10.1111/%28ISSN%291365-2133/homepage/VirtualIssuesPage. html (last accessed 24 February 2014). 17 Chan AW, Hr objartsson A, Haahr MT et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291:2457–65. 18 Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol 2012; 132:2727–34. 19 Ormerod AD, Williams HC. Compulsory registration of trials. Br J Dermatol 2005; 152:859–60. 20 Williams HC, Goldsmith L. The JID opens its doors to high-quality randomized controlled clinical trials. J Invest Dermatol 2006; 126:1683–4.
British Journal of Dermatology (2014) 170, pp761–763
British Journal of Dermatology
E DI TO R IA L
The outstanding record of clinical trials in the British Journal of Dermatology
DOI: 10.1111/bjd.13011
Biggest publisher of dermatology trials In the year that the British Journal of Dermatology (BJD) celebrates its 125th year of production, it is timely to look back at the journal’s contribution to promoting and publishing randomized controlled clinical trials (RCTs). Along with systematic reviews, RCTs are the cornerstones of evidence-based decisions about treatment effectiveness for people with skin diseases.1 Since the first modern RCT of streptomycin for tuberculosis was published in 1948,2 RCTs and their principles of random allocation of participants and blinded assessment of outcomes have remained the study design most likely to minimize bias when assessing treatments. Most readers will be familiar with coming across trial reports in the BJD, but what may surprise many is that the BJD has published more trials than any other dermatology journal, whether one looks over the last 70 years or the last 10 years, as shown in Table 1. This is an outstanding record for a British journal.
But what about the quality? Although a large quantity of trials might sound impressive, it is important to consider the quality of those published trials. Although no recent head-to-head comparison of trial-reporting quality across the main dermatology journals is available, previous surveys of RCTs, including those conducted by the European Dermato-Epidemiology Network, suggest that the quality of reporting in the BJD is as good as, or better than, other dermatology journals depending on the items examined.3–7 Scrutiny of the topics of trials published in the BJD would suggest that they are not publishing the dregs that others journals reject. Although some high-impact dermatology trials are lost to the general medical journals,8 the BJD has published several influential trials such as the CHAMPION trial of adalimumab against methotrexate for psoriasis,9 the pivotal study of oral alitretinoin for chronic hand dermatitis10 and the demonstration of the superiority of photodynamic therapy over cryotherapy for Bowen disease,11 with a combined citation count of almost 500.
Consorting with CONSORT One important development in improving the quality of reporting of clinical trials in the BJD has been the adoption of the Consolidated Standards of Reporting Trials (CONSORT) © 2014 British Association of Dermatologists
Statement (http://www.consort-statement.org/), which the author introduced with the late Neil Cox in 2000.12 The CONSORT checklist is not just yet another annoying list of things to do after completing research, but a way of making the architecture and methods of a trial visible so that readers can judge the quality of the trial and make decisions about its utility. Just as one would not buy a second-hand car without seeing a service history record, evidence of roadworthiness, registration and tax – so readers should not accept a clinical trial without a clear description of basic features such as the method of generating and concealing the randomization sequence, blinding and whether all randomized items were included in the final analysis.13 As Table 2 shows, quality of reporting is not quite the same as study quality, but unless the key features of a trial are reported, it is difficult to distinguish between a poor-quality study and a sparkling diamond that has been poorly reported. It is also worth bearing in mind that clinical trials have a second life as they contribute to systematic summaries of RCTs such as those conducted by the Cochrane Skin Group. Time and time again, the risk of bias for individual clinical trials included in Cochrane reviews is simply unclear, as key items from constituent trials are completely missing.14 There is some evidence that adoption of the CONSORT checklist has resulted in improvement in quality of reporting of RCTs published in the BJD,6 but there is still room for improvement in implementing complete Table 1 Number of clinical trials published in leading dermatology journals from 1946 to 30 January 2014 and in the last 10 years
British Journal of Dermatology Journal of the American Academy of Dermatology Journal of Investigative Dermatology Archives/JAMA Dermatology
Medline (1946 to 30 January 2014)
Medline (2004 to 30 January 2014)
950
327
765
223
148
36
573
165
Articles were extracted from Medline using the terms randomized controlled trial.pt or controlled clinical trial.pt and the journal name. Data kindly provided by Ms Elizabeth Doney, trials search coordinator of the Cochrane Skin Group.
British Journal of Dermatology (2014) 170, pp761–763
761
762 Editorial Table 2 Trial-reporting and study quality may be related but are not necessarily the same. All poorly reported trials reside in the grey zone of uncertainty. From Williams13
Reporting quality
Study quality Good
Flawed
Clear
May be helpful for clinical practice
Poor
A sparkling diamond – but how do you know?
At least you can tell it is flawed and make a judgement on utility Difficult to distinguish from a good but poorly reported study
reporting – a task that is being addressed by the appointment of Dr Jonathan Batchelor to the journal. Indeed the author, who teaches critical appraisal on RCTs, is now finding it
increasingly difficult to find good examples of significantly flawed RCTs published in the BJD to teach on.15
New developments and challenges around the corner The publication in 2010 of a virtual issue of RCTs published over a 2-year period by the BJD,16 with direct links to the original articles, was an interesting idea that could be repeated every 5 years or so for those with an interest in reviewing the content and quality of published clinical trials. One area that the BJD could improve on is making sure that all published trials have been registered, along with their plan of investigation, in a publically accessible trial registry before recruitment takes place. The problem here is a simple one – that of researchers highlighting those outcomes that ‘look good’ rather than those primary outcomes on which the sample size and hypothesis testing were predicated upon – a bit like throwing a dart and then drawing a dartboard around where the dart lands (Fig. 1). Also known as selective reporting outcome bias, its pernicious effect in distorting the scientific record has been well documented in the general and dermatological literature.17,18 Although the BJD supported the notion of trial registration as far back in 2005 in an editorial by a section editor of the time,19 trial registration is still not compulsory with the BJD, even though it is mentioned as an item on the CONSORT checklist. The tension in imposing mandatory trial registration is that some trials might end up not being reported at all, and hence contribute to the arguably worse situation of not publishing trials or favouring those larger well-funded trials that produce ‘positive’ results (i.e. publication bias). Some sort of short amnesty period might therefore be needed for existing trials in the editorial process to be accepted without trial registration, but the journal should not sway from the challenge of trial registration that has been adopted by other leading dermatology journals.20 The BJD has a remarkable record for publishing a large number of clinical trials that inform patient care and for taking steps for continuous improvement of the quality of reporting of such trials. In this regard, the journal is a beacon for evidence-based dermatology. Long may the BJD continue to play a key role in influencing improved standards in the design, conduct and reporting of clinical trials in skin disease.
Acknowledgments The author wishes to thank Ms Elizabeth Doney (trials search coordinator of the Cochrane Skin Group) and Mr John Caulfield (Editorial Manager of the BJD) for the data on clinical trials shown in Table 1, and Ms Maria Khan (Editorial Director for Global Research at Wiley-Blackwell) for providing data on trial citations. Fig 1. Failure to declare the primary outcomes for a clinical trial up front, and just highlighting the ones that look impressive after analysis has taken place, is akin to throwing a dart and then drawing a dartboard around where it lands. British Journal of Dermatology (2014) 170, pp761–763
Conflicts of interest H.W. is a member of the BJD’s editorial advisory board and is clinical trials editor for the Journal of Investigative Dermatology. He is © 2014 British Association of Dermatologists
Editorial
an enthusiast of evidence-based medicine and chair of the U.K. Dermatology Clinical Trials Network. Centre of Evidence-Based Dermatology, King’s Meadow Campus, University of Nottingham, Lenton Lane, Nottingham NG7 2NR, U.K. E-mail: [email protected]
H. WILLIAMS
References 1 Williams HC, Dellavalle RP. The growth of clinical trials and systematic reviews in informing dermatological patient care. J Invest Dermatol 2012; 132:1008–17. 2 A Medical Research Council investigation. Streptomycin treatment of pulmonary tuberculosis. Br Med J 1948; 2:769–82. 3 Adetugbo K, Williams H. How well are randomized controlled trials reported in the dermatology literature? Arch Dermatol 2000; 136:381–5. 4 Naldi N, Svensson A, Diepgen T et al. Randomized clinical trials for psoriasis 1987–2000: the EDEN survey. J Invest Dermatol 2003; 120:738–41. 5 van Coevorden AM, Coenraads PJ, Svensson A et al. Overview of therapeutic studies of treatment for hand eczema – the EDEN hand eczema survey. Br J Dermatol 2004; 151:446–51. 6 Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol 2009; 161:1159–65. 7 Naldi L, Svensson A, Zenoni D et al. Comparators, study duration, outcome measures and sponsorship in therapeutic trials of psoriasis: update of the EDEN Psoriasis Survey 2001–2006. Br J Dermatol 2010; 162:384–9. 8 Thomas KS, Crook AM, Nunn AJ et al. Penicillin to prevent leg cellulitis recurrence. N Engl J Med 2013; 368:1695–703.
© 2014 British Association of Dermatologists
763
9 Saurat JH, Stingl G, Dubertret L et al. Efficacy and safety results from the randomized controlled comparative study of adalimumab vs. methotrexate vs. placebo in patients with psoriasis (CHAMPION). Br J Dermatol 2008; 158:558–66. 10 Ruzicka T, Lynde CW, Jemec GB et al. Efficacy and safety of oral alitretinoin (9-cis retinoic acid) in patients with severe chronic hand eczema refractory to topical corticosteroids: results of a randomized, double-blind, placebo-controlled, multicentre trial. Br J Dermatol 2008; 158:808–17. 11 Morton CA, Whitehurst C, Moseley H et al. Comparison of photodynamic therapy with cryotherapy in the treatment of Bowen’s disease. Br J Dermatol 1996; 135:766–71. 12 Cox N, Williams HC. Can you COPE with CONSORT? Br J Dermatol 2000; 142:1–7. 13 Williams HC. Cars, CONSORT 2010, and clinical practice. Trials 2010; 11:33. 14 Whitton ME, Pinart M, Batchelor J et al. Interventions for vitiligo. Cochrane Database Syst Rev 2010; 1:CD003263. 15 Rigopoulos D, Gregoriou S, Belyayeva E et al. Efficacy and safety of tacrolimus ointment 0.1% vs. betamethasone 17-valerate 0.1% in the treatment of chronic paronychia: an unblinded randomized study. Br J Dermatol 2009; 160:858–60. 16 Wiley Online Library. Virtual issues. Randomised controlled trials. Br J Dermatol. Available at: http://onlinelibrary.wiley.com/journal/ 10.1111/%28ISSN%291365-2133/homepage/VirtualIssuesPage. html (last accessed 24 February 2014). 17 Chan AW, Hr objartsson A, Haahr MT et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291:2457–65. 18 Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol 2012; 132:2727–34. 19 Ormerod AD, Williams HC. Compulsory registration of trials. Br J Dermatol 2005; 152:859–60. 20 Williams HC, Goldsmith L. The JID opens its doors to high-quality randomized controlled clinical trials. J Invest Dermatol 2006; 126:1683–4.
British Journal of Dermatology (2014) 170, pp761–763
