Downloaded from http://qualitysafety.bmj.com/ on September 21, 2015 - Published by group.bmj.com
BMJ Quality & Safety Online First, published on 12 February 2015 as 10.1136/bmjqs-2014-003737 RESEARCH AND REPORTING METHODOLOGY
The effect of the SQUIRE (Standards of QUality Improvement Reporting Excellence) guidelines on reporting standards in the quality improvement literature: a before-and-after study Victoria Howell,1,2 Amanda Eva Schwartz,1 James Daniel O’Leary,1,2 Conor Mc Donnell1,2
▸ Additional material is published online only. To view please visit the journal online (http://dx.doi.org/10.1136/bmjqs2014-003737). 1
Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario, Canada 2 Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada Correspondence to Dr Conor Mc Donnell at Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario M5G 1X8, Canada; [email protected] Received 31 October 2014 Revised 21 January 2015 Accepted 25 January 2015
To cite: Howell V, Schwartz AE, O’Leary JD, et al. BMJ Qual Saf Published Online First: [ please include Day Month Year] doi:10.1136/bmjqs-2014003737
ABSTRACT Background The SQUIRE (Standards of QUality Improvement Reporting Excellence) guidelines were developed to improve the reporting of quality improvement (QI) projects. The effect of the guidelines on the completeness of reporting in the QI literature is unknown. Objectives Our primary objective was to determine if the completeness of reporting in the QI literature has been improved[OUP_CE13] since the introduction of the SQUIRE guidelines. Methods We performed a before-and-after evaluation of QI articles selected from four prominent journals of healthcare quality. Twentyfive articles published in each of two time periods (2006–2008 and 2010–2011) were confirmed to be QI projects using a standardised definition and were independently evaluated by two investigators as an interim evaluation of a planned larger sample. Articles were assessed using 50 statements of the SQUIRE guidelines, and the overall change in the completeness of reporting between the two groups was determined. The value of p50%) of the descriptors were reported. Text relevant to a specific SQUIRE statement that was completed in a different section than that recommended was recorded as ‘complete’ during analysis. In the event of disagreement between both data extractors, a consensus was reached or a third investigator (CM or JDO) was consulted to resolve any dispute. Both investigators evaluating completeness of reporting of study articles received training in the SQUIRE guidelines prior to the study to ensure consistency and accuracy of data extraction. The education sessions were led by a faculty member of the University of Toronto’s Centre of Quality Improvement and Patient Safety (CM) and consisted of directed reading and discussion of the SQUIRE literature. Data collectors then independently scored five QI articles for completeness of reporting of SQUIRE statements using the same criteria as the study, for which substantial interobserver agreement was observed (overall agreement for 250 SQUIRE statements in five papers was 88%, κ=0.77). Howell V, et al. BMJ Qual Saf 2015;0:1–7. doi:10.1136/bmjqs-2014-003737
Downloaded from http://qualitysafety.bmj.com/ on September 21, 2015 - Published by group.bmj.com
Research and reporting methodology As this study is the first to investigate the effect of the SQUIRE guidelines on the completeness of QI reporting in the healthcare literature, the minimum sample size needed was difficult to determine a priori. High-impact medical journals previously demonstrated completeness of reporting rates varying from 35% to 73% prior to publication of the CONSORT (Consolidated Standards for Reporting of Trials) statement, and the completeness of reporting tended to be lower again for some subspecialty publications.6 Based on the findings of other studies evaluating reporting guidelines, a relative increase in overall statement reporting of at least 50% was considered to be significant and anticipated by other studies investigating the effect of reporting guidelines. We estimated that a minimum sample size of 97 publications per group ( pre-SQUIRE and post-SQUIRE) would be required to detect a difference in the proportion of overall SQUIRE statements reported from 0.4 to 0.6, at twotailed values of α=0.05 and β=0.2. Using these data, we anticipated that a total of 194 articles would be required for inclusion in this study. We planned an interim analysis using the first 25 articles in each of the pre-SQUIRE and post-SQUIRE groups to determine the accuracy of our initial sample size estimation as it was anticipated that significant variation in the completeness of reporting among publication types and specialties might occur and our sample size estimation was based on studies investigating the effectiveness of the CONSORT statement. The primary outcome was the overall number of SQUIRE statements (n=50) addressed by authors. Data were recorded in Microsoft Excel 2010 and analysed using Prism 6 (GraphPad Software, San Diego, California, USA). Data are presented as medians or means with measures of variance appropriate to the data distribution. The differences in the number of statements completed between groups were compared using the χ2 test or Fisher’s exact test. OR and 95% CIs were determined where appropriate. The value of p
BMJ Quality & Safety Online First, published on 12 February 2015 as 10.1136/bmjqs-2014-003737 RESEARCH AND REPORTING METHODOLOGY
The effect of the SQUIRE (Standards of QUality Improvement Reporting Excellence) guidelines on reporting standards in the quality improvement literature: a before-and-after study Victoria Howell,1,2 Amanda Eva Schwartz,1 James Daniel O’Leary,1,2 Conor Mc Donnell1,2
▸ Additional material is published online only. To view please visit the journal online (http://dx.doi.org/10.1136/bmjqs2014-003737). 1
Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario, Canada 2 Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada Correspondence to Dr Conor Mc Donnell at Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario M5G 1X8, Canada; [email protected] Received 31 October 2014 Revised 21 January 2015 Accepted 25 January 2015
To cite: Howell V, Schwartz AE, O’Leary JD, et al. BMJ Qual Saf Published Online First: [ please include Day Month Year] doi:10.1136/bmjqs-2014003737
ABSTRACT Background The SQUIRE (Standards of QUality Improvement Reporting Excellence) guidelines were developed to improve the reporting of quality improvement (QI) projects. The effect of the guidelines on the completeness of reporting in the QI literature is unknown. Objectives Our primary objective was to determine if the completeness of reporting in the QI literature has been improved[OUP_CE13] since the introduction of the SQUIRE guidelines. Methods We performed a before-and-after evaluation of QI articles selected from four prominent journals of healthcare quality. Twentyfive articles published in each of two time periods (2006–2008 and 2010–2011) were confirmed to be QI projects using a standardised definition and were independently evaluated by two investigators as an interim evaluation of a planned larger sample. Articles were assessed using 50 statements of the SQUIRE guidelines, and the overall change in the completeness of reporting between the two groups was determined. The value of p50%) of the descriptors were reported. Text relevant to a specific SQUIRE statement that was completed in a different section than that recommended was recorded as ‘complete’ during analysis. In the event of disagreement between both data extractors, a consensus was reached or a third investigator (CM or JDO) was consulted to resolve any dispute. Both investigators evaluating completeness of reporting of study articles received training in the SQUIRE guidelines prior to the study to ensure consistency and accuracy of data extraction. The education sessions were led by a faculty member of the University of Toronto’s Centre of Quality Improvement and Patient Safety (CM) and consisted of directed reading and discussion of the SQUIRE literature. Data collectors then independently scored five QI articles for completeness of reporting of SQUIRE statements using the same criteria as the study, for which substantial interobserver agreement was observed (overall agreement for 250 SQUIRE statements in five papers was 88%, κ=0.77). Howell V, et al. BMJ Qual Saf 2015;0:1–7. doi:10.1136/bmjqs-2014-003737
Downloaded from http://qualitysafety.bmj.com/ on September 21, 2015 - Published by group.bmj.com
Research and reporting methodology As this study is the first to investigate the effect of the SQUIRE guidelines on the completeness of QI reporting in the healthcare literature, the minimum sample size needed was difficult to determine a priori. High-impact medical journals previously demonstrated completeness of reporting rates varying from 35% to 73% prior to publication of the CONSORT (Consolidated Standards for Reporting of Trials) statement, and the completeness of reporting tended to be lower again for some subspecialty publications.6 Based on the findings of other studies evaluating reporting guidelines, a relative increase in overall statement reporting of at least 50% was considered to be significant and anticipated by other studies investigating the effect of reporting guidelines. We estimated that a minimum sample size of 97 publications per group ( pre-SQUIRE and post-SQUIRE) would be required to detect a difference in the proportion of overall SQUIRE statements reported from 0.4 to 0.6, at twotailed values of α=0.05 and β=0.2. Using these data, we anticipated that a total of 194 articles would be required for inclusion in this study. We planned an interim analysis using the first 25 articles in each of the pre-SQUIRE and post-SQUIRE groups to determine the accuracy of our initial sample size estimation as it was anticipated that significant variation in the completeness of reporting among publication types and specialties might occur and our sample size estimation was based on studies investigating the effectiveness of the CONSORT statement. The primary outcome was the overall number of SQUIRE statements (n=50) addressed by authors. Data were recorded in Microsoft Excel 2010 and analysed using Prism 6 (GraphPad Software, San Diego, California, USA). Data are presented as medians or means with measures of variance appropriate to the data distribution. The differences in the number of statements completed between groups were compared using the χ2 test or Fisher’s exact test. OR and 95% CIs were determined where appropriate. The value of p
