SPECIAL ARTICLE
The fate and reliability of endoscopy research presented at Digestive Disease Week Jonah N. Rubin, MD, MA,1 Christopher D. Atkinson, MD,1 Artur Viana, MD,1 Jordan Neviackas, MD,1 Kevin P. deHaan, MD,1 Karanpreet Shergill, MPH,2 Glenn M. Eisen, MD, MPH,3 Brian C. Jacobson, MD, MPH1 Boston, Massachusetts; Portland, Oregon, USA
The peer-review process for acceptance of abstract data presented at national conferences is limited compared with that of a medical journal considering a manuscript for publication. Data presented at national conferences are either ready for publication or are preliminary, requiring increased numbers of subjects, further investigation, or additional analyses before publication. The reliability of data presented at national conferences, such as Digestive Disease Week (DDW), is therefore unknown. Clinicians looking to cutting-edge research to inform their endoscopic practice may benefit from a clearer understanding of the fate of research data presented. We therefore sought to quantify the likelihood that endoscopic research findings presented at DDW are ultimately published in a peer-reviewed journal and how often the data and conclusions change. To accomplish these aims, we performed an analysis restricted to the “Best of DDW” review articles published subsequent to annual meetings. We chose this approach, speculating that review articles might exert additional influence over clinicians’ practice of endoscopy.
Abbreviations: CI, confidence interval; DDW, Digestive Disease Week; OR, odds ratio. DISCLOSURE: The following authors disclosed financial relationships relevant to this article: Dr Eisen is on the speakers’ bureau of Takeda, Salix, and Forrest and has received research support from Mirocam. Dr Jacobson is a consultant for Olympus and MOTUS GI. The other authors disclosed no financial relationships relevant to this article. Copyright ª 2014 by the American Society for Gastrointestinal Endoscopy 0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.05.313 Received March 29, 2014. Accepted May 18, 2014. Current affiliations: Boston University Medical Center, (1), Boston University School of Public Health (2), Boston, Massachusetts, The Oregon Clinic, Portland, Oregon (3), USA. Presented at Digestive Disease Week 2014, Chicago, Illinois (Gastrointest Endosc 2014;79:AB236). Reprint requests: Brian C. Jacobson, MD, MPH, Boston University Medical Center, 85 East Concord Street, Room 7721, Boston, MA 02118.
504 GASTROINTESTINAL ENDOSCOPY Volume 80, No. 3 : 2014
METHODS This study was approved by the Boston Medical Center Institutional Review Board. We included all abstracts cited in the “Best of DDW” review articles published in Endoscopy between 2004 and 2007. Although the journal continued to publish such reviews after 2007, with Gastrointestinal Endoscopy publishing such articles in 2012 and 2013, we wanted to restrict our analysis to those abstracts that would then have at least 6 years to appear as published articles. We developed a search strategy to determine whether abstracts cited in the DDW review articles were subsequently published in peer-reviewed journals. The online database PubMed was searched for the abstract’s title, first author (last name, first initial) and senior author (last name, first initial) from May 2004 through November 2013. The authors and methodology of the originally cited abstract and any subsequently identified published manuscript were compared to verify that the manuscript represented the final form of the DDW abstract. We did not require an exact match on title and/or both first and senior authors. Instead, we reviewed all published papers associated with either author if the title seemed somewhat related to the abstract’s title. We included for analysis only full manuscript articles, excluding letters to the editor. To validate our search methodology, we contacted the first 37 authors whose abstracts were considered not to have been published in full manuscript form. This was a convenience sample based on the number of papers identified as not having been published by our algorithm after the first month of the project. We wanted to clarify fairly early in the process whether our algorithm needed adjustment. Authors were asked whether they had ever published the data presented in the original abstract and whether they could provide a reference. Only 5 of 37 (14%) provided evidence of a publication consistent with data from the original abstract. All 5 manuscripts had been identified during the initial search, but had not been considered the same as the original DDW abstract because of a different research design. Nonetheless, we subsequently altered our search strategy to permit inclusion of published manuscripts by the same authors as DDW abstracts when there were minor alterations in research design, such as a change in the months of a trial’s enrollment period. www.giejournal.org
Rubin et al
The fate and reliability of endoscopy research presented at DDW
TABLE 1. Summary of endoscopic topics of abstracts cited in the “Best of DDW” review articles from 2004 to 2007 Topic
Oral, no. (%)
Poster, no. (%)
International, no. (%)
United States, no. (%)
Small-bowel endoscopy and capsule endoscopy
41/164 (25)
123/164 (75)
110/164 (67)
62/164 (33)
Colon tumors and colonoscopy
47/114 (41)
67/114 (59)
80/114 (70)
34/114 (30)
Endoscopy diagnosis and treatment of upper GI tumors
21/154 (14)
133/154 (86)
126/154 (82)
28/154 (18)
ERCP
35/93 (28)
58/93 (62)
43/93 (46)
50/93 (54)
EUS
28/90 (31)
62/90 (69)
34/90 (38)
56/90 (62)
GI bleeding
27/104 (26)
77/104 (74)
73/104 (70)
31/104 (30)
GERD and Barrett’s esophagus
56/128 (44)
72/128 (56)
64/128 (50)
64/128 (50)
Total
255/847 (30)
592/847 (70)
530/847 (63)
317/847 (37)
DDW, Digestive Disease Week.
We recorded how the original abstract was presented at DDW (oral vs poster), the country of origin of the cited study, and whether the data and/or conclusion changed between presentation at DDW and final manuscript publication. Changes in data included a change in the sample size and/or a change in the study’s numerical findings. Changes in conclusion included a change in study outcome (eg, from a positive outcome to negative outcome), a change in the strength of a reported association, and a change in recommendation from the authors.
Between 2004 and 2007, 29 DDW review articles were published in Endoscopy referencing 847 abstracts. Only 7
of those review articles described a systematic methodology used by the review’s authors to select the abstracts cited. Among the abstracts in the “Best of DDW” reviews, 146 (17%) appeared initially in Gastroenterology DDW abstract supplements and 701 (83%) appeared initially in Gastrointestinal Endoscopy DDW abstract supplements. Among the 847 abstracts, 592 (70%) were presented as posters and 255 (30%) were presented orally. United States–based authors accounted for 317 (37%) of the cited abstracts and non-U.S.–based authors accounted for 530 (63%) of the cited abstracts. Table 1 presents the abstract data in summary form and by endoscopic topic. The rate of conversion of cited abstracts to published manuscripts was 408/847 (48%). The topic most likely to be published was EUS. The abstracts’ data changed in 265 of 408 of final publications (65%), whereas the conclusions changed in 91 of 408 of final published manuscripts (22%) (Table 2). The topic most likely to have the data change was colon tumors and colonoscopy and conclusion change was endoscopy diagnosis and treatment of upper GI tumors. Rates of full manuscript publication were similar across each year analyzed (Table 2). The median number of years between abstract presentation at DDW and publication of a full manuscript was 1 (interquartile range 1-2) (Fig. 1). Of articles that were eventually published, 90% were published within 4 years of DDW. Increasing number of years until publication was signficantly associated with changes in data and conclusion. For each additional year until publication, the odds ratio (OR) for data change was 1.79 (95% confidence interval [CI], 1.462.18), and for conclusion change, it was 1.33 (95% CI, 1.121.57). Oral presentations were not more reliable than poster presentations in relation to changes in data (OR 1.08; 95% CI, 0.71-1.65) or conclusion (OR 0.78; 95% CI, 0.47-1.35). Compared with abstracts presented as posters, abstracts presented orally were twice as likely to be published as a
www.giejournal.org
Volume 80, No. 3 : 2014 GASTROINTESTINAL ENDOSCOPY 505
Statistical methods We calculated the percentage of abstracts cited in DDW review articles that were eventually published as full manuscripts and the median number of years between abstract publication and manuscript publication. We also calculated the percentage of abstracts with data and conclusions that changed before publication of the full manuscript and publication rates by topic. We used logistic regression models to measure the strength of association between eventual publication of a full manuscript and (1) the type of presentation at DDW, (2) the country of origin of the cited abstract, and (3) whether the original abstract was published in the Gastroenterology or Gastrointestinal Endoscopy DDW abstract supplement. We also measured the association between the number of years before an abstract was published as a full manuscript and whether the data or conclusion(s) changed. P values were calculated by using the Fisher exact test. Data were analyzed by using SAS version 9.1 (SAS Institute, Cary, NC).
RESULTS
The fate and reliability of endoscopy research presented at DDW
Rubin et al
TABLE 2. The yearly rate of publication, changes in data, and changes in conclusion Year
No. of abstracts
Published as full manuscripts, no. (%)
Data changed, no. (%)
Conclusion changed, no. (%)
2004
340
147/340 (43)
93/147 (63)
33/147 (22)
2005
160
81/160 (51)
57/81 (70)
15/81 (19)
2006
147
79/147 (54)
52/79 (66)
19/79 (24)
2007
200
101/200 (51)
63/101 (62)
24/101 (24)
Total
847
408/847 (48)
265/408 (65)
91/408 (22)
Among a select group of endoscopic research abstracts considered for the “Best of DDW” review articles between 2004 and 2007, less than half were subsequently published as full manuscripts in peer-reviewed journals. Among those abstracts published, data changed in the final publication in nearly two-thirds of cases, and conclusions drawn from the data changed in 22% of cases. We also found that although studies with oral presentations were more likely to yield published manuscripts, their data changed with the same frequency as studies presented in poster format.
The publication rate that we observed is consistent with data from other medical and surgical specialties.1 Investigators in various fields including anesthesiology, cardiology, cardiothoracic surgery, perinatal medicine, emergency medicine, orthopedics, toxicology, urology, radiology, and occupational health have found that anywhere from 5% to 81% of abstracts presented at national meetings were eventually published as full manuscripts.1-27 More than 80% of these studies report a publication rate of less than 50%.1-27 Similarly, others have reported a greater likelihood of publication for data presented orally (ie, “podium presentations”) compared with a poster presentation.1 Our finding that endoscopic research publications appear at a median of 1 year after DDW and that 90% of publications appear by 4 years after DDW is also consistent with the findings of others relating to educational conferences.28 Increasing number of years between an abstract’s presentation at DDW and eventual publication of the full manuscript was associated with a greater likelihood that the study’s data and/or conclusion would change. This finding may indicate a greater number of subjects enrolled before publication of a study’s findings, as was observed among abstracts presented at orthopedic conferences in the United Kingdom.20 A random sampling of abstracts presented at occupational health conferences demonstrated that data changed in 25% of cases when the abstract was subsequently published in full manuscript form.8 Of note, in that study, the conclusions remained unchanged in all but 1 manuscript. Our study has several strengths. First, we used a systematic search strategy that we validated on a subsample of cases. Second, our findings were very comparable to those of other investigators, suggesting generalizability. Third, by including several years’ worth of abstracts we were able to assess for consistency in findings over time. Nonetheless, we acknowledge several limitations. First, the scope of our study was narrow, focusing only on endoscopic topics. However, we thought that this would be of greatest interest to the readers of Gastrointestinal Endoscopy. Furthermore, as Gastrointestinal Endoscopy began publishing “Best of DDW” reviews in 2012, it seemed to
506 GASTROINTESTINAL ENDOSCOPY Volume 80, No. 3 : 2014
www.giejournal.org
Figure 1. The number of years elapsed between presentation of an abstract at Digestive Disease Week and the eventual publication of a full, peer-reviewed manuscript.
full manuscript (OR 2.08; 95% CI, 1.54-2.80). Abstracts from the United States were just as likely to be published in full manuscript form as abstracts from non-U.S. sites (OR 0.90; 95% CI, 0.68-1.19), suggesting a lack of publication bias related to country of origin. Abstracts published in the Gastroenterology DDW supplement were more likely to be subsequently published as a full manuscript compared with abstracts published in the Gastrointestinal Endoscopy DDW supplement with an OR for publication of 1.63 (95% CI, 1.14-2.34).
DISCUSSION
Rubin et al
be a relevant undertaking to better inform the Journal’s editors and readers as to the reliability of such reviews. Second, we limited our search to abstracts included in DDW review articles and not the entire set of abstracts presented at DDW. This limitation enabled us to include several years’ worth of data, as even this limited set required a search of more than 800 articles. Furthermore, because the abstracts included were considered by experts to represent the best or most interesting research from DDW, the publication rate that we observed is likely an overestimation of the fate of all endoscopy-related DDW abstracts. In summary, although DDW provides an unparalleled opportunity for investigators to share research findings, attendees should refrain from altering their clinical management based on the abstracts presented. Furthermore, although “Best of DDW” review articles may better disseminate some of the more interesting findings presented at this prestigious meeting, the publication of such articles should not be interpreted as an endorsement of the results or conclusions of the cited abstracts.
REFERENCES 1. Scherer R, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007 Apr 18; (2): MR000005. 2. Tyagi A, Chugh V, Kumar S, et al. Presentation of research in anesthesia: culmination into publication? J Anaesthesiol Clin Pharmacol 2013;29:216-20. 3. Fosbol EL, Fosbol PL, Harrington RA, et al. Conversion of cardiovascular conference abstracts to publications. Circulation 2012;126:2819-25. 4. Drury NE, Maniakis-Grivas G, Rogers VJ, et al. The fate of abstracts presented at annual meetings of the Society for Cardiothoracic Surgery in Great Britain and Ireland from 1993 to 2007. Eur J Cardiothorac Surg 2012;42:885-9. 5. Macdonald AL, Parsons C, Davenport M. Outcome of abstracts presented at the British Association of Paediatric Surgeons congresses (1999-2008). J Pediatr Surg 2012;47:386-90. 6. Yoon PD, Chalasani V, Woo HH. Conversion rates of abstracts presented at the Urological Society of Australia and New Zealand (USANZ) Annual Scientific Meeting into full-text journal articles. BJU Int 2012;110:485-9. 7. Chan JW, Graham CA. Full text publication rates of studies presented at an international emergency medicine scientific meeting. Emerg Med J 2011;28:802-3. 8. Rollin L, Darmoni S, Caillard J-F, et al. Fate of abstracts presented at an International Commission on Occupational Health (ICOH) congress – followed by publication in peer-reviewed journals? Scand J Work Environ Health 2009;35:461-5. 9. Oliveira LR, Figueiredo AA, Choi M, et al. The publication rate of abstracts presented at the 2003 urological Brazilian meeting. Clinics (Sao Paulo) 2009;64:345-9.
www.giejournal.org
The fate and reliability of endoscopy research presented at DDW 10. Kho ME, Brouwers MC. Conference abstracts of a new oncology drug do not always lead to full publication: proceed with caution. J Clin Epidemiol 2009;62:752-8. 11. Ha TH, Yoon DY, Goo DH, et al. Publication rates for abstracts presented by Korean investigators at major radiology meetings. Korean J Radiol 2008;9:303-11. 12. Kleweno CP, Bryant WK, Jacir AM, et al. Discrepancies and rates of publication in orthopaedic sports medicine abstracts. Am J Sports Med 2008;36:1875-9. 13. Autorino R, Quarto G, Di Lorenzo G, et al. What happens to the abstracts presented at the Societe Internationale d’Urologie meeting? Urology 2008;71:367-71. 14. Smollin CG, Nelson LS. Publication of abstracts presented at 2001 NACCT. J Med Toxicol 2006;2:97-100. 15. Macmillan CD, Moore AK, Cook RJ, et al. Abstract-to-publication ratio for papers presented at scientific meetings: a quality marker for UK emergency medicine research. Emerg Med J 2007;24:425-6. 16. Autorino R, Quarto G, Sio MD, et al. Fate of abstracts presented at the World Congress of Endourology: are they followed by publication in peer-reviewed journals? J Endourol 2006;20:996-1001. 17. Hoag CC, Elterman DS, Macneily AE. Abstracts presented at the American Urological Association Annual Meeting: determinants of subsequent peer reviewed publication. J Urol 2006;176:2624-9; discussion 9. 18. Kwong Y, Kwong FN, Patel J. Publication rate of trauma abstracts presented at an international orthopaedic conference. Injury 2007;38: 745-9. 19. Secil M, Ucar G, Dicle O. Scientific papers presented at the 2000-2001 European Society of Gastrointestinal and Abdominal Radiology (ESGAR) meetings: publication rates during the period 2000-2004. Eur Radiol 2007;17:2183-8. 20. Guryel E, Durrant AW, Alakeson R, et al. From presentation to publication: the natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J 2006;82:70-2. 21. Seçil M, Uçar G, S¸entürk C, et al. Publication rates of scientific presentations in Turkish national radiology congresses. Diagn Interv Radiol 2005;11:69-73. 22. Ng L, Hersey K, Fleshner N. Publication rate of abstracts presented at the annual meeting of the American Urological Association. BJU Int 2004;94:79-81. 23. Arrivé L, Boelle PY, Dono P, et al. Subsequent publication of orally presented original studies within 5 years after 1995 RSNA Scientific Assembly. Radiology 2004;232:101-6. 24. Bydder SA, Joseph DJ, Spry NA. Publication rates of abstracts presented at annual scientific meetings: how does the Royal Australian and New Zealand College of Radiologists compare? Australas Radiol 2004;48:25-8. 25. Sprague S, Bhandari M, Devereaux P, et al. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am 2003 Jan;85-A:158-63. 26. Davies MW, Dunster KR, East CE, et al. Fate of abstracts published in the proceedings of the first annual Perinatal Society of Australia and New Zealand Congress in 1997. J Paediatr Child Health 2002;38: 501-6. 27. Walby A, Kelly AM, Georgakas C. Abstract to publication ratio for papers presented at scientific meetings: how does emergency medicine compare? Emerg Med (Fremantle) 2001;13:460-4. 28. Walsh C, Fung M, Ginsburg S. Publication of results of abstracts presented at medical education conferences. JAMA 2013;310:2307-9.
Volume 80, No. 3 : 2014 GASTROINTESTINAL ENDOSCOPY 507
The fate and reliability of endoscopy research presented at Digestive Disease Week Jonah N. Rubin, MD, MA,1 Christopher D. Atkinson, MD,1 Artur Viana, MD,1 Jordan Neviackas, MD,1 Kevin P. deHaan, MD,1 Karanpreet Shergill, MPH,2 Glenn M. Eisen, MD, MPH,3 Brian C. Jacobson, MD, MPH1 Boston, Massachusetts; Portland, Oregon, USA
The peer-review process for acceptance of abstract data presented at national conferences is limited compared with that of a medical journal considering a manuscript for publication. Data presented at national conferences are either ready for publication or are preliminary, requiring increased numbers of subjects, further investigation, or additional analyses before publication. The reliability of data presented at national conferences, such as Digestive Disease Week (DDW), is therefore unknown. Clinicians looking to cutting-edge research to inform their endoscopic practice may benefit from a clearer understanding of the fate of research data presented. We therefore sought to quantify the likelihood that endoscopic research findings presented at DDW are ultimately published in a peer-reviewed journal and how often the data and conclusions change. To accomplish these aims, we performed an analysis restricted to the “Best of DDW” review articles published subsequent to annual meetings. We chose this approach, speculating that review articles might exert additional influence over clinicians’ practice of endoscopy.
Abbreviations: CI, confidence interval; DDW, Digestive Disease Week; OR, odds ratio. DISCLOSURE: The following authors disclosed financial relationships relevant to this article: Dr Eisen is on the speakers’ bureau of Takeda, Salix, and Forrest and has received research support from Mirocam. Dr Jacobson is a consultant for Olympus and MOTUS GI. The other authors disclosed no financial relationships relevant to this article. Copyright ª 2014 by the American Society for Gastrointestinal Endoscopy 0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.05.313 Received March 29, 2014. Accepted May 18, 2014. Current affiliations: Boston University Medical Center, (1), Boston University School of Public Health (2), Boston, Massachusetts, The Oregon Clinic, Portland, Oregon (3), USA. Presented at Digestive Disease Week 2014, Chicago, Illinois (Gastrointest Endosc 2014;79:AB236). Reprint requests: Brian C. Jacobson, MD, MPH, Boston University Medical Center, 85 East Concord Street, Room 7721, Boston, MA 02118.
504 GASTROINTESTINAL ENDOSCOPY Volume 80, No. 3 : 2014
METHODS This study was approved by the Boston Medical Center Institutional Review Board. We included all abstracts cited in the “Best of DDW” review articles published in Endoscopy between 2004 and 2007. Although the journal continued to publish such reviews after 2007, with Gastrointestinal Endoscopy publishing such articles in 2012 and 2013, we wanted to restrict our analysis to those abstracts that would then have at least 6 years to appear as published articles. We developed a search strategy to determine whether abstracts cited in the DDW review articles were subsequently published in peer-reviewed journals. The online database PubMed was searched for the abstract’s title, first author (last name, first initial) and senior author (last name, first initial) from May 2004 through November 2013. The authors and methodology of the originally cited abstract and any subsequently identified published manuscript were compared to verify that the manuscript represented the final form of the DDW abstract. We did not require an exact match on title and/or both first and senior authors. Instead, we reviewed all published papers associated with either author if the title seemed somewhat related to the abstract’s title. We included for analysis only full manuscript articles, excluding letters to the editor. To validate our search methodology, we contacted the first 37 authors whose abstracts were considered not to have been published in full manuscript form. This was a convenience sample based on the number of papers identified as not having been published by our algorithm after the first month of the project. We wanted to clarify fairly early in the process whether our algorithm needed adjustment. Authors were asked whether they had ever published the data presented in the original abstract and whether they could provide a reference. Only 5 of 37 (14%) provided evidence of a publication consistent with data from the original abstract. All 5 manuscripts had been identified during the initial search, but had not been considered the same as the original DDW abstract because of a different research design. Nonetheless, we subsequently altered our search strategy to permit inclusion of published manuscripts by the same authors as DDW abstracts when there were minor alterations in research design, such as a change in the months of a trial’s enrollment period. www.giejournal.org
Rubin et al
The fate and reliability of endoscopy research presented at DDW
TABLE 1. Summary of endoscopic topics of abstracts cited in the “Best of DDW” review articles from 2004 to 2007 Topic
Oral, no. (%)
Poster, no. (%)
International, no. (%)
United States, no. (%)
Small-bowel endoscopy and capsule endoscopy
41/164 (25)
123/164 (75)
110/164 (67)
62/164 (33)
Colon tumors and colonoscopy
47/114 (41)
67/114 (59)
80/114 (70)
34/114 (30)
Endoscopy diagnosis and treatment of upper GI tumors
21/154 (14)
133/154 (86)
126/154 (82)
28/154 (18)
ERCP
35/93 (28)
58/93 (62)
43/93 (46)
50/93 (54)
EUS
28/90 (31)
62/90 (69)
34/90 (38)
56/90 (62)
GI bleeding
27/104 (26)
77/104 (74)
73/104 (70)
31/104 (30)
GERD and Barrett’s esophagus
56/128 (44)
72/128 (56)
64/128 (50)
64/128 (50)
Total
255/847 (30)
592/847 (70)
530/847 (63)
317/847 (37)
DDW, Digestive Disease Week.
We recorded how the original abstract was presented at DDW (oral vs poster), the country of origin of the cited study, and whether the data and/or conclusion changed between presentation at DDW and final manuscript publication. Changes in data included a change in the sample size and/or a change in the study’s numerical findings. Changes in conclusion included a change in study outcome (eg, from a positive outcome to negative outcome), a change in the strength of a reported association, and a change in recommendation from the authors.
Between 2004 and 2007, 29 DDW review articles were published in Endoscopy referencing 847 abstracts. Only 7
of those review articles described a systematic methodology used by the review’s authors to select the abstracts cited. Among the abstracts in the “Best of DDW” reviews, 146 (17%) appeared initially in Gastroenterology DDW abstract supplements and 701 (83%) appeared initially in Gastrointestinal Endoscopy DDW abstract supplements. Among the 847 abstracts, 592 (70%) were presented as posters and 255 (30%) were presented orally. United States–based authors accounted for 317 (37%) of the cited abstracts and non-U.S.–based authors accounted for 530 (63%) of the cited abstracts. Table 1 presents the abstract data in summary form and by endoscopic topic. The rate of conversion of cited abstracts to published manuscripts was 408/847 (48%). The topic most likely to be published was EUS. The abstracts’ data changed in 265 of 408 of final publications (65%), whereas the conclusions changed in 91 of 408 of final published manuscripts (22%) (Table 2). The topic most likely to have the data change was colon tumors and colonoscopy and conclusion change was endoscopy diagnosis and treatment of upper GI tumors. Rates of full manuscript publication were similar across each year analyzed (Table 2). The median number of years between abstract presentation at DDW and publication of a full manuscript was 1 (interquartile range 1-2) (Fig. 1). Of articles that were eventually published, 90% were published within 4 years of DDW. Increasing number of years until publication was signficantly associated with changes in data and conclusion. For each additional year until publication, the odds ratio (OR) for data change was 1.79 (95% confidence interval [CI], 1.462.18), and for conclusion change, it was 1.33 (95% CI, 1.121.57). Oral presentations were not more reliable than poster presentations in relation to changes in data (OR 1.08; 95% CI, 0.71-1.65) or conclusion (OR 0.78; 95% CI, 0.47-1.35). Compared with abstracts presented as posters, abstracts presented orally were twice as likely to be published as a
www.giejournal.org
Volume 80, No. 3 : 2014 GASTROINTESTINAL ENDOSCOPY 505
Statistical methods We calculated the percentage of abstracts cited in DDW review articles that were eventually published as full manuscripts and the median number of years between abstract publication and manuscript publication. We also calculated the percentage of abstracts with data and conclusions that changed before publication of the full manuscript and publication rates by topic. We used logistic regression models to measure the strength of association between eventual publication of a full manuscript and (1) the type of presentation at DDW, (2) the country of origin of the cited abstract, and (3) whether the original abstract was published in the Gastroenterology or Gastrointestinal Endoscopy DDW abstract supplement. We also measured the association between the number of years before an abstract was published as a full manuscript and whether the data or conclusion(s) changed. P values were calculated by using the Fisher exact test. Data were analyzed by using SAS version 9.1 (SAS Institute, Cary, NC).
RESULTS
The fate and reliability of endoscopy research presented at DDW
Rubin et al
TABLE 2. The yearly rate of publication, changes in data, and changes in conclusion Year
No. of abstracts
Published as full manuscripts, no. (%)
Data changed, no. (%)
Conclusion changed, no. (%)
2004
340
147/340 (43)
93/147 (63)
33/147 (22)
2005
160
81/160 (51)
57/81 (70)
15/81 (19)
2006
147
79/147 (54)
52/79 (66)
19/79 (24)
2007
200
101/200 (51)
63/101 (62)
24/101 (24)
Total
847
408/847 (48)
265/408 (65)
91/408 (22)
Among a select group of endoscopic research abstracts considered for the “Best of DDW” review articles between 2004 and 2007, less than half were subsequently published as full manuscripts in peer-reviewed journals. Among those abstracts published, data changed in the final publication in nearly two-thirds of cases, and conclusions drawn from the data changed in 22% of cases. We also found that although studies with oral presentations were more likely to yield published manuscripts, their data changed with the same frequency as studies presented in poster format.
The publication rate that we observed is consistent with data from other medical and surgical specialties.1 Investigators in various fields including anesthesiology, cardiology, cardiothoracic surgery, perinatal medicine, emergency medicine, orthopedics, toxicology, urology, radiology, and occupational health have found that anywhere from 5% to 81% of abstracts presented at national meetings were eventually published as full manuscripts.1-27 More than 80% of these studies report a publication rate of less than 50%.1-27 Similarly, others have reported a greater likelihood of publication for data presented orally (ie, “podium presentations”) compared with a poster presentation.1 Our finding that endoscopic research publications appear at a median of 1 year after DDW and that 90% of publications appear by 4 years after DDW is also consistent with the findings of others relating to educational conferences.28 Increasing number of years between an abstract’s presentation at DDW and eventual publication of the full manuscript was associated with a greater likelihood that the study’s data and/or conclusion would change. This finding may indicate a greater number of subjects enrolled before publication of a study’s findings, as was observed among abstracts presented at orthopedic conferences in the United Kingdom.20 A random sampling of abstracts presented at occupational health conferences demonstrated that data changed in 25% of cases when the abstract was subsequently published in full manuscript form.8 Of note, in that study, the conclusions remained unchanged in all but 1 manuscript. Our study has several strengths. First, we used a systematic search strategy that we validated on a subsample of cases. Second, our findings were very comparable to those of other investigators, suggesting generalizability. Third, by including several years’ worth of abstracts we were able to assess for consistency in findings over time. Nonetheless, we acknowledge several limitations. First, the scope of our study was narrow, focusing only on endoscopic topics. However, we thought that this would be of greatest interest to the readers of Gastrointestinal Endoscopy. Furthermore, as Gastrointestinal Endoscopy began publishing “Best of DDW” reviews in 2012, it seemed to
506 GASTROINTESTINAL ENDOSCOPY Volume 80, No. 3 : 2014
www.giejournal.org
Figure 1. The number of years elapsed between presentation of an abstract at Digestive Disease Week and the eventual publication of a full, peer-reviewed manuscript.
full manuscript (OR 2.08; 95% CI, 1.54-2.80). Abstracts from the United States were just as likely to be published in full manuscript form as abstracts from non-U.S. sites (OR 0.90; 95% CI, 0.68-1.19), suggesting a lack of publication bias related to country of origin. Abstracts published in the Gastroenterology DDW supplement were more likely to be subsequently published as a full manuscript compared with abstracts published in the Gastrointestinal Endoscopy DDW supplement with an OR for publication of 1.63 (95% CI, 1.14-2.34).
DISCUSSION
Rubin et al
be a relevant undertaking to better inform the Journal’s editors and readers as to the reliability of such reviews. Second, we limited our search to abstracts included in DDW review articles and not the entire set of abstracts presented at DDW. This limitation enabled us to include several years’ worth of data, as even this limited set required a search of more than 800 articles. Furthermore, because the abstracts included were considered by experts to represent the best or most interesting research from DDW, the publication rate that we observed is likely an overestimation of the fate of all endoscopy-related DDW abstracts. In summary, although DDW provides an unparalleled opportunity for investigators to share research findings, attendees should refrain from altering their clinical management based on the abstracts presented. Furthermore, although “Best of DDW” review articles may better disseminate some of the more interesting findings presented at this prestigious meeting, the publication of such articles should not be interpreted as an endorsement of the results or conclusions of the cited abstracts.
REFERENCES 1. Scherer R, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007 Apr 18; (2): MR000005. 2. Tyagi A, Chugh V, Kumar S, et al. Presentation of research in anesthesia: culmination into publication? J Anaesthesiol Clin Pharmacol 2013;29:216-20. 3. Fosbol EL, Fosbol PL, Harrington RA, et al. Conversion of cardiovascular conference abstracts to publications. Circulation 2012;126:2819-25. 4. Drury NE, Maniakis-Grivas G, Rogers VJ, et al. The fate of abstracts presented at annual meetings of the Society for Cardiothoracic Surgery in Great Britain and Ireland from 1993 to 2007. Eur J Cardiothorac Surg 2012;42:885-9. 5. Macdonald AL, Parsons C, Davenport M. Outcome of abstracts presented at the British Association of Paediatric Surgeons congresses (1999-2008). J Pediatr Surg 2012;47:386-90. 6. Yoon PD, Chalasani V, Woo HH. Conversion rates of abstracts presented at the Urological Society of Australia and New Zealand (USANZ) Annual Scientific Meeting into full-text journal articles. BJU Int 2012;110:485-9. 7. Chan JW, Graham CA. Full text publication rates of studies presented at an international emergency medicine scientific meeting. Emerg Med J 2011;28:802-3. 8. Rollin L, Darmoni S, Caillard J-F, et al. Fate of abstracts presented at an International Commission on Occupational Health (ICOH) congress – followed by publication in peer-reviewed journals? Scand J Work Environ Health 2009;35:461-5. 9. Oliveira LR, Figueiredo AA, Choi M, et al. The publication rate of abstracts presented at the 2003 urological Brazilian meeting. Clinics (Sao Paulo) 2009;64:345-9.
www.giejournal.org
The fate and reliability of endoscopy research presented at DDW 10. Kho ME, Brouwers MC. Conference abstracts of a new oncology drug do not always lead to full publication: proceed with caution. J Clin Epidemiol 2009;62:752-8. 11. Ha TH, Yoon DY, Goo DH, et al. Publication rates for abstracts presented by Korean investigators at major radiology meetings. Korean J Radiol 2008;9:303-11. 12. Kleweno CP, Bryant WK, Jacir AM, et al. Discrepancies and rates of publication in orthopaedic sports medicine abstracts. Am J Sports Med 2008;36:1875-9. 13. Autorino R, Quarto G, Di Lorenzo G, et al. What happens to the abstracts presented at the Societe Internationale d’Urologie meeting? Urology 2008;71:367-71. 14. Smollin CG, Nelson LS. Publication of abstracts presented at 2001 NACCT. J Med Toxicol 2006;2:97-100. 15. Macmillan CD, Moore AK, Cook RJ, et al. Abstract-to-publication ratio for papers presented at scientific meetings: a quality marker for UK emergency medicine research. Emerg Med J 2007;24:425-6. 16. Autorino R, Quarto G, Sio MD, et al. Fate of abstracts presented at the World Congress of Endourology: are they followed by publication in peer-reviewed journals? J Endourol 2006;20:996-1001. 17. Hoag CC, Elterman DS, Macneily AE. Abstracts presented at the American Urological Association Annual Meeting: determinants of subsequent peer reviewed publication. J Urol 2006;176:2624-9; discussion 9. 18. Kwong Y, Kwong FN, Patel J. Publication rate of trauma abstracts presented at an international orthopaedic conference. Injury 2007;38: 745-9. 19. Secil M, Ucar G, Dicle O. Scientific papers presented at the 2000-2001 European Society of Gastrointestinal and Abdominal Radiology (ESGAR) meetings: publication rates during the period 2000-2004. Eur Radiol 2007;17:2183-8. 20. Guryel E, Durrant AW, Alakeson R, et al. From presentation to publication: the natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J 2006;82:70-2. 21. Seçil M, Uçar G, S¸entürk C, et al. Publication rates of scientific presentations in Turkish national radiology congresses. Diagn Interv Radiol 2005;11:69-73. 22. Ng L, Hersey K, Fleshner N. Publication rate of abstracts presented at the annual meeting of the American Urological Association. BJU Int 2004;94:79-81. 23. Arrivé L, Boelle PY, Dono P, et al. Subsequent publication of orally presented original studies within 5 years after 1995 RSNA Scientific Assembly. Radiology 2004;232:101-6. 24. Bydder SA, Joseph DJ, Spry NA. Publication rates of abstracts presented at annual scientific meetings: how does the Royal Australian and New Zealand College of Radiologists compare? Australas Radiol 2004;48:25-8. 25. Sprague S, Bhandari M, Devereaux P, et al. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am 2003 Jan;85-A:158-63. 26. Davies MW, Dunster KR, East CE, et al. Fate of abstracts published in the proceedings of the first annual Perinatal Society of Australia and New Zealand Congress in 1997. J Paediatr Child Health 2002;38: 501-6. 27. Walby A, Kelly AM, Georgakas C. Abstract to publication ratio for papers presented at scientific meetings: how does emergency medicine compare? Emerg Med (Fremantle) 2001;13:460-4. 28. Walsh C, Fung M, Ginsburg S. Publication of results of abstracts presented at medical education conferences. JAMA 2013;310:2307-9.
Volume 80, No. 3 : 2014 GASTROINTESTINAL ENDOSCOPY 507
