FEATURE ARTICLE
The Trend of Quality of Publications in Endodontic Surgery: A 10-year Systematic Survey of the Literature Massimo Del Fabbroa, Stefano Corbellab, Igor Tsesisc, and Silvio Taschieria a
Universita degli Studi di Milano, Department of Biomedical, Surgical and Dental Sciences, Oral Health Research Centre, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy b Universita degli Studi di Milano, Department of Biomedical, Surgical and Dental Sciences, Oral Implantology Research Centre, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy c Department of Endodontology, Maurice and Gabriela Goldschleger School of Dental Medicine, Tel Aviv University, Tel Aviv, Israel
Abstract
Objectives: The aims of the present systematic literature analysis were to evaluate, over a 10-year period, the trend of the proportion of RCT, SR, MA published on endodontic surgery, and to investigate if the impact factor (IF) of the main endodontic Journals correlates with the proportion of RCT, SR, MA they publish. Methods: An electronic search of the RCT, SR and MA published on the topic ‘‘endodontic surgery’’ from 2001 to 2010 was performed on Medline and Cochrane CENTRAL database using specific search terms combined with Boolean operators. Endodontic Journals impact factor was retrieved by the Thomson Scientific database. The proportion of each study type over the total number of articles on endodontic surgery published per year was estimated. The correlation between the number of high-evidence level studies published on the main endodontic Journals and the IF of such Journals per year was estimated. Results: From a total of 900 articles published in 2001–2010 on endodontic surgery, there were 114 studies of high evidence level. A significant increase of the proportion of either RCT, SR and MA over the years was found. A modest to unclear correlation was found between the Journal IF and the number of high-evidence articles published. Conclusions: There is a positive trend over the years among researchers in performing studies of good quality in endodontic surgery. The impact factor of endodontic Journals is not consistently influenced by publication of highevidence level articles. Keywords: Endodontics, Evidence-based dentistry, Journal impact factor.
INTRODUCTION Corresponding author. Tel.: +39 02 50319950; fax: +39 02 50319960;
E-mail: [email protected]. J Evid Base Dent Pract 2014;-:1-6 1532-3382/$36.00 Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jebdp.2014.09.002
The development of new techniques, instrumentation and biomaterials used in endodontic surgery has made possible the extension of its clinical indications. However, with the increase of health care costs, there has been a paradigm shift in health care toward evidence-based research.
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Many manufacturers and corporations tend to use effective marketing rather than peer-reviewed studies to promote their technological and biological advances.1 This trend can create a confusing picture for the endodontic surgeon, who has the responsibility for recommending the most appropriate surgical approach using a conscious critical analysis based on accurate diagnostic path. When a clinician discusses treatment planning with a patient, it is necessary to provide the patient with information related to the efficacy and long-term outcome of the various treatment options. These data are needed for informed decisions.1 Ranking the available evidence into different levels and grades of recommendation was first described by Fletcher & Sackett more than 25 years ago to give an idea of the quality of the evidence on the basis of the level of bias and flaws of the various types of study design adopted in the biomedical research.2 In general the level of evidence of a study is considered as inversely related to its level of bias. The latter can be defined as any uncontrolled trait of the experimental design that may affect the outcome, therefore producing a distorted result, which may not reflect the true effect of a given treatment.3,4 Sackett et al defined evidence-based medicine, also termed evidence-based practice (EBP), as ‘‘the conscientious, explicit and judicious use of current best evidence about the care of individual patients’ integrated with clinical expertise and patient values to optimize outcomes and quality of life.’’5 In the hierarchy of study designs used in clinical research, randomized controlled trials (RCT), prospective controlled trials (CT) and meta-analyses (MA) or systematic reviews (SR) (dealing with RCTs or controlled clinical trials) are considered to provide the highest level of evidence.6–8 Conversely, uncontrolled studies like case series and case reports, as well as retrospective studies are associated with a lower level of evidence. In other words, the latter types of investigation, due to the features of the study design such as the choice of the patients, the allocation of treatments, the absence of blinding procedures and many other methodological aspects that may somehow affect the outcomes, are considered to have a higher level of bias as compared to randomized controlled studies.9 RCTs are specifically designed to minimize the experimental bias in any steps of the study procedures, so as to provide the most reliable possible outcomes.10 Since the volume of published information is steadily increasing in the field of endodontics (as well as in many other fields of medical sciences) it is extremely important to assess the level of evidence of the publications, in order to discern which information should be relied upon to formulate an evidence-based treatment plan and provide the patients with the most accurate, up-to date and trustworthy information. 2
The most accredited tool for evaluating the weight of evidence in addition to the type of study design is the journal impact factor (IF), which indicates the average frequency of citation to any indexed journal.11,12 The higher the number of citations received by a Journal, the higher the impact of the Journal in the scientific community, which is considered as related to the importance and the reliability of the information provided by the articles published. Since the latter are normally chosen based upon a rigorous selection process it is believed that the best quality journals have a manuscript review process able to select the best quality information to be published and that, consequently, they have a high probability of receiving a high number of citations. Though the use of IF as an index of quality of the journals is still a matter of debate among scientists, in the absence of an alternative reliable index, the IF is currently adopted as a marker of the value of published scientific information. Similarly, scientists and clinicians able to have their studies published on a high-IF journal are considered as valuable researchers, independent of the number of citations that their specific articles will receive. The purpose of this study was to examine, in the field of endodontic surgery (ES), the relationship between quality of research in terms of levels of evidence and IF of main journals dealing with endodontic surgery, in an attempt to understanding the pattern of citation and the pattern of publication of high-evidence level studies on this topic over the last 10 years. In particular, the main hypothesis was that journals with higher IF tend to publish more frequently studies of high level of evidence as compared to journals with lower IF in which the proportion of high evidence level studies published tends to be lower. Another hypothesis was that the proportion of RCTs and meta-analysis/systematic reviews among the total number of articles published in the field of endodontic surgery tends to increase in the last 10 years, due to the spreading of the concept of evidence-based dentistry.
MATERIALS AND METHODS An electronic search was performed on Pubmed, on the Cochrane Central Register of Controlled Trials (CENTRAL) and on the Thomson Scientific database, which includes the Journal of Citation Reports containing the IF of indexed and impacted journals. The search was limited to 10 years between 2001 and 2010. The keywords used were: apical surgery, apicectomy, apicoectomy, endodontic surgery, periapical surgery, periradicular surgery, root-end management, surgical endodontics, surgical retreatment, alone or combined among them. No restriction was placed regarding the language. Technical studies and editorials were excluded. The reference lists of meta-analysis (MA) and systematic reviews (SR) were checked. A hand-search -
2014
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
was also performed on the main journals publishing articles in the field of endodontic surgery from 2001 to 2010: Journal of Endodontics, International Endodontic Journal, Journal of Oral & Maxillofacial Surgery, International Journal of Oral & Maxillofacial Surgery, Oral Surgery Oral Medicine Oral Pathology Oral Radiology and Endodontology, British Journal of Oral & Maxillofacial Surgery. The variables searched with Pubmed were: the total number of articles, the number of controlled clinical studies (CT), the number of RCTs, the number of systematic reviews and meta-analyses. The data were searched separately per each year and then tabulated and reported in graphic form as both absolute value and percentages. For the allocation of the articles to a given year the date on which they first appeared on Pubmed was taken into consideration (i.e. articles early online since 2007 and published in a given Journal issue in 2009 were allocated to 2007). The pattern of IF over the years considered was also tabulated for all the journals that were previously handsearched. For each study type of interest (MA, SR, RCT, CT) the proportion of studies published in each of the main endodontic journals was calculated. Subsequently, MA and SR, as well as CT and RCT were grouped so as to compare the trend of the best primary studies and of the best secondary studies. A correlation was searched between journals’ IF and level of evidence of the studies, grouping together all types of high-evidence level studies for each journal of interest. All studies of interest not published in any of the above journals were scored as ‘‘other’’ and not considered for the correlation with IF. Data were summarized by means of tables and histograms.
RESULTS The Pubmed search provided a total of 900 articles dealing with endodontic surgery, of which 114 were selected based on the study design (MA, SR, RCT, CT). Nineteen further articles were added after searching on the CENTRAL database. Therefore, out of 919 articles published over 10 years on endodontic surgery, a total of 133 articles were selected as high-evidence level studies (14.5%). No further study was added after handsearching. Out of a total of 110 articles indexed on Pubmed as reviews over 10 years, there were 23 systematic reviews and 13 meta-analyses. Taken together they represented 32.7% of the total reviews and 3.9% of the total articles. Controlled trials and RCTs published over 10 years were 35 and 62 (3.8% and 6.7% of total articles), respectively. Taken together they represented 10.6% of the total articles retrieved. The above numbers provide only a general idea of the mean percentage (less than 15%) of high-quality articles Volume
-,
Number
-
Figure 1. Endodontic surgery-high evidence-level articles. The number of high evidence-level articles published in the endodontic surgery field is steadily increasing over the years.
Figure 2. Endodontic surgery-high evidence-level articles. The trend of the percentage of high evidence-level articles published in the endodontic surgery field also reflects an increase over the years.
published in the scientific literature on endodontic surgery. Trends and correlations were then evaluated. Figures 1 and 2 show, respectively, the trend for the absolute number and the percentage of high evidencelevel articles in endodontic surgery. When comparing primary studies (CT + RCT) and secondary studies (SR + MA), it appears that both types of studies tend to progressively increase over time, suggesting a positive trend of researchers toward improving the level of publications’ quality. The increase in the proportion of such articles tends to be similar, being the trend lines parallel (Figure 2), with greater values for clinical studies as compared to reviews. For comparison, the same trends have been evaluated for the topic ‘‘endodontics.’’ Figures 3 and 4 show, respectively, the trend for the absolute number and the 3
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Figure 3. Endodontics-high evidence-level articles. The percentage of high evidence-level articles published in the endodontics field seems to increase much less than for endodontic surgery over the years. Instead, the percentage of systematic reviews is about half than for endodontic surgery (see Figure 2) and seems unchanged over time. It has to be considered that this graph includes articles considered in Figure 2.
Figure 4. Endodontics-high evidence-level articles. The number of high evidence-level articles published in the endodontics field is steadily increasing over the years. This well compares to the Figure 1 for clinical articles and, on a lesser degree, for the systematic reviews. This graph does not include endodontic surgery articles considered in Figure 1.
percentage of high evidence-level articles in endodontics, without considering the data for endodontic surgery that have been removed. A trend of increasing the amount (Figure 1) and the proportion (Figure 2) of clinical high-evidence level publications was found, similar to endodontic surgery. Overall proportion of high evidence-level clinical studies in endodontics was higher than for endodontic surgery (16.5% vs 10.6%). Conversely, the increase for SR and MA seems less pronounced than for corresponding data of endodontic 4
Figure 5. Trend of impact factor. Among articles publishing endodontic surgery articles, JOE and IEJ show the most consistent increase in impact factor over 10 years (only the regression line is shown).
Figure 6. 10-y Journal IF vs high evidence-level articles. This graph shows that the higher the (mean) journal impact factor, the higher the number of high evidence-level articles published over 10 years. There is a good correlation (r2 ¼ 0.83) between IF & quality of article.
surgery, and also the overall proportion of SR and MA is about half than for endodontic surgery (2.1% vs 3.9%). In Figure 5 is reported the trend for the journal impact factor, which shows a consistent increase for the two main endodontic journals (Journal of Endodontics and International Endodontic Journal) as compared to the other journals considered for which a very small increase was found. The table attached to Figure 5 reports the actual values of IF from 2001 to 2010 for the six journals mainly involved in publishing articles on endodontic surgery. Figure 6 shows a very interesting correlation between the cumulative number of high evidence-level articles published on a journal and the mean journal impact factor over 10 years. A strong correlation was found (r2 ¼ 0.83), suggesting that the higher the IF, the greater -
2014
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Figure 7. Impact factor & number of articles. This graph considers data per each year of the three journals with the higher number of high evidence-level articles. Only for JOE there is a modest correlation between IF and no. of good quality articles per given year.
the amount of good quality articles published in the journal. This finding was further explored in greater detail. The three journals with the higher number of high evidence level articles were selected for a further analysis in which the data of each single year were considered. As shown in Figure 7 only for the Journal of Endodontics a modest correlation was found, while for other journals publishing less high evidence level articles such a correlation was not clear. This would suggest that the more high evidence level articles are published, the higher the number of citations received by the journal.
DISCUSSION The present study aimed at exploring the level of evidence of published articles in the field of endodontics and endodontic surgery. The number of RCTs on a particular journal was considered the main parameter to evaluate the level of evidence. Theoretically, a well-designed and conducted RCT offers the highest level of evidence considering the outcome of a medical treatment, because the study design ideally minimize the risk of bias, increasing internal validity as stated by some authors.13 However, some peculiar aspects may limit importantly the external validity of the outcomes retrieved from RCTs. In fact, the blindness of patients and especially the operators could often be difficult, or (sometimes) impossible, to obtain. Moreover if we contemplate the use of a placebo group, it could be frequently not ethically acceptable. When performing a surgical procedure, such as endodontic surgery, an adequate comparison of two treatments could be limited by the strict inclusion criteria, established considering the surgical indications of two different treatments. Taking in Volume
-,
Number
-
consideration these assumptions, in a sense retrospective investigations and non-randomized prospective clinical trials may be exempted from the limitations of RCTs. However, a pragmatic clinical choice should be based on the existing evidence based on systematic reviews and meta-analysis of RCTs without ignoring the operator’s experience and the peculiar characteristics and expectations of the patient, particularly when evaluating surgical interventions. Several investigations explored the level of evidence of published articles in endodontic surgery, without attempting a correlation with journal impact factor.1 Mead et al found very few randomized clinical trials of level of evidence 2 according to the classification of the Center for Evidence-Based Medicine (www.cebm.net), while the majority of the so-called ‘‘success and failure’’ studies were case series. In the field of endodontics, other researches were published about the level of evidence in the published articles.14–16 As regard the outcome of nonsurgical endodontic treatment Torabinejad and coworkers15 concluded that in a period of 38 years, only few studies with high level of evidence were published related to outcomes of nonsurgical root canal therapy. An article evaluating the level of evidence for the outcome of endodontic retreatment was also published.16 The authors reported that the level of evidence, measured through the evaluation of available RCTs, was low due to the low number of high-level studies. However, none of the previously cited reviews related the number of publications with high level of evidence to the impact factor of the journal. To our knowledge, this is the first study that evaluated the correlation between the trend of high-quality articles published in a group of journals and the trend of impact factor of the same journals over a period of 10 years. This correlation presented many points of interest in the evaluation of the level of evidence in endodontics. In fact, this evaluation explored implicitly the attitude to be cited of high-level articles and can provide an important information to authors and editors in the management of the publications. This aspect, which is evident in the field of endodontics may be the results of a growing attitude of editors to request a high standard quality in the publications and of an increasing interest of authors to perform highlevel researches. Another review of the literature identified a statistically significant relationship between levels of evidence and impact factor in oral and maxillofacial surgery journals.17 This study, however, limited the analysis to a single year (2004) for impact factor and the two previous years (2002–2003) for the studies classified according to their level of evidence. Other researches, in many fields of medicine and surgery investigated the correlation between IFs and number of high-level published articles.13,18,19 5
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Impact factor is widely considered as a measure of the scientific weight of a journal or an author, and, as known, it is based on the number of citations a published article has. However, it is disputed the relevance of IF in the evaluation of the scientific ‘‘value’’ of a journal. In fact, there’s no correction for self-citations in IF calculation and one study reported that the most cited 50% of articles account for almost 90% of citations,20 which could be considered a limitation. Other authors criticize the use of IF as a measure for the quality of a journal.21 Even though the present review found a significant correlation between IF and number of high-level articles published, a recent review of the literature regarding the most cited articles in the field of endodontics presented data partially in contradiction with this consideration.22 The authors found that none of the 100 most-cited articles in endodontics is an RCT and only 17% of them were reviews (systematic or narrative). So, it has to be considered that most articles that accounted for IF calculation might be studies with low level of evidence but nevertheless significant for researchers.
CONCLUSIONS Considering the limitations of the present systematic review of the literature, an increasing amount of highlevel articles were published on endodontic journals over the 10-year period considered, demonstrating an increasing attention to the experimental design. The number of high-level articles is directly proportional to the IF of the journal even though the significance of this assumption is disputable.
REFERENCES 1. Mead C, Javidan-Nejad S, Mego ME, Nash B, Torabinejad M. Levels of evidence for the outcome of endodontic surgery. J Endod 2005;31:19-24. 2. Fletcher S, Sackett DL. The periodic health examination. Canadian Task Force on the periodic health examination. Can Med Assoc J 1979;121:1193-254.
6
3. Sackett DL. Bias in analytic research. J Chronic Dis 1979;32:51-63. 4. Sica GT. Bias in research studies. Radiology 2006;238:780-9. 5. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312:71-2. 6. Evidence Based Medicine Working Group. Evidence based medicine. A new approach to teaching the practice of Medicine. J Am Med Assoc 1992;268:2420-5. 7. Sutherland SE. Evidence-based dentistry: Part IV. Research design and levels of evidence. J Can Dent Assoc 2001;67:375-8. 8. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490. 9. Hess DR. Retrospective studies and chart reviews. Respir Care 2004;49:1171-4. 10. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887-92. 11. Garfield E. Citation indexes to science: a new dimension in documentation through association of ideas. Science 1955;122:108-11. 12. Garfield E. The history and meaning of the Journal impact factor. J Am Med Assoc 2006;295:90-3. 13. Amiri AR, Kanesalingam K, Cro S, Casey ATH. Level of evidence of clinical spinal research and its correlation with journal impact factor. Spine J 2013;13:1148-53. 14. Torabinejad M, Bahjri K. Essential elements of evidenced-based endodontics: steps involved in conducting clinical research. J Endod 2005;31:563-9. 15. Torabinejad M, Kutsenko D, Machnick TK, Ismail A, Newton CW. Levels of evidence for the outcome of nonsurgical endodontic treatment. J Endod 2005;31:637-46. 16. Paik S, Sechrist C, Torabinejad M. Levels of evidence for the outcome of endodontic retreatment. J Endod 2004;30:745-50. 17. Lau SL, Samman N. Levels of evidence and journal impact factor in oral and maxillofacial surgery. Int J Oral Maxillofac Surg 2007;36:1-5. 18. Ahn CS, Li RJ, Ahn BS, Kuo P, Bryant J, Day CS. Hand and wrist research productivity in journals with high impact factors: a 20 year analysis. J Hand Surg Eur Vol 2012;37:275-83. 19. Kuroki LM, Allsworth JE, Peipert JF. Methodology and analytic techniques used in clinical research: associations with journal impact factor. Obstet Gynecol 2009;114:877-84. 20. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314:498-502. 21. Hansson S. Impact factor as a misleading tool in evaluation of medical journals. Lancet 1995;346:906. 22. Fardi A, Kodonos K, Gogos C, Economides N. Top-cited articles in endodontic journals. J Endod 2011;37:1183-90.
-
2014
The Trend of Quality of Publications in Endodontic Surgery: A 10-year Systematic Survey of the Literature Massimo Del Fabbroa, Stefano Corbellab, Igor Tsesisc, and Silvio Taschieria a
Universita degli Studi di Milano, Department of Biomedical, Surgical and Dental Sciences, Oral Health Research Centre, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy b Universita degli Studi di Milano, Department of Biomedical, Surgical and Dental Sciences, Oral Implantology Research Centre, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy c Department of Endodontology, Maurice and Gabriela Goldschleger School of Dental Medicine, Tel Aviv University, Tel Aviv, Israel
Abstract
Objectives: The aims of the present systematic literature analysis were to evaluate, over a 10-year period, the trend of the proportion of RCT, SR, MA published on endodontic surgery, and to investigate if the impact factor (IF) of the main endodontic Journals correlates with the proportion of RCT, SR, MA they publish. Methods: An electronic search of the RCT, SR and MA published on the topic ‘‘endodontic surgery’’ from 2001 to 2010 was performed on Medline and Cochrane CENTRAL database using specific search terms combined with Boolean operators. Endodontic Journals impact factor was retrieved by the Thomson Scientific database. The proportion of each study type over the total number of articles on endodontic surgery published per year was estimated. The correlation between the number of high-evidence level studies published on the main endodontic Journals and the IF of such Journals per year was estimated. Results: From a total of 900 articles published in 2001–2010 on endodontic surgery, there were 114 studies of high evidence level. A significant increase of the proportion of either RCT, SR and MA over the years was found. A modest to unclear correlation was found between the Journal IF and the number of high-evidence articles published. Conclusions: There is a positive trend over the years among researchers in performing studies of good quality in endodontic surgery. The impact factor of endodontic Journals is not consistently influenced by publication of highevidence level articles. Keywords: Endodontics, Evidence-based dentistry, Journal impact factor.
INTRODUCTION Corresponding author. Tel.: +39 02 50319950; fax: +39 02 50319960;
E-mail: [email protected]. J Evid Base Dent Pract 2014;-:1-6 1532-3382/$36.00 Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jebdp.2014.09.002
The development of new techniques, instrumentation and biomaterials used in endodontic surgery has made possible the extension of its clinical indications. However, with the increase of health care costs, there has been a paradigm shift in health care toward evidence-based research.
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Many manufacturers and corporations tend to use effective marketing rather than peer-reviewed studies to promote their technological and biological advances.1 This trend can create a confusing picture for the endodontic surgeon, who has the responsibility for recommending the most appropriate surgical approach using a conscious critical analysis based on accurate diagnostic path. When a clinician discusses treatment planning with a patient, it is necessary to provide the patient with information related to the efficacy and long-term outcome of the various treatment options. These data are needed for informed decisions.1 Ranking the available evidence into different levels and grades of recommendation was first described by Fletcher & Sackett more than 25 years ago to give an idea of the quality of the evidence on the basis of the level of bias and flaws of the various types of study design adopted in the biomedical research.2 In general the level of evidence of a study is considered as inversely related to its level of bias. The latter can be defined as any uncontrolled trait of the experimental design that may affect the outcome, therefore producing a distorted result, which may not reflect the true effect of a given treatment.3,4 Sackett et al defined evidence-based medicine, also termed evidence-based practice (EBP), as ‘‘the conscientious, explicit and judicious use of current best evidence about the care of individual patients’ integrated with clinical expertise and patient values to optimize outcomes and quality of life.’’5 In the hierarchy of study designs used in clinical research, randomized controlled trials (RCT), prospective controlled trials (CT) and meta-analyses (MA) or systematic reviews (SR) (dealing with RCTs or controlled clinical trials) are considered to provide the highest level of evidence.6–8 Conversely, uncontrolled studies like case series and case reports, as well as retrospective studies are associated with a lower level of evidence. In other words, the latter types of investigation, due to the features of the study design such as the choice of the patients, the allocation of treatments, the absence of blinding procedures and many other methodological aspects that may somehow affect the outcomes, are considered to have a higher level of bias as compared to randomized controlled studies.9 RCTs are specifically designed to minimize the experimental bias in any steps of the study procedures, so as to provide the most reliable possible outcomes.10 Since the volume of published information is steadily increasing in the field of endodontics (as well as in many other fields of medical sciences) it is extremely important to assess the level of evidence of the publications, in order to discern which information should be relied upon to formulate an evidence-based treatment plan and provide the patients with the most accurate, up-to date and trustworthy information. 2
The most accredited tool for evaluating the weight of evidence in addition to the type of study design is the journal impact factor (IF), which indicates the average frequency of citation to any indexed journal.11,12 The higher the number of citations received by a Journal, the higher the impact of the Journal in the scientific community, which is considered as related to the importance and the reliability of the information provided by the articles published. Since the latter are normally chosen based upon a rigorous selection process it is believed that the best quality journals have a manuscript review process able to select the best quality information to be published and that, consequently, they have a high probability of receiving a high number of citations. Though the use of IF as an index of quality of the journals is still a matter of debate among scientists, in the absence of an alternative reliable index, the IF is currently adopted as a marker of the value of published scientific information. Similarly, scientists and clinicians able to have their studies published on a high-IF journal are considered as valuable researchers, independent of the number of citations that their specific articles will receive. The purpose of this study was to examine, in the field of endodontic surgery (ES), the relationship between quality of research in terms of levels of evidence and IF of main journals dealing with endodontic surgery, in an attempt to understanding the pattern of citation and the pattern of publication of high-evidence level studies on this topic over the last 10 years. In particular, the main hypothesis was that journals with higher IF tend to publish more frequently studies of high level of evidence as compared to journals with lower IF in which the proportion of high evidence level studies published tends to be lower. Another hypothesis was that the proportion of RCTs and meta-analysis/systematic reviews among the total number of articles published in the field of endodontic surgery tends to increase in the last 10 years, due to the spreading of the concept of evidence-based dentistry.
MATERIALS AND METHODS An electronic search was performed on Pubmed, on the Cochrane Central Register of Controlled Trials (CENTRAL) and on the Thomson Scientific database, which includes the Journal of Citation Reports containing the IF of indexed and impacted journals. The search was limited to 10 years between 2001 and 2010. The keywords used were: apical surgery, apicectomy, apicoectomy, endodontic surgery, periapical surgery, periradicular surgery, root-end management, surgical endodontics, surgical retreatment, alone or combined among them. No restriction was placed regarding the language. Technical studies and editorials were excluded. The reference lists of meta-analysis (MA) and systematic reviews (SR) were checked. A hand-search -
2014
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
was also performed on the main journals publishing articles in the field of endodontic surgery from 2001 to 2010: Journal of Endodontics, International Endodontic Journal, Journal of Oral & Maxillofacial Surgery, International Journal of Oral & Maxillofacial Surgery, Oral Surgery Oral Medicine Oral Pathology Oral Radiology and Endodontology, British Journal of Oral & Maxillofacial Surgery. The variables searched with Pubmed were: the total number of articles, the number of controlled clinical studies (CT), the number of RCTs, the number of systematic reviews and meta-analyses. The data were searched separately per each year and then tabulated and reported in graphic form as both absolute value and percentages. For the allocation of the articles to a given year the date on which they first appeared on Pubmed was taken into consideration (i.e. articles early online since 2007 and published in a given Journal issue in 2009 were allocated to 2007). The pattern of IF over the years considered was also tabulated for all the journals that were previously handsearched. For each study type of interest (MA, SR, RCT, CT) the proportion of studies published in each of the main endodontic journals was calculated. Subsequently, MA and SR, as well as CT and RCT were grouped so as to compare the trend of the best primary studies and of the best secondary studies. A correlation was searched between journals’ IF and level of evidence of the studies, grouping together all types of high-evidence level studies for each journal of interest. All studies of interest not published in any of the above journals were scored as ‘‘other’’ and not considered for the correlation with IF. Data were summarized by means of tables and histograms.
RESULTS The Pubmed search provided a total of 900 articles dealing with endodontic surgery, of which 114 were selected based on the study design (MA, SR, RCT, CT). Nineteen further articles were added after searching on the CENTRAL database. Therefore, out of 919 articles published over 10 years on endodontic surgery, a total of 133 articles were selected as high-evidence level studies (14.5%). No further study was added after handsearching. Out of a total of 110 articles indexed on Pubmed as reviews over 10 years, there were 23 systematic reviews and 13 meta-analyses. Taken together they represented 32.7% of the total reviews and 3.9% of the total articles. Controlled trials and RCTs published over 10 years were 35 and 62 (3.8% and 6.7% of total articles), respectively. Taken together they represented 10.6% of the total articles retrieved. The above numbers provide only a general idea of the mean percentage (less than 15%) of high-quality articles Volume
-,
Number
-
Figure 1. Endodontic surgery-high evidence-level articles. The number of high evidence-level articles published in the endodontic surgery field is steadily increasing over the years.
Figure 2. Endodontic surgery-high evidence-level articles. The trend of the percentage of high evidence-level articles published in the endodontic surgery field also reflects an increase over the years.
published in the scientific literature on endodontic surgery. Trends and correlations were then evaluated. Figures 1 and 2 show, respectively, the trend for the absolute number and the percentage of high evidencelevel articles in endodontic surgery. When comparing primary studies (CT + RCT) and secondary studies (SR + MA), it appears that both types of studies tend to progressively increase over time, suggesting a positive trend of researchers toward improving the level of publications’ quality. The increase in the proportion of such articles tends to be similar, being the trend lines parallel (Figure 2), with greater values for clinical studies as compared to reviews. For comparison, the same trends have been evaluated for the topic ‘‘endodontics.’’ Figures 3 and 4 show, respectively, the trend for the absolute number and the 3
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Figure 3. Endodontics-high evidence-level articles. The percentage of high evidence-level articles published in the endodontics field seems to increase much less than for endodontic surgery over the years. Instead, the percentage of systematic reviews is about half than for endodontic surgery (see Figure 2) and seems unchanged over time. It has to be considered that this graph includes articles considered in Figure 2.
Figure 4. Endodontics-high evidence-level articles. The number of high evidence-level articles published in the endodontics field is steadily increasing over the years. This well compares to the Figure 1 for clinical articles and, on a lesser degree, for the systematic reviews. This graph does not include endodontic surgery articles considered in Figure 1.
percentage of high evidence-level articles in endodontics, without considering the data for endodontic surgery that have been removed. A trend of increasing the amount (Figure 1) and the proportion (Figure 2) of clinical high-evidence level publications was found, similar to endodontic surgery. Overall proportion of high evidence-level clinical studies in endodontics was higher than for endodontic surgery (16.5% vs 10.6%). Conversely, the increase for SR and MA seems less pronounced than for corresponding data of endodontic 4
Figure 5. Trend of impact factor. Among articles publishing endodontic surgery articles, JOE and IEJ show the most consistent increase in impact factor over 10 years (only the regression line is shown).
Figure 6. 10-y Journal IF vs high evidence-level articles. This graph shows that the higher the (mean) journal impact factor, the higher the number of high evidence-level articles published over 10 years. There is a good correlation (r2 ¼ 0.83) between IF & quality of article.
surgery, and also the overall proportion of SR and MA is about half than for endodontic surgery (2.1% vs 3.9%). In Figure 5 is reported the trend for the journal impact factor, which shows a consistent increase for the two main endodontic journals (Journal of Endodontics and International Endodontic Journal) as compared to the other journals considered for which a very small increase was found. The table attached to Figure 5 reports the actual values of IF from 2001 to 2010 for the six journals mainly involved in publishing articles on endodontic surgery. Figure 6 shows a very interesting correlation between the cumulative number of high evidence-level articles published on a journal and the mean journal impact factor over 10 years. A strong correlation was found (r2 ¼ 0.83), suggesting that the higher the IF, the greater -
2014
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Figure 7. Impact factor & number of articles. This graph considers data per each year of the three journals with the higher number of high evidence-level articles. Only for JOE there is a modest correlation between IF and no. of good quality articles per given year.
the amount of good quality articles published in the journal. This finding was further explored in greater detail. The three journals with the higher number of high evidence level articles were selected for a further analysis in which the data of each single year were considered. As shown in Figure 7 only for the Journal of Endodontics a modest correlation was found, while for other journals publishing less high evidence level articles such a correlation was not clear. This would suggest that the more high evidence level articles are published, the higher the number of citations received by the journal.
DISCUSSION The present study aimed at exploring the level of evidence of published articles in the field of endodontics and endodontic surgery. The number of RCTs on a particular journal was considered the main parameter to evaluate the level of evidence. Theoretically, a well-designed and conducted RCT offers the highest level of evidence considering the outcome of a medical treatment, because the study design ideally minimize the risk of bias, increasing internal validity as stated by some authors.13 However, some peculiar aspects may limit importantly the external validity of the outcomes retrieved from RCTs. In fact, the blindness of patients and especially the operators could often be difficult, or (sometimes) impossible, to obtain. Moreover if we contemplate the use of a placebo group, it could be frequently not ethically acceptable. When performing a surgical procedure, such as endodontic surgery, an adequate comparison of two treatments could be limited by the strict inclusion criteria, established considering the surgical indications of two different treatments. Taking in Volume
-,
Number
-
consideration these assumptions, in a sense retrospective investigations and non-randomized prospective clinical trials may be exempted from the limitations of RCTs. However, a pragmatic clinical choice should be based on the existing evidence based on systematic reviews and meta-analysis of RCTs without ignoring the operator’s experience and the peculiar characteristics and expectations of the patient, particularly when evaluating surgical interventions. Several investigations explored the level of evidence of published articles in endodontic surgery, without attempting a correlation with journal impact factor.1 Mead et al found very few randomized clinical trials of level of evidence 2 according to the classification of the Center for Evidence-Based Medicine (www.cebm.net), while the majority of the so-called ‘‘success and failure’’ studies were case series. In the field of endodontics, other researches were published about the level of evidence in the published articles.14–16 As regard the outcome of nonsurgical endodontic treatment Torabinejad and coworkers15 concluded that in a period of 38 years, only few studies with high level of evidence were published related to outcomes of nonsurgical root canal therapy. An article evaluating the level of evidence for the outcome of endodontic retreatment was also published.16 The authors reported that the level of evidence, measured through the evaluation of available RCTs, was low due to the low number of high-level studies. However, none of the previously cited reviews related the number of publications with high level of evidence to the impact factor of the journal. To our knowledge, this is the first study that evaluated the correlation between the trend of high-quality articles published in a group of journals and the trend of impact factor of the same journals over a period of 10 years. This correlation presented many points of interest in the evaluation of the level of evidence in endodontics. In fact, this evaluation explored implicitly the attitude to be cited of high-level articles and can provide an important information to authors and editors in the management of the publications. This aspect, which is evident in the field of endodontics may be the results of a growing attitude of editors to request a high standard quality in the publications and of an increasing interest of authors to perform highlevel researches. Another review of the literature identified a statistically significant relationship between levels of evidence and impact factor in oral and maxillofacial surgery journals.17 This study, however, limited the analysis to a single year (2004) for impact factor and the two previous years (2002–2003) for the studies classified according to their level of evidence. Other researches, in many fields of medicine and surgery investigated the correlation between IFs and number of high-level published articles.13,18,19 5
JOURNAL OF EVIDENCE-BASED DENTAL PRACTICE
Impact factor is widely considered as a measure of the scientific weight of a journal or an author, and, as known, it is based on the number of citations a published article has. However, it is disputed the relevance of IF in the evaluation of the scientific ‘‘value’’ of a journal. In fact, there’s no correction for self-citations in IF calculation and one study reported that the most cited 50% of articles account for almost 90% of citations,20 which could be considered a limitation. Other authors criticize the use of IF as a measure for the quality of a journal.21 Even though the present review found a significant correlation between IF and number of high-level articles published, a recent review of the literature regarding the most cited articles in the field of endodontics presented data partially in contradiction with this consideration.22 The authors found that none of the 100 most-cited articles in endodontics is an RCT and only 17% of them were reviews (systematic or narrative). So, it has to be considered that most articles that accounted for IF calculation might be studies with low level of evidence but nevertheless significant for researchers.
CONCLUSIONS Considering the limitations of the present systematic review of the literature, an increasing amount of highlevel articles were published on endodontic journals over the 10-year period considered, demonstrating an increasing attention to the experimental design. The number of high-level articles is directly proportional to the IF of the journal even though the significance of this assumption is disputable.
REFERENCES 1. Mead C, Javidan-Nejad S, Mego ME, Nash B, Torabinejad M. Levels of evidence for the outcome of endodontic surgery. J Endod 2005;31:19-24. 2. Fletcher S, Sackett DL. The periodic health examination. Canadian Task Force on the periodic health examination. Can Med Assoc J 1979;121:1193-254.
6
3. Sackett DL. Bias in analytic research. J Chronic Dis 1979;32:51-63. 4. Sica GT. Bias in research studies. Radiology 2006;238:780-9. 5. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312:71-2. 6. Evidence Based Medicine Working Group. Evidence based medicine. A new approach to teaching the practice of Medicine. J Am Med Assoc 1992;268:2420-5. 7. Sutherland SE. Evidence-based dentistry: Part IV. Research design and levels of evidence. J Can Dent Assoc 2001;67:375-8. 8. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490. 9. Hess DR. Retrospective studies and chart reviews. Respir Care 2004;49:1171-4. 10. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887-92. 11. Garfield E. Citation indexes to science: a new dimension in documentation through association of ideas. Science 1955;122:108-11. 12. Garfield E. The history and meaning of the Journal impact factor. J Am Med Assoc 2006;295:90-3. 13. Amiri AR, Kanesalingam K, Cro S, Casey ATH. Level of evidence of clinical spinal research and its correlation with journal impact factor. Spine J 2013;13:1148-53. 14. Torabinejad M, Bahjri K. Essential elements of evidenced-based endodontics: steps involved in conducting clinical research. J Endod 2005;31:563-9. 15. Torabinejad M, Kutsenko D, Machnick TK, Ismail A, Newton CW. Levels of evidence for the outcome of nonsurgical endodontic treatment. J Endod 2005;31:637-46. 16. Paik S, Sechrist C, Torabinejad M. Levels of evidence for the outcome of endodontic retreatment. J Endod 2004;30:745-50. 17. Lau SL, Samman N. Levels of evidence and journal impact factor in oral and maxillofacial surgery. Int J Oral Maxillofac Surg 2007;36:1-5. 18. Ahn CS, Li RJ, Ahn BS, Kuo P, Bryant J, Day CS. Hand and wrist research productivity in journals with high impact factors: a 20 year analysis. J Hand Surg Eur Vol 2012;37:275-83. 19. Kuroki LM, Allsworth JE, Peipert JF. Methodology and analytic techniques used in clinical research: associations with journal impact factor. Obstet Gynecol 2009;114:877-84. 20. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314:498-502. 21. Hansson S. Impact factor as a misleading tool in evaluation of medical journals. Lancet 1995;346:906. 22. Fardi A, Kodonos K, Gogos C, Economides N. Top-cited articles in endodontic journals. J Endod 2011;37:1183-90.
-
2014
