REVIEW REVIEW Venkatesh S. Madhugiri, MCh‡ Gopalakrishnan M. Sasidharan, MCh‡ Venkatesan Subeikshanan*§ Akshat Dutt*§ Sudheer Ambekar, MCh¶ Shane F. Strom, BSk ‡Department of Neurosurgery, Jawaharlal Institute of Post-graduate Medical Education and Research, Pondicherry, India; §Jawaharlal Institute of Post-graduate Medical Education and Research, Pondicherry, India; ¶Clinical Fellow, Department of Neurosurgery, Louisiana State University Health Sciences Center, Shreveport, Louisiana; kLouisiana State University Health Sciences Center, Shreveport, Louisiana *These authors have contributed equally to this article. Correspondence: Venkatesh S. Madhugiri, MCh, Department of Neurosurgery, 2nd Floor, SS Block, Jawaharlal Institute of Post-graduate Medical Education and Research, Pondicherry 605006, India. E-mail: [email protected] Received, April 1, 2014. Accepted, December 3, 2014. Published Online, January 29, 2015. Copyright © 2015 by the Congress of Neurological Surgeons.

An Analysis of the Citation Climate in Neurosurgical Literature and Description of an Interfield Citation Metric BACKGROUND: The citation climate in neurosurgical literature is largely undefined. OBJECTIVE: To study the patterns of citation of articles in neurosurgery as a scientific field and to evaluate the performance of neurosurgery journals vis-a`-vis journals in other fields. METHODS: References cited in articles published in neurosurgery journals during a specified time period were analyzed to determine the age of articles cited in neurosurgical literature. In the next analysis, articles published in neurosurgical journals were followed up for 13 years after publication. The postpublication citation patterns were analyzed to determine the time taken to reach the maximally cited state and the time when articles stopped being cited. The final part of the study dealt with the evolution of a new interfield citation metric, which was then compared with other standardized citation indexes. RESULTS: The mean 6 SD age of articles cited in neurosurgical literature was 11.6 6 11.7 years (median, 8 years). Citations received by articles gradually increased to a peak (at 6.25 years after publication in neurosurgery) and then reached a steady state; articles were still cited well into the late postpublication period. Neurosurgical articles published in nonneurosurgical high-impact journals were cited more highly than those in neurosurgical journals, although they took approximately the same time to reach the maximally cited state (7.2 years). The most cited pure neurosurgery journal was Neurosurgery. CONCLUSION: The citation climate for neurosurgery was adequately described. The interfield citation metric was able to ensure cross-field comparability of journal performance. KEY WORDS: Citations, H index, Impact factor, Interfield variations, Metrics, Neurosurgery Neurosurgery 76:505–513, 2015

DOI: 10.1227/NEU.0000000000000656

T

he current knowledge paradigm in science comprises a cycle of events: research / submission of findings to journal / peer review / publication / further research based on published data / publication of fresh data and citation of an older article in a new publication.1 Thus, the impact of a medical article WHAT IS THIS BOX? A QR Code is a matrix barcode readable by QR scanners, mobile phones with cameras, and smartphones. The QR Code above links to Supplemental Digital Content from this article.

NEUROSURGERY

ABBREVIATIONS: G1, group 1; G2, group 2; G3, group 3; G4, group 4; IFCM, interfield citation metric Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.neurosurgery-online.com).

www.neurosurgery-online.com

on its readers could lead either to a change in clinical practice or to the accrual of further research. However, these are imponderables that are difficult to quantify. Possibly the only measurable metrics of the impact an article (or journal) are citation counts and various modifications thereof.2,3 Citation metrics provide valuable information about the process of clinical practice and research. Several factors influence the dynamics of a scientific or medical field. One of them would be how soon newly published data gain currency. This factor could be estimated by determining 2 parameters: the age of the articles cited by authors in current literature and the time taken by an article, once published, to reach its maximally

VOLUME 76 | NUMBER 5 | MAY 2015 | 505

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

MADHUGIRI ET AL

cited state. A common occurrence in science is that, as newer articles and fresh data come into vogue, older articles may “die,” ie, are not cited any more. Thus, the pace of change of medical practice would be established by determining when older articles cease to be cited. The preceding data could collectively be called the citation climate for a particular field.4 The citation climate has not been adequately described for most medical fields. Citation metrics are often used to monitor the performance of individuals, departments, and even universities as a whole.5 Despite their manifest utility, citation count–based metrics have major limitations. Citation metrics would obviously depend on the writing and citing habits of authors in a field.6 Medical authors cite more articles on average than, for example, mathematicians.3 This would be reflected in the journal metrics. For instance, the Journal of the American Mathematical Society (considered the leading general mathematical journal in the world) had an impact factor of 3.567 in 2012. In the same year, the impact factor for the New England Journal of Medicine was 51.658. Although the Journal of the American Mathematical Society may be as useful to mathematicians as the New England Journal of Medicine is to physicians, citation metrics do not reflect this. Several standardized indexes have therefore been developed to ensure comparability of journal metrics across fields.6 Two of these are the relative index by Radicchi et al7 and the Bornmann and Daniel z score.8 These metrics normalize the citation performance of an article (or journal) against the mean performance or mean variance for that field, respectively. However, the derivation of these indexes is not intuitive, and the imputation of a Radicchi index or z score value is not readily apparent. An ideal interfield metric should not only ensure cross-field comparability but also be easy to understand and have broad utility. In this article, we first describe the citation climate for neurosurgery. Then we use the citation climate data to compare the performance of neurosurgery journals with that of journals pertaining to other scientific disciplines. For this purpose, we evolved a new interfield citation metric (IFCM) and compared it with the existing metrics.

Age of the Articles Cited in Neurosurgical Literature Indexed neurosurgery journals identified from the National Center for Biotechnology Information database are listed in Table 1. Articles published in the January through March 2012 issues of these journals were accessed. The references cited in each of these articles were collated to generate a database of articles cited in neurosurgery journals. The age of a cited article was computed as 2012 minus the year of publication. The mean and median ages of the articles in this database were then calculated.

TABLE 1. Journals Selected for the Analysis of Citation Patternsa

Journal Group 1: neurosurgery journals Stroke Journal of Neurosurgery Neurosurgery Journal of Neurotrauma Journal of Neurology, Neurosurgery and Psychiatry Spine Journal of Neuro-oncology Canadian Journal of Neurological Sciences Neuro-oncology Surgical Neurology European Spine Journal Spinal Cord Acta Neurochirurgica Journal of Spinal Disorders and Techniques Pediatric Neurosurgery Clinical Neurology and Neurosurgery Childs Nervous System British Journal of Neurosurgery Journal of Clinical Neurosciences Neurology India Turkish Neurosurgery Journal of Neurosurgery-Spine Neurosurgical Focus Group 2: high-impact general medical journals New England Journal of Medicine JAMA The Lancet Group 3: high-impact basic science journals Cell Nature

METHODS The Citation Climate in Neurosurgery The analysis of the citation climate in neurosurgery was performed in 2 parts. The first part focused on determining the age of the articles cited by current neurosurgical authors. The second part analyzed the postpublication citation patterns for articles published in neurosurgical journals to determine the time taken by articles to reach the maximally cited state and the point when they ceased to be cited. Only MEDLINE-indexed journals listed in the National Center for Biotechnology Information database were included in the analyses.9 The categories of articles included were original research articles, case series ($ 4 cases), clinical research and investigation reports, randomized controlled trials, review articles, systematic reviews of literature, and meta-analyses.10 These are the categories of articles that are considered for the computation of journal metrics.

506 | VOLUME 76 | NUMBER 5 | MAY 2015

Impact Factor in 2000

SCImago Journal Rank Index in 2000

6.008 2.918 2.899 2.877 2.846

3.13 1.53 1.55 1.17 1.12

1.843 1.581 1.504

1.12 0.58 0.63

1.31 1.018 0.98 0.913 0.817 0.816

0.40 0.63 0.68 0.70 0.51 0.83

0.811 0.619

0.51 0.38

0.563 0.539 0.178 0.092 0.02 0.00 0.00

0.36 0.27 0.16 0.13 0.11 0.00 0.00

18.01

6.869

7.31 6.92

3.466 2.651

30.25 14.74

30.318 10.767

a

The neurosurgery group was populated by searching the National Center for Biotechnology Information database using the string “neurosurgery.” Only Medlineindexed journals were included. These journals are sorted by their impact factors. The group 2 and journals are the highest-ranked journals in these groups and merely serve as a comparison group for group 1. The group 4 journals are listed separately.

www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

CITATION CLIMATE IN NEUROSURGERY

FIGURE 1. Determination of the age of the articles cited in current neurosurgical literature. A, the relation between the year of publication and the number of citations drawn from that year, from 1800 to 2009. It is evident that, from the 1960s on, the number of citations from each year dramatically increases. The subsequent graphs split this pattern for before and after 1962. B, the age distribution of citations for articles .50 years old (published before 1962). There is no clear mathematical relationship in this plot. C, the age distribution of citations ,50 years old. The regression function takes the form of an exponential curve, y = 2817.1e20.106x, with a robust predictive power (R2 = 0.9822).

Postpublication Citation Patterns For this analysis, neurosurgery journals were designated as group 1 (G1). To check whether the postpublication citation trends would look different for other medical journals, we generated 3 informal comparison groups (Table 1). Group 2 (G2) included high-impact general medical journals (New England Journal of Medicine, Journal of the American Medical Association, and The Lancet). Group 3 (G3) included the basic biological science journals Nature and Cell. Group 4 (G4) consisted of neurosurgery articles published in nonneurosurgical highimpact journals. All medical (nonneurosurgical) journals with an impact factor .5 listed in the Thomson Reuters database were included in G4 (See the Table, Supplemental Digital Content 1, available at http://links.lww.com/NEU/A711). Neurosurgery articles published in these journals were identified by use of the medical subject heading “neurosurgery” in a PubMed search.9

NEUROSURGERY

Articles published in the January through March 2000 issues of journals in all 4 groups were tracked from the time of publication to the time this analysis concluded (November 2013). The number of citations received by every article in each postpublication year was tabulated and plotted on graphs. The early postpublication period was defined as the year of publication and the first postpublication year; the late period was beyond 10 years.11 The following parameters were computed: the mean citations received by articles published in each journal, the mean time taken for an article to reach its maximally cited state, and the time when articles stopped being cited.

Analysis of the IFCMs Five citation metrics were compared to evaluate which ensured the best interfield comparison. These included impact factor, the H index, the Radicchi index, the z score, and the IFCM. These metrics were validated

VOLUME 76 | NUMBER 5 | MAY 2015 | 507

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

MADHUGIRI ET AL

over neurosurgery and 4 other arbitrarily chosen fields: anesthesiology, neuroradiology, basic biological sciences, and material science. Medical journals and basic biological science journals were identified from the National Center for Biotechnology Information database.10 Material science journals were picked from the SCImago Journal & Country Rank list.12 The impact factor and the H index for each journal were obtained from the Thomson Reuters and SCImago databases.12 Articles published between January and March 2000 were listed, and the total number of citations received by each article (up to 2013) was determined with the use of Google Scholar. Three parameters were calculated: c, the mean citations received by articles in a particular journal; c0, the mean of the citations received by all articles in that field; and s, the standard deviation for c0. The Radicchi index was calculated as cf 5 c=c0 .7 The z score was calculated as z = (c0 2 c)/s.8 The new metric IFCM was defined as IFCM = (c 2 c0)/c0.

TABLE 2. Citation Statistics for the 4 Groups Studieda

Group Group 1: Neurosurgery Group 2: New England Journal of Medicine, JAMA, The Lancet Group 3: Nature and Cell Group 4: Neurosurgery articles published in nonneurosurgery journals with an impact factor .5

Mean 6 SD Citations Per Article

Median Citations Per Article

60.82 6 63.3 550.41 6 740.03

44 315.5

721.15 6 1435.47 489.64 6 505.14

312.5 318.5

a The mean citations per article (c0) values for journals belonging to each of the groups compared in this study. This value of c0 is for the 13-year period beginning in 2000 (when the articles were published) through 2013, when this analysis concluded.

RESULTS Age of the Articles Cited in Neurosurgical Literature Neurosurgery authors had cited 23 046 articles in articles published between January and March 2012; these citations had been drawn from across 2522 journals. The oldest cited article was published in 1818. The mean 6 SD age of articles cited in neurosurgical literature was 11.6 6 11.7 years; the median age was 8 years. Nearly 99% of all cited articles had been published within the past 50 years. Articles published after 1962 were cited significantly more often than those published before (Figure 1A). For the period before 1962, there was no clear relationship between the year of publication (and thus the age) of the citations and the number of articles cited from that year (Figure 1B). In contrast, beginning in 1962, articles published in each subsequent year were cited in exponentially increasing numbers (Figure 1C). Postpublication Citation Patterns Postpublication citation patterns were studied for G1 through G4. The 23 neurosurgery journals in G1 (Table 1) had published 576 articles that met the inclusion criteria. The mean number of citations (over the period of 2000-2013) for neurosurgery articles published in these journals was 60.82 6 63.3; the median was 44 (Table 2). The mean number of citations received in the early postpublication period was 3.45 6 4.5 per article (median, 22); 5.67% of the total citations occurred in the early postpublication period, and 24.7% of published articles were not cited in the early postpublication period. Articles that were not cited in the early period received significantly fewer citations overall (19.64 6 20.3) than those that received at least 1 citation in the early period (74.33 6 66.69; P , .01). Articles received on average 12.76 6 15.67 citations in the late period (median, 29); 20.98% of the total citations occurred in the late period. In the neurosurgery group (G1), 2.96% of the articles were never cited in the 13 years after their publication that this study spanned. The pattern of citations that articles in G1 through G4 received after publication is depicted in Figure 2. Citation rates were low in the early period, gradually increased to a peak, and stabilized

508 | VOLUME 76 | NUMBER 5 | MAY 2015

thereafter. Articles in G1 neurosurgery journals received their maximum annual citations 6.25 6 3.4 years after publication, whereas articles in G2 and G3 journals took a significantly shorter time (3.89 6 3.53 years) to reach their maximally cited state (t = 26.8; P , .001). G4 articles took 7.2 6 4.3 years after publication to reach their maximally cited state. Although this was not significantly different from the G1 articles, a t test revealed that G4 articles took significantly longer than G2 and G3 articles to reach their maximally cited state (t = 3.25; P = .001). The point at which neurosurgery articles “died” could not be determined because articles continued to be cited even 13 years after publication (2000-2013), when this analysis ended (Figure 2). Interestingly, the median citations per article in G2, G3, and G4 were nearly the same (and significantly higher than for G1 articles; Table 2). Expectedly, therefore, the citation curves for these groups were clustered together. However, the citation pattern curves for all groups were parallel and could be mathematically represented by the same type of function (fourth-order polynomial; Figure 2). It appears that this may be the universal shape of the curve depicting postpublication citations of biological science articles. Analysis of the IFCMs The IFCM was calculated for 115 indexed journals representing the 5 chosen fields (neurosurgery, 23; neuroradiology, 5; anesthesiology, 23; basic biological sciences, 33; and material science, 31; see Table 3 and Supplemental Digital Content 2, available at http:// links.lww.com/NEU/A712). The worst possible citation performance by a journal would be if articles published in the journal were not cited at all; in such an instance, c = 0. If this were to be inserted into the formula IFCM = (c 2 c0)/c0, it is evident that IFCM = 21. Therefore, the minimum possible value of the IFCM is 21. Journals that are not cited at all (eg, articles published in Turkish Neurosurgery during the period selected for this analysis) would have an IFCM = 21 and perform 100% (21 · 100%) worse than the average citation performance of the group. Articles published in a journal with IFCM = 20.5 (eg, Acta Neurochirurgica and Journal

www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

CITATION CLIMATE IN NEUROSURGERY

FIGURE 2. The postpublication citation patterns. The postpublication citation trend for group 1 neurosurgery journals (blue line) compared with that for group 2 (general medical journals), group 3 (basic biological sciences), and group 4 (neurosurgery articles published in nonneurosurgery journals). The citation curves for all 4 groups are mathematically similar and take the form of a fourth-order polynomial function of the form y = ax4 1 bx3 1 cx2 1 dx 1 e, where a, b, c, d, and e vary between groups.

of Neuro-oncology; Supplemental Digital Content 2, available at http://links.lww.com/NEU/A712) are, on average, 50% less cited than the average performance expected of a neurosurgical article. Thus, a journal (or article) with IFCM = 0 would meet the expectation for the average performance in that field (c = c0). There is theoretically no upper bound to the value of the IFCM. For example, Nature had an IFCM of 3.90 (390% more cited than the group average), and Progress in Materials Science had an IFCM of 45.62. The highest-ranked pure neurosurgery journal was Neurosurgery (IFCM = 0.34). Only the multispecialty journals Stroke (IFCM = 0.86) and Spine (IFCM = 0.48) scored higher.13 The plots of the metrics for the journals in the various fields analyzed are displayed in Figure 3. Each panel represents the plot for 1 metric. A higher degree of overlap between the lines would imply better normalization and interfield comparability with that metric. The plots for impact factor and H index reveal the disparity between the basic sciences group and the other fields, implying poor cross-field comparability (Figures 3A and 3B). The z scores also showed poor overlap, especially for the very poorly and very highly cited journals (Figure 3C). The Radicchi index showed good concordance across fields (Figure 3D), as did the plots for the IFCM (Figures 3E and 3F).

DISCUSSION In 1965, Price11 found that nearly 10% of published articles were never cited and that articles . 10 years old were rarely cited. Since the publication of Price’s article, use of the Internet, social media, and major scientific compendia such as PubMed has become nearly universal.14-16 The impact of these events on current knowledge-sharing paradigms and citation patterns is not known.

NEUROSURGERY

Neurosurgery articles received a median of 44 citations over the first 13 years of their lives. Neurosurgical articles in neurosurgery journals took 6.25 years and those in nonneurosurgery high-impact journals took 7.2 years after publication to attain their maximally cited state. In comparison, the time taken by articles in high-impact general medical or basic science journals to become maximally cited was 3.89 years. Therefore, regardless of the journal in which a neurosurgery article was published, it appeared to take 6.25 to 7.2 years for neurosurgery articles to reach their maximally cited state. Articles cited in current neurosurgical literature were about 8 years old (median age). This indicates that current practice and research in neurosurgery are based on data that are 6.25 to 8 years old. This reflects the time taken for new data to gain currency in neurosurgery. The rate at which fresh findings replace old data (ie, the time at which neurosurgery articles died) could not be determined because articles published in 2000 continued to be cited even 13 years after publication (the span of this study). These data provide a reasonable picture of the citation climate in neurosurgery. The citation climate of a field could be used to monitor the rate of progress of science in that field. When considered vis-à-vis funding levels and the number of scientists working in that domain, this information could be used to generate estimates of productivity for each field. Neurosurgery Articles in Nonneurosurgical Journals Neurosurgery articles published in nonneurosurgical highimpact journals form an interesting subset of neurosurgical literature. These articles were much more cited (median citations, 318.5) than articles in neurosurgery journals (mean citations, 44). However, they took nearly the same time to become maximally cited (7.2 years) as articles in neurosurgery journals (6.25 years). Thus, although citation rate appears to be a function of journal

VOLUME 76 | NUMBER 5 | MAY 2015 | 509

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

MADHUGIRI ET AL

TABLE 3. Analysis of the Standardized Journal Metricsa Journal Neurosurgery (median citations per article in this field = 44) Stroke Spine Neurosurgeryb Journal of Neurosurgery Journal of Neurology, Neurosurgery and Psychiatry European Spine Journal Journal of Neurotrauma Journal of Neurosurgery—Spine Journal of Spinal Disorders and Techniques Neuro-oncology Spinal Cord Surgical Neurology Childs Nervous System Canadian Journal of Neurological Sciences Acta Neurochirurgica Journal of Neuro-oncology Neurosurgical Focus Pediatric Neurosurgery British Journal of Neurosurgery Clinical Neurology and Neurosurgery Journal of Clinical Neurosciences Neurology India Turkish Neurosurgery Neuroradiology (median citations per article in this field = 39) Human Brain Mapping AJNR. American Journal of Neuroradiologyb Anesthesiology (median citations per article in this field = 20) Anesthesia and Analgesiab Basic biological sciences (median citations per article in this field = 177) Nature Cellb Material science (median citations per article in this field = 18) Progress in Materials Scienceb

Impact Factor

H Index

Interfield Citation Metric (2000-2013)

z Score

Radicchi Index

5.729 2.159 2.532 3.148 4.924 2.133 4.295 1.978 1.767 6.18 1.899 1.765 1.241 3.987 1.546 3.115 2.487 0.423 0.858 1.234 1.253 1.044 0.577

208 156 132 138 129 73 88 43 59 54 64 48 45 59 72 42 50 40 41 43 25 9

0.86 0.48 0.34 0.32 0.17 0.10 0.05 0.02 20.05 20.21 20.28 20.34 20.42 20.47 20.50 20.50 20.53 20.55 20.63 20.78 20.82 20.91 21.00

20.87 20.49 20.36 20.34 20.19 20.12 20.07 20.05 0.03 0.18 0.25 0.31 0.39 0.44 0.47 0.47 0.49 0.52 0.60 0.74 0.78 0.87 0.96

1.91 1.51 1.38 1.35 1.20 1.13 1.07 1.05 0.97 0.81 0.73 0.68 0.59 0.55 0.52 0.51 0.49 0.46 0.38 0.23 0.19 0.09 0.00

6.878 3.167

113 113

1.05 0.18

20.64 20.11

2.05 1.18

3.3

130

1.15

20.70

2.15

38.597 31.957

768 521

3.90 2.39

21.41 20.87

4.90 3.39

23.194

77

45.62

214.08

a

The journals in each of the 5 fields are sorted by the interfield citation metric score they received. The complete list of journals in fields other than neurosurgery is given in Supplemental Digital Content 2 (http://links.lww.com/NEU/A712). b The highest-ranked single-field journal in each field.

metrics, the time taken to become maximally cited seems to depend on the field rather than the journal. The high citation rate for articles published in cross-field journals has important implications. The significance of this fact for authors is evident: An article would obviously be cited more if it were published in a nonneurosurgical cross-specialty high-impact journal. The impact on readers is more indirect: Neurosurgeons would generally follow pure neurosurgery or neurosurgical subspecialty journals. Thus, these highly cited neurosurgical articles published in nonneurosurgical journals may escape readers’ attention for several years after they are published. These articles would be expected to become highly cited by neurosurgeons 7.2 years after publication, and only then would they gain the attention of the mainstream neurosurgical community.

510 | VOLUME 76 | NUMBER 5 | MAY 2015

Analysis of the IFCMs Physicians on average read more scientific literature than engineers and scientists do.17 Despite the widespread prevalence of the Internet, the popularity of online compendia such as PubMed, and the availability of options for subscribing to specialty rich site summary feeds, studies show that the major sources of information for most physicians are print issues of the journals on an individual’s subscription list.17,18 Thus, it is important to decide which journals to include on an individual or institutional subscription list. Citation metrics are often used for this purpose (Table 4). The impact factor is a simple average of citations to articles in a journal in the first 2 postpublication years. The very early citation pattern (year 0) is described by the immediacy index.

www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

CITATION CLIMATE IN NEUROSURGERY

FIGURE 3. Plots of the interfield journal metrics. Each line on the plots corresponds to 1 field, and every point on that line corresponds to the value of the metric for 1 particular journal. Greater overlap between the lines representing various fields implies better cross-field comparability with that particular metric. A, plot of the raw impact factors. B, plot of the H indexes of journals. C, plot of the z score calculated as z = (c0 2 c)/s (see text). D, plot of the Radicchi index calculated as cf = c/c0 (see text). E, plot of the interfield citation metric (IFCM) values calculated using the mean. F, plot of IFCM calculated using the median.

NEUROSURGERY

VOLUME 76 | NUMBER 5 | MAY 2015 | 511

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

MADHUGIRI ET AL

TABLE 4. The Common Journal Metrics in Regular Use Metric Journal impact factor Journal immediacy index Journal cited half-life Eigenfactor Article influence H index Citation rank list Radicchi normalized score SCImago Journal Rank z Score Interfield citation metric

Definition Citations of articles from the last 2 full years divided by the total number of articles published over the same 2 y Citations to articles from the current year divided by the total number of articles from the current year Median age of journal articles cited Similar to impact factor but eliminates self-citations Eigenfactor score divided by the number of articles published in journal H is the largest number of articles that have each been cited H times Ranks journals by number of citations received only in other journals in the same field Mean number of citations per article divided by the mean for that field Prestige of the citing journal is taken into account Uses the SD of the mean Normalized with the mean for that field

The Eigenfactor is similar to the impact factor but is probably a more accurate reflection of the performance a journal because self-citations are eliminated. The H index is the number of articles in a journal receiving a certain minimum number of citations. Many of these indexes (H index, immediacy index) may be used to evaluate the performance of individuals, departments, institutions, articles, and journals.5,19 There are 2 major problems with citation-count based metrics. The first is that these metrics do not evaluate the performance of a neurosurgical journal on the basis of its utility to the neurosurgical community. Although published in a neurosurgery journal, an article may be of greater use to other specialties. This problem may be overcome by ranking journals on the basis of the citations received in other neurosurgical journals only.1 However, this technique (which uses the Bradford distribution) cannot be applied to subspecialties that are represented by a small number of journals. The second issue is that these metrics do nothing to address the variation in citation habits across fields. These habits would evidently determine the citation metrics of journals in that field. The various standardized metrics that have been developed to render journals comparable across subjects (Table 4) could potentially overcome both shortcomings of citation-based metrics. In this study, the Radicchi index and the IFCM were equally able to normalize citation metrics across fields. The z score, however, was less effective for journals that were very highly or poorly cited. Regardless of the ability to normalize the citation performance, the implications of the Radicchi index and the z score are not readily understood. A Radicchi score of 1.38 (as for Neurosurgery; Table 3) would mean that articles published in Neurosurgery are 1.38 times as cited as the mean for neurosurgery as a field. A z score of 20.36 (Neurosurgery) would imply that the difference between mean citations for articles in Neurosurgery and the mean of the field was 0.36 times less than the mean for all other journals in the field (lower is better). In contrast, the IFCM is a simple metric that has a more readily understood implication. For

512 | VOLUME 76 | NUMBER 5 | MAY 2015

Suitable for Interfield Comparison? No No No No No No No Yes Possibly Yes Yes

example, a journal with IFCM = 20.5 (Journal of Neuro-oncology) is simply 50% less cited than the mean expected for that group. Therefore, the IFCM could be used to rank the performance of journals within a field and to populate subscription lists. Journals that have cross-field relevance require a special mention. Table 3 shows that the top 2 journals for neurosurgery are Stroke (IFCM = 0.86) and Spine (IFCM = 0.48). Stroke and Spine are both multifield journals relevant to several medical, surgical, and research groups. These multifield journals would be cited by authors from all those fields, obviously receiving higher journal metric scores.13 The highest-ranked pure neurosurgery journal was Neurosurgery (IFCM = 0.34).

CONCLUSION This study provides a baseline analysis of the citation climate in current neurosurgical literature. Current neurosurgical science appears to be based on data that are between 6.25 and 8 years old. Neurosurgery articles published in cross-specialty nonneurosurgical journals were much more highly cited than those published in neurosurgery journals. This is a set of articles that may escape the attention of the mainstream neurosurgical community. The IFCM was validated across several fields and could be a step toward achieving universal comparability of journal usefulness. With the use of the IFCM, the top-ranked pure neurosurgery journal was Neurosurgery. Disclosure The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article.

REFERENCES 1. Madhugiri VS, Ambekar S, Strom SF, Nanda A. A technique to identify core journals for neurosurgery using citation scatter analysis and the Bradford distribution across neurosurgery journals. J Neurosurg. 2013;119(5):1274-1287. 2. Greenberg SA. How citation distortions create unfounded authority: analysis of a citation network. BMJ. 2009;339:b2680.

www.neurosurgery-online.com

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

CITATION CLIMATE IN NEUROSURGERY

3. Van Noorden R, Maher B, Nuzzo R. The top 100 papers. Nature. 2014;514 (7524):550-553. 4. Force MM, Robinson NJ. Encouraging data citation and discovery with the data citation index. J Comput Aided Mol Des. 2014;28(10):1043-1048. 5. Khan NR, Thompson CJ, Taylor DR, et al. An analysis of publication productivity for 1,225 academic neurosurgeons and 99 departments in the United States. J Neurosurg. 2014;120(3):746-755. 6. Groneberg-Kloft B, Kreiter C, Welte T, Fischer A, Quarcoo D, Scutaru C. Interfield dysbalances in research input and output benchmarking: visualisation by density equalizing procedures. Int J Health Geogr. 2008;7:48. 7. Radicchi F, Fortunato S, Castellano C. Universality of citation distributions: toward an objective measure of scientific impact. Proc Natl Acad Sci USA. 2008; 105(45):17268-17272. 8. Bornmann L, Daniel HD. Universality of citation distributions: a validation of Radicchi et al.’s relative indicator cf = c/c0 at the micro level using data from chemistry. J Am Soc Inf Sci Technol. 2009;60(8):1664-1670. 9. NCBI. NLM Catalog: Journals Referenced in the NCBI Databases. 2009. Available at: http://www.ncbi.nlm.nih.gov/nlmcatalog/journals. Accessed June 19, 2014. 10. Wu XF, Fu Q, Rousseau R. On indexing in the web of science and predicting journal impact factor. J Zhejiang Univ Sci B. 2008;9(7):582-590. 11. Price DJ. Networks of scientific papers. Science. 1965;149(3683):510-515. 12. SCImago. SJR: SCImago Journal and Country Rank. 2007. Available at: http://www. scimagojr.com. Accessed June 19, 2014. 13. Stringer MJ, Sales-Pardo M, Nunes Amaral LA. Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal. J Am Soc Inf Sci Technol. 2010;61 (7):1377-1385. 14. Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011;13(4):e123. 15. Wren JD. URL decay in MEDLINE: a 4-year follow-up study. Bioinformatics. 2008;24(11):1381-1385. 16. Jorgensen HL, Praetorius L, Ingwersen P. Trends concerning medical articles 1989-1998: a study of Danish articles compared to other members of the European Union [in Danish]. Ugeskr Laeger. 1999;161(46):6339-6343. 17. Tenopir C, King DW, Bush A. Medical faculty’s use of print and electronic journals: changes over time and in comparison with scientists. J Med Libr Assoc. 2004;92(2):233-241. 18. Tenopir C, King DW, Clarke MT, Na K, Zhou X. Journal reading patterns and preferences of pediatricians. J Med Libr Assoc. 2007;95(1):56-63. 19. Lipsman N, Lozano AM. Measuring impact in stereotactic and functional neurosurgery: an analysis of the top 100 most highly cited works and the citation classics in the field. Stereotact Funct Neurosurg. 2012;90(3):201-209.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.neurosurgery-online.com).

NEUROSURGERY

Acknowledgment We would like to acknowledge Amrutha Bindu Nagella, MD, assistant professor of anesthesiology, Mahatma Gandhi Medical College and Research Institute, Pondicherry, India, for help with proofreading the manuscript.

COMMENTS

M

eta: A term, especially in art, used to characterize something that is characteristically self-referential. We have neurosurgical articles, published in (mainly) neurosurgical journals, and we have meta-articles, those that address the patterns of such publications. In this article, the authors analyze the citation patterns of neurosurgical papers and propose a new way to measure the significance of publications: the interfield citation metric. They found that articles are cited in increasing numbers after first appearing, peaking at about 6 years, with few citations after 12 years. The “fate” of articles is indicated by the number of citations that occur within a year or 2, although no statistical prediction regarding this could be made. Unsurprisingly, multispecialty journals were more impactful than those that are purely neurosurgical. It is useful and important to have tools to measure the publication impact of neurosurgical articles, given the relative smallness of our specialty in the larger world of medicine, let alone science. These metrics can influence research funding and faculty promotion. Periodically, it is good to assess these factors, as the authors have done. Still, the business of neurosurgical research and publication is to bring new knowledge to our community and the world. Let’s remember that and not get too concerned with examining our academic work in an infinity mirror. Michael Schulder Manhasset, New York

T

his article addresses the disappointment that authors of neurosurgical publications may experience. Although their publications are worthy of publication in our best journals, their value is not reflected in the metrics used to evaluate their significance in our field. Given the restricted circulation of our publications, our articles receive less exposure and hence less citations. The new metric, the interfield citation metric, may better reflect the significance of a publication within its own field and thereby validate the efforts of our academic colleagues. Maria Li Montreal, Quebec, Canada

VOLUME 76 | NUMBER 5 | MAY 2015 | 513

Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.

An analysis of the citation climate in neurosurgical literature and description of an interfield citation metric.

The citation climate in neurosurgical literature is largely undefined...
583KB Sizes 0 Downloads 8 Views