Archives of Environmental & Occupational Health (2015) 70, 110–115 C Taylor & Francis Group, LLC Copyright ISSN: 1933-8244 print / 2154-4700 online DOI: 10.1080/19338244.2013.845136
Scientific Research of Senior Italian Academics of Occupational Medicine: A Citation Analysis of Products Published During the Decade 2001–2010 GIULIANO FRANCO Department of Medical and Surgical Sciences, School of Medicine of Modena University of Modena and Reggio Emilia, Modena, Italy Received 21 September 2012, Accepted 25 August 2013
This article analyzes 10 years of scientific publications among senior Italian academics in occupational medicine by means of citation analysis. Articles published during the decade 2001–2010 were analyzed by means of Elsevier’s Scopus. Scientific performance was assessed by means of 9 different indices (including total number of papers, total citations, h-index). Most papers were submitted to journals of allergy and respiratory medicine, biochemistry, and toxicology. Only 11.9% of the 1,689 papers were published in journals of occupational medicine. The authors’ h-index was 10.1 (mean) and 9.5 (median) for the overall production. Productivity was associated with number of contributing authors. Most papers cover topics in the mainstream of other disciplines, evidencing that journals of occupational medicine do not play a primary role in the scientific panorama of medical sciences. This could imply consequences for the discipline. Keywords: academics, evaluation, citation analysis, impact factor, occupational medicine
The interest in assessing scientific production of universities increased in recent years. In fact, research evaluation is a key issue in benchmarking performance, orienting research policies, and making informed decisions about investments. Citation analysis is widely used to assess the quality of research as a part of recruitment decisions and academic promotion. Citation analysis consists in applying a series of indices of impact, influence, or quality derived from the references cited in scientific documents.1 Although citation analysis dates back to many years ago, in the last few years several indicators were developed because of the availability of databases such as the Thomson Reuters’ Web of Knowledge, Elsevier’s Scopus, and Google Scholar. The “impact factor” (IF) is the best-known measurement directly related to citations.2 The IF is related to the journal’s prestige, and it has been widely adopted as a proxy for journal quality. However, it is doubtful whether the journal IF is representative of the citation rate of individual articles.3 In addition, the IF shows limited benefits as an objective measure of individual performance.4 More recently, a new indicator (the h-index) for assessing scientific publications by their citations was introduced.5 The h-index reflects both a set of the most cited papers of an author and the number of citations they received: a scholar with an index of n published Address correspondence to G. Franco, Dipartimento di Scienze mediche e chirurgiche, Universit`a di Modena e Reggio Emilia, Largo del Pozzo, 71, 41100 Modena, Italy. E-mail: [email protected]
n papers, each of which was cited at least h times. Since h-index conveys information about quantitative productivity (papers number) and quality (citations number), it was suggested as a proxy index for assessing scientific publications of countries, universities, departments, scientific areas, and disciplines.6 Although widely used in different scientific fields, in recent years the h-index was applied in assessing scientific research in the medical field,7–10 and it was proposed as a threshold for academic appointments.11 In Italy, a new regulation was adopted with the aim of improving research productivity.12 It stated that a positive evaluation of scientific publications is required for accessing research funds, participating in the selection committee for recruitment and promotion, and obtaining economic benefits. In 2011, an authority (National Agency for the Evaluation of Universities and Research Institutes) responsible for scientific performance assessment was established. After much debate within the scientific and humanistic communities, the authority decided the criteria for assessing scientific publications and, as a part of the assessment process aiming at evaluating scientific performance of individual academics, it identified 3 metric indices: (i) overall number of publications, (ii) number of citations, (iii) h-index.13 This study aims at providing an analysis of the publications published during the decade 2001–2010 by senior academics of occupational medicine in Italy by employing the indices proposed by the authority responsible for assessing their scientific performance.
Scientific Research of Senior Italian Academics
111
Methods
Results
Population
Results are shown in Table 1, which reports the data of each academic ranked by the total number of publications. In the period 2001–2010 the Italian academics of occupational medicine published a total of 1,689 papers (mean 58.2, median 75.5 per academic). Papers were cited 11,150 times. Mean and median citations per author were 384.5 and 421, respectively. The h-index was 10.1 (mean) and 9.5 (median) for the overall 2001–2010 production and 7.3 (mean) and 7 (median) for the papers with the academic as first or last author. One hundred forty-one papers (11.9%) were published in core journals, ie, 1 paper every 9.1 papers. Most academics published both in core and in noncore journals. However, 8 professors out of 29 (including the subject with the higher number of citations) published no papers in core journals. The percentage of papers published in core journals ranged from 0 to 39% (5% among authors who published in core journals). All academics submitted at least 1 paper to journals not targeted to occupational medicine readers in the following categories of the Journal Citation Reports (JCR): Allergy, Biology, Biochemistry and Molecular Biology, Biochemistry Research Methods, Cardiac and Cardiovascular Systems, Chemistry, Medicine, Environmental Sciences, Food Science and Technology, Hematology, Oncology, Respiratory System, Toxicology, Transplantation. Papers published in core journals were slightly more cited (mean = 7.1; 95% confidence interval [CI] = 3.9–10.3) than papers published in noncore journals (mean = 5.3; 95% CI = 3.7–6.8). Overall productivity of the authors, expressed by the total number of documents, was associated with metric indices (citation number: r = .705, p < .001; h-index: r = .833, p < .001). Citation number and h-index were associated (r = .937, p < .001). A relationship was found also between the overall productivity and the number of papers published in core journals (r = .443, p = .015). The number of contributing authors was significantly associated with both productivity (expressed by the number of documents) (r = .371, p = .047) and citation number (r = .376, p = .044), but not with the h-index (r = .354, p = .059). The h-index related to the byline of authors as first or last author (h-index f/l) was associated with the overall production (r = .783, p = .001), the number of citations (r = .743, p = .001), the h-index (r = .843, p = .001), and the h-index core (r = .394, p = .001). No association was found between subject birthdate or year of academic appointment and any of the indices.
The population consisted in the 29 senior academics (full professors, professori ordinari) of occupational medicine whose names were included in the official database of MIUR (Ministry of Instruction, University and Research) on November 5, 2011 (http://www.istruzione.it/web/universita/docenti-ericercatori). Universities included 1 professor (Bari, Bologna, Brescia, Chieti, Florence, Foggia, Messina, Modena, Palermo, Parma, Trieste, Verona), 2 professors (Milan, Naples, Pavia, Perugia, Rome, Siena, Turin), or 3 professors (Padua). All academics were male, with a mean age of 61.4 years (range: 50–69). Birthdate was retrieved from the open access database of the College of Physicians (Ordine dei Medici). Elsevier’s Scopus was used as a database to search for information on scientific publications. The following procedure was followed to retrieve the publications: (i) each subject was identified on the basis of name and university in order to build a profile match. The profile match allowed discriminating between different individuals with the same name according to different pieces of information (affiliation, subject area, source title, citations, coauthors). In some cases, different profile matches were merged to obtain a unique profile for each subject; (ii) documents of each profile match were retrieved; (iii) documents of the period 2001–2010 were selected; (iv) documents were sorted according to the citation number; (v) citations and h-index of each profile match were automatically provided by the system; (vi) citations per paper and author mean and median per document were calculated for each academic; (vii) the h-index of each subject was calculated by selecting the documents with the author’s name in first or last position; (viii) the number of documents published in 7 journals of occupational medicine (American Journal of Industrial Medicine, Archives of Environmental & Occupational Health, International Archives of Occupational and Environmental Health, Journal of Occupational and Environmental Medicine, Occupational and Environmental Medicine, Occupational Medicine, Scandinavian Journal of Work, Environment and Health) was calculated. These journals were defined “core journals”14,15; and (ix) citations and h-index of papers published in core journals were calculated. The analysis was carried out in the period 2–8 May 2012. For each subject, the following indices were calculated: (i) number of papers; (ii) citations of papers published in the period 2001–2010; (iii) citations per paper; (iv) h-index of papers published in the period 2001–2010; (v) author mean and median; (vi) h-index of first-authored or last-authored papers (h-index f/l); (vii) number of documents published in core journals; (viii) citations of papers published in core journals (citation core); and (ix) h-index of papers published in core journals (h-index core). The measurement of the indices was based on the raw number of documents (including letters, editorials, commentaries, reviews). Neither the pertinence of the document to the discipline nor the target of hosting journals were considered. A free software package was used for descriptive statistical analysis (Wessa P. Free Statistics Software, Office for Research Development and Education, version 1.1.23-r7, 2012).
Comment Citation Analysis as A Proxy of Scientific Performance The progressive emphasis on the transparency and meritocracy in academia created a “culture of numbers.” Therefore, research institutions recognized that decisions on resources and appointments should rely on numerical information. Numerical information is derived from citation analysis by
112
Franco et al.
Table 1. Ten Years of Scientific Publications by Senior Italian Academics in Occupational Medicine (2001–2010) Documents
Citations
Authors
h-index
Subject
Total
Core
Total
Per document
Mean
Median
h-index
h-index f/l
h-index core
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Mean 95% CI Q1 Q2 Q3
150 139 128 91 90 90 80 77 69 66 61 54 50 49 47 46 46 45 43 41 41 41 39 27 25 24 15 14 1 58.2 45.1–71.4 41 75.5 69
10 7 7 6 5 20 2 5 0 26 8 16 5 3 1 3 0 3 3 5 1 3 0 0 0 0 2 0 0 4.9 2.6–7.1 0 5 5
841 835 1,472 683 971 508 299 398 1,563 381 520 227 244 177 101 178 96 421 113 263 147 127 276 3 151 40 82 32 1 384.5 236.1–532.9 113 421 421
5.6 6.0 11.5 7.5 10.8 5.6 3.7 5.2 22.7 5.8 8.5 4.2 4.9 3.6 2.1 3.9 2.1 9.4 2.6 6.4 3.6 3.1 7.1 0.1 6.0 1.7 5.5 2.3 1.0 5.6 4–7.2 3.1 3.3 6.4
7.4 7.1 8.2 7.5 9.5 7.8 9.3 8.2 8 5.3 7.5 5.1 6 9 5.2 7.4 6.4 4.6 7.6 6.9 7.4 6.3 6.2 5 2.6 6.8 5.1 5.8 8 6.8 6.2–7.4 5.8 7.7 7.8
7 7 8 9 8 7 9 8 7 4 7 2 6 8 5 7 6 3 7 5 7 6 6 5 2 6 6 6 8 6.3 5.6–6.9 6 7.5 7
18 16 26 15 21 14 11 12 21 13 12 9 11 8 4 8 5 12 7 9 9 4 9 1 6 4 4 3 1 10.1 7.8–12.4 5 9.5 12
13 10 19 13 8 12 7 7 14 10 9 9 6 5 3 7 3 10 5 7 7 4 7 1 6 2 3 3 1 7.3 5.7–8.8 4 7 9
5 3 7 4 2 9 2 4 0 9 5 8 3 3 0 3 0 3 3 2 1 2 0 0 0 0 2 0 0 2.8 1.8–3.8 0 2.5 3
Note. Subjects are ranked according to number of documents found on the database Scopus. The study includes the measurement of bibliometric indices (number of publications, number of citations, h-index, h-index first/last author, h-index of publications in core journals) established by the National Agency for Evaluation of University Research (ANVUR). Statistical data include mean, 95% confidence intervals, and quartiles.
means of simple and objective indices capturing the quality of publications.3 Although quantifying the quality of individual research is a complex task, there is an intuitive understanding that articles of high quality are highly cited and it is reasonably accepted that it is better to publish more than less and that the number of citations of a paper is related to its quality.16 Citation analysis for assessing the quality of scientific publications has a long history. However, several concerns were raised on its use in evaluating scientific productivity of researchers. Whereas the IF is the best-known citation-based index, other more recent indices were suggested and applied for ranking scientists. The most used is the h-index, defined as “an easily computable index, which gives an estimate of the importance, significance, and broad impact of a scientist’s cumulative research contributions.”5 This index was extensively used in scientific areas such as physics and chemistry,
where it shows a better capability of predicting future scientific achievement than indices such as publications number, total citation count, and citations per paper.17 In recent years, it was applied in different medical fields.8–11 This study provided a variety of findings derived from citation analysis. The productivity of senior academics was higher (mean documents per subject = 5.8 per year) than the already reported productivity of 40 academics in the period 2005–2008 (mean = 1.6 per year).18 However, the data of the 2 periods are not comparable because of the different source of information: the past 2005–2008 survey considered only journals listed in JCR, whereas the present study was based on Scopus, which includes a number of journals not included in JCR. The significant relationship between number of papers and h-index (r = .833) demonstrated that the overall productivity
Scientific Research of Senior Italian Academics is associated with the quality of publications. However, the hindex of Italian authors showed a great variability and its mean was lower in comparison with that of authors of other specialties.11 This observation is consistent with the observation that papers of other medical specialties show higher h-index.19 In fact, scientific papers of the category Public, Environmental and Occupational Health have less impact than papers of disciplines such as hematology, oncology, cardiology, and respiratory medicine.6 Finally, the current study showed that most indices derived from citation analysis are well associated with each other, suggesting that, in spite of some criticisms,20 they convey the same information. Therefore, they can be regarded as useful tools for assessing the scientific career of authors in the field of occupational medicine.
Teamwork and Multiple Authorship Because of its increasing complexity, research is a collaborative effort and teamwork is at the basis of knowledge advancement. Therefore, multiauthored papers are the norm. Hence, it is not surprising that scientific performance is associated with the number of researchers.21 An association between indices and number of authors was already observed in a previous survey, showing that numerous research groups are more productive, suggesting a relationship between human resources and publications.18 This study confirmed that the number of contributing authors is associated with overall productivity (number of documents and citation number) and is consistent with recent findings showing that productivity of occupational medicine structures is highly linked to the availability of human resources.22 However, no information was available on financial support needed to judge the efficiency of the research in terms of outcome.23 The upward trend in multiple-authorship makes it more and more difficult to assess the specific contribution of each author. In this respect, citation analysis shows some weaknesses. The author’s role remains unknown or uncertain if it is not clarified in some way. Therefore, if the authors’ role in multiauthored papers is not explicitly declared, their contribution might be overestimated. As expected, this study showed that most publications were multiauthored, with a mean number of authors ranging from 2.6 up to 9.5 per paper. This observation agrees with previous findings in the population of senior academics (including 11 individuals now retired), showing similar means (6.8 of this study vs 6.3 of the previous study) and an increase in median number of authors per paper (7.7 vs 6.3).18 These data are consistent with recent observations of an increasing number of authors in papers of journals of occupational medicine, a trend shared with other journals of medical and health science disciplines.24
Authorship Criteria, Seniority, and Individual Credit Authorship remains a critical issue in medical literature. Criteria for its definition and for the inclusion of researchers among the authors were well defined by the International Committee of Medical Journal Editors.25 However, inappropriate authorship involving honorary or gift authorship (authors not
113 meeting authorship criteria) was recently evidenced in 21% of papers published in major medical journals.26 This observation suggests that authorship criteria are not fully followed. Despite authorship appropriateness, it is uncertain how to apportion credit to each author of a paper. Coauthors usually do not equally contribute to a paper, and their position in the paper establishes credit as well as accountability. The authors in the first or last position are viewed as more creditworthy,27 whereas a large number of authors is thought to reduce the scientific credit of each participant, and the middle positions of a paper with a large number of authors make it difficult to appreciate individual contribution.28 This study showed that the h-index related to the byline of authors as first or last author (h-index f/l) was significantly associated with other indices (overall production, number of citations, h-index), suggesting that metric indices give the same information. These findings confirmed that h-index provides the right information about quantity and quality of papers within the cluster of academics, although it is difficult to extrapolate the contribution of each individual author unless it is clearly stated. It is usually recognized that senior academics are engaged mostly in administrative tasks, leaving a limited amount of time for research studies. In fact, it was observed that seniority (both age and career years) was negatively related to scientific production among academics of occupational medicine.21 However, the current study did not confirm these observations. In the present population of 29 subjects (a relatively younger population after the retirement of 11 members considered in the previous study), age was not associated with scientific production (r = .154). This might be explained with a persisting interest in research, although other factors cannot be excluded, such as gift authorship, mainly in papers with a large number of authors, especially when no information on the extent of the contribution is given. These results agree with other findings suggesting that productivity is not age related and that scholars of all ages can play a significant role in the process of teamwork research.29 Publishing in Occupational Medicine Journals The problem of choosing the right journal for publishing studies in the field of occupational medicine is not new.30 Occupational health and occupational medicine researchers are aware that journals of the discipline have a relatively small impact as compared with those of general medicine and other medical specialties.14 In fact, already in the past it was observed that occupational medicine journals are nearly 50 times more likely to cite the general medical literature than the converse.32 More recently, it was reported that only 0.48% of the articles published in The Lancet, New England Journal of Medicine, Journal of the American Medical Association, and British Medical Journal had occupational medicine as a main topic and that the discipline ranked 44th among 73 specialties.33 Although in recent years journals of occupational medicine benefited from a growing number of citations leading to an increased IF,31 this upward trend is similar for all medical disciplines. Therefore, it is not surprising that this study showed that no paper was published in the major journals of general and internal medicine, probably due to the limited importance or priority given to the topic by these journals. These reasons ac-
114 count for the preference of Italian academics to submit their papers to high-impact specialized journals, as suggested by our findings. In fact, Italian academics submitted a limited number (a percentage ranging from 0% to 39%) of papers to journals of occupational medicine (core journals), whereas most papers dealing with allergy and respiratory medicine, biochemistry and molecular biology, and toxicology were submitted to other journals. Our findings overlapped those published in 2009 showing that most Italian papers were published in journals not included in the Public, Environmental and Occupational Health category of the JCR.21 Authors likely believe that submitting their papers to specialized journals other than core could reach a more specialized target, could have more impact, and could deserve more citations from researchers of other medical areas. These choices could represent a challenge for most journals of the discipline aiming at attracting high-quality papers.34 Surprisingly, this study showed that the papers published in core journals were on average slightly more cited as compared with papers published in noncore journals (7.1 vs 5.3). However, this result is the consequence of the different journal coverage of the database used in this study. In fact, all core journals are included in the more selective JCR, whereas the coverage of Scopus includes, among a large number of JCR journals, also some journals of regional interest not included in JCR. These journals are of limited value for the international audience, and their papers rarely deserve citation. In fact, the overall h-index of noncore journals was higher than h-index of core journals (mean = 7.3 vs 2.8, median = 7 vs 2.5, respectively).
The Assessment of Research is a Multifaceted Process An effective system for research assessment should be based on different parameters, the first of which should be a proxy measure of scientific performance (publications, citations, metric indices).35 However, a second way of assessing scientific performance should be introduced beyond citation analysis through new measures of professional proficiency.36 In the field of occupational medicine, a variety of measures could be assessed and properly valued: (i) evidence of knowledge transfer, (ii) evidence of benefits of interventions, (iii) evidence of economic benefits, (iv) evidence of improved well-being of employees, (v) evidence of environmental and social improvement, and (vi) evidence of the overall quality of work. A final relevant aspect to consider is the financial performance expressed as the ratio of different outputs to outcome in terms of economics.36 An overall assessment of the scientific performance should take into account a variety of parameters (including originality of ideas and methods, availability of funds and staff, relevance to knowledge advancement, impact on social environment). Unfortunately, this study did not include information on professional activity and did not include financial information. In fact, funding information was available only for a very limited number of papers. In spite of these limits, the study showed that the assessment of research performance, even in a relatively homogeneous and small group of authors, is a complex task. There are several difficulties in detecting
Franco et al. the individual contribution of each author, and this would require a retrospective analysis to compare successful decisions of selecting committees for recruitment or academic promotion. Two points require attention: (i) authors need to better clarify their role in the research process, and (ii) methods and tools to assess creativity and innovation of research are needed. Conclusion This study found that there is a large difference in scientific performance in a homogeneous group of scholars. Whereas it is difficult to explain the reasons of metric differences, 2 aspects might have influenced the different scientific outcomes. The first aspect consists in the difficulty of comparing scientific activities with other, sometimes prevailing, commitments such as administrative responsibilities, teaching, and clinical duties. All these time-consuming activities are difficult to quantify so that much more credit is usually given to the objective scientific performance. This fact leads to an increasing difficulty in playing the double role of physician-scientist.37 The second aspect is related to the trend of research: the science-driven versus the society-driven research model. The former is mostly focused on developing and processing ideas on pathophysiology and mechanisms, the latter at applying knowledge for its practical use by occupational health practitioners. The different patterns of research are usually linked to individual choices and preferences but also to circumstances or even to chance. These aspects result in different bibliometrics: greater in science-driven research, lower in society-driven research. Different choices and different circumstances can therefore explain different metric outcomes. Although it was observed that researchers’ career cannot be synthesized by a number,38 citation data play a primary role in research evaluation and provide information about journals, papers, departments, and authors. Bibliometrics, however, should be properly used, cautiously interpreted, and represent only part of the evaluation process. A positive assessment of research outcomes may have multiple effects. For individual researchers, it undoubtedly has a positive effect on researchers’ career. For universities and departments, it represents a factor of success for obtaining human and financial resources. For occupational medicine, a positive assessment of its scholars can influence the trend and address the future of the discipline. This could make the difference between a thriving discipline and a languishing one.
References 1. Van Noorden R. Metrics: a profusion of measures. Nature. 2010;465:864–866. 2. Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295:90–93. 3. Adler R, Ewing J, Taylor P. Citation statistics. Stat Sci. 2009;24:1–14. 4. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314:498–502. 5. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102:16569–16572.
Scientific Research of Senior Italian Academics 6. Franco G. Research evaluation and competition for academic positions in occupational medicine. Arch Environ Occup Health. 2013;68:123–127. 7. Hunt GE, Cleary M, Walter G. Psychiatry and the Hirsch h-index: the relationship between journal impact factors and accrued citations. Harv Rev Psychiatry. 2010;18:207–219. 8. Benway BM, Kalidas P, Cabello JM, Bhayani SB. Does citation analysis reveal association between h-index and academic rank in urology? Urology. 2009;74:30–33. 9. Fuller CD, Choi M, Thomas CR Jr. Bibliometric analysis of radiation oncology departmental scholarly publication productivity at domestic residency training institutions. J Am Coll Radiol. 2009;6:112–118. 10. Lee J, Kraus KL, Couldwell WT. Use of the h index in neurosurgery. J Neurosurg. 2009;111:387–392. 11. Rad AE, Brinjikji W, Cloft HJ, et al. The H-index in academic radiology. Acad Radiol. 2010;17:817–821. 12. Legge 9 gennaio 2009, n. 1. Conversione in legge, con modificazioni, del decreto-legge 10 novembre 2008, n. 180, recante disposizioni urgenti per il diritto allo studio, la valorizzazione del merito e la qualit`a del sistema universitario e della ricerca. Gazzetta Ufficiale n. 6 del 9 gennaio 2009. 13. Agenzia Nazionale di Valutazione del sistema Universitario e della Ricerca (ANVUR). Criteri e parametri di valutazione dei candidati e dei commissari dell’abilitazione scientifica nazionale. Documento 1/2011. Available at: http://www. anvur.org/documenti.aspx?q=lista-documenti. Last accessed February 28, 2012. 14. Gehanno JF, Takahashi K, Darmoni S, et al. Citation classics in occupational medicine journals. Scand J Work Environ Health. 2007;33:245–251. 15. Smith DR. Identifying a set of ‘core’ journals in occupational health, part 2: lists derived by bibliometric techniques. Arch Environ Occup Health. 2010;65:173–175. 16. Lehmann S, Jackson AD, Lautrup BE. Measures for measures. Nature. 2006;444:1003–1004. 17. Hirsch JE. Does the h index have predictive power? Proc Natl Acad Sci U S A. 2007;104:19193–19198. 18. Franco G. Publish or perish: the scientific productivity of academics in the field of occupational medicine [Italian]. Med Lav. 2009;100:163–170. 19. Opthof T, Wilde AA. The Hirsch-index: a simple, new tool for the assessment of scientific output of individual scientists: the case of Dutch professors in clinical cardiology. Neth Heart J. 2009;17:145–154. 20. Thompson DF, Callen EC, Nahata MC. New indices in scholarship assessment. Am J Pharm Educ. 2009;73:111. 21. Itagaki MW, Pile-Spellman J. Factors associated with academic radiology research productivity. Radiology. 2005;237: 774–780.
115 22. Franco G. Scientific performance of occupational medicine research in the Italian Universities in the period 2001–2010 [Italian]. Med Lav. 2012;103:72–74. 23. Hendrix D. An analysis of bibliometric indicators, National Institutes of Health funding, and faculty size at Association of American Medical Colleges medical schools, 1997–2007. J Med Libr Assoc. 2008;96:324–334. 24. Shaban S, Aw TC. Trend towards multiple authorship in occupational medicine journals. J Occup Med Toxicol. 2009;4:3. 25. International Committee of Medical Journal Editors (ICMJE). Uniform requirements for manuscripts submitted to biomedical journals: ethical considerations in the conduct and reporting of research. Available at: http://www.icmje.org/ethical 1author.html. Published 2009. Last accessed July 10, 2012. 26. Wislar JS, Flanagin A, Fontanarosa PB, et al. Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey. BMJ. 2011;343:d6128. 27. Tscharntke T, Hochberg ME, Rand TA, et al. Author sequence and credit for contributions in multiauthored publications. PLoS Biol. 2007;5:e18. 28. Wren JD, Kozak KZ, Johnson KR, et al. The write position. A survey of perceived contributions to papers based on byline position and number of authors. EMBO Rep. 2007;8:988–991. 29. Gingras Y, Larivi´ere V, Macaluso B, et al. The effects of aging on researchers’ publication and citation patterns. PLoS ONE. 2008;3:e4048. 30. Viikari-Juntura E, Burdorf A. Focus and future of occupational health journals. Scand J Work Environ Health. 2007;33:241–243. 31. Smith DR. Citation analysis and impact factor trends of 5 core journals in occupational medicine, 1985–2006. Arch Environ Occup Health. 2008;63:114–122. 32. McCunney RJ, Harzbecker J. The influence of occupational medicine on general medicine: a look at the journals. J Occup Med 1992;34:279–286. 33. Gehanno JF, Rollin L, Ladner J, et al. How is occupational medicine represented in the major journals in general medicine? Occup Environ Med. 2012;69:603–605. 34. Smith DR. Highly cited articles in environmental and occupational health, 1919–1960. Arch Environ Occup Health. 2009;64(Suppl 1):32–42. 35. Lane J, Bertuzzi L. Measuring the results of science investments Science. 2011:331:678–680. 36. Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med Libr Assoc. 2010;98:17–23. 37. Zemlo TR, Garrison HH, Partridge NC, et al. The physicianscientist: career issues and challenges at the year 2000. FASEB J. 2000;14:221–230. 38. Abbott A, Cyranoski D, Jones N, et al. Do metrics matter? Nature. 2010;465:860–862.
Copyright of Archives of Environmental & Occupational Health is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Scientific Research of Senior Italian Academics of Occupational Medicine: A Citation Analysis of Products Published During the Decade 2001–2010 GIULIANO FRANCO Department of Medical and Surgical Sciences, School of Medicine of Modena University of Modena and Reggio Emilia, Modena, Italy Received 21 September 2012, Accepted 25 August 2013
This article analyzes 10 years of scientific publications among senior Italian academics in occupational medicine by means of citation analysis. Articles published during the decade 2001–2010 were analyzed by means of Elsevier’s Scopus. Scientific performance was assessed by means of 9 different indices (including total number of papers, total citations, h-index). Most papers were submitted to journals of allergy and respiratory medicine, biochemistry, and toxicology. Only 11.9% of the 1,689 papers were published in journals of occupational medicine. The authors’ h-index was 10.1 (mean) and 9.5 (median) for the overall production. Productivity was associated with number of contributing authors. Most papers cover topics in the mainstream of other disciplines, evidencing that journals of occupational medicine do not play a primary role in the scientific panorama of medical sciences. This could imply consequences for the discipline. Keywords: academics, evaluation, citation analysis, impact factor, occupational medicine
The interest in assessing scientific production of universities increased in recent years. In fact, research evaluation is a key issue in benchmarking performance, orienting research policies, and making informed decisions about investments. Citation analysis is widely used to assess the quality of research as a part of recruitment decisions and academic promotion. Citation analysis consists in applying a series of indices of impact, influence, or quality derived from the references cited in scientific documents.1 Although citation analysis dates back to many years ago, in the last few years several indicators were developed because of the availability of databases such as the Thomson Reuters’ Web of Knowledge, Elsevier’s Scopus, and Google Scholar. The “impact factor” (IF) is the best-known measurement directly related to citations.2 The IF is related to the journal’s prestige, and it has been widely adopted as a proxy for journal quality. However, it is doubtful whether the journal IF is representative of the citation rate of individual articles.3 In addition, the IF shows limited benefits as an objective measure of individual performance.4 More recently, a new indicator (the h-index) for assessing scientific publications by their citations was introduced.5 The h-index reflects both a set of the most cited papers of an author and the number of citations they received: a scholar with an index of n published Address correspondence to G. Franco, Dipartimento di Scienze mediche e chirurgiche, Universit`a di Modena e Reggio Emilia, Largo del Pozzo, 71, 41100 Modena, Italy. E-mail: [email protected]
n papers, each of which was cited at least h times. Since h-index conveys information about quantitative productivity (papers number) and quality (citations number), it was suggested as a proxy index for assessing scientific publications of countries, universities, departments, scientific areas, and disciplines.6 Although widely used in different scientific fields, in recent years the h-index was applied in assessing scientific research in the medical field,7–10 and it was proposed as a threshold for academic appointments.11 In Italy, a new regulation was adopted with the aim of improving research productivity.12 It stated that a positive evaluation of scientific publications is required for accessing research funds, participating in the selection committee for recruitment and promotion, and obtaining economic benefits. In 2011, an authority (National Agency for the Evaluation of Universities and Research Institutes) responsible for scientific performance assessment was established. After much debate within the scientific and humanistic communities, the authority decided the criteria for assessing scientific publications and, as a part of the assessment process aiming at evaluating scientific performance of individual academics, it identified 3 metric indices: (i) overall number of publications, (ii) number of citations, (iii) h-index.13 This study aims at providing an analysis of the publications published during the decade 2001–2010 by senior academics of occupational medicine in Italy by employing the indices proposed by the authority responsible for assessing their scientific performance.
Scientific Research of Senior Italian Academics
111
Methods
Results
Population
Results are shown in Table 1, which reports the data of each academic ranked by the total number of publications. In the period 2001–2010 the Italian academics of occupational medicine published a total of 1,689 papers (mean 58.2, median 75.5 per academic). Papers were cited 11,150 times. Mean and median citations per author were 384.5 and 421, respectively. The h-index was 10.1 (mean) and 9.5 (median) for the overall 2001–2010 production and 7.3 (mean) and 7 (median) for the papers with the academic as first or last author. One hundred forty-one papers (11.9%) were published in core journals, ie, 1 paper every 9.1 papers. Most academics published both in core and in noncore journals. However, 8 professors out of 29 (including the subject with the higher number of citations) published no papers in core journals. The percentage of papers published in core journals ranged from 0 to 39% (5% among authors who published in core journals). All academics submitted at least 1 paper to journals not targeted to occupational medicine readers in the following categories of the Journal Citation Reports (JCR): Allergy, Biology, Biochemistry and Molecular Biology, Biochemistry Research Methods, Cardiac and Cardiovascular Systems, Chemistry, Medicine, Environmental Sciences, Food Science and Technology, Hematology, Oncology, Respiratory System, Toxicology, Transplantation. Papers published in core journals were slightly more cited (mean = 7.1; 95% confidence interval [CI] = 3.9–10.3) than papers published in noncore journals (mean = 5.3; 95% CI = 3.7–6.8). Overall productivity of the authors, expressed by the total number of documents, was associated with metric indices (citation number: r = .705, p < .001; h-index: r = .833, p < .001). Citation number and h-index were associated (r = .937, p < .001). A relationship was found also between the overall productivity and the number of papers published in core journals (r = .443, p = .015). The number of contributing authors was significantly associated with both productivity (expressed by the number of documents) (r = .371, p = .047) and citation number (r = .376, p = .044), but not with the h-index (r = .354, p = .059). The h-index related to the byline of authors as first or last author (h-index f/l) was associated with the overall production (r = .783, p = .001), the number of citations (r = .743, p = .001), the h-index (r = .843, p = .001), and the h-index core (r = .394, p = .001). No association was found between subject birthdate or year of academic appointment and any of the indices.
The population consisted in the 29 senior academics (full professors, professori ordinari) of occupational medicine whose names were included in the official database of MIUR (Ministry of Instruction, University and Research) on November 5, 2011 (http://www.istruzione.it/web/universita/docenti-ericercatori). Universities included 1 professor (Bari, Bologna, Brescia, Chieti, Florence, Foggia, Messina, Modena, Palermo, Parma, Trieste, Verona), 2 professors (Milan, Naples, Pavia, Perugia, Rome, Siena, Turin), or 3 professors (Padua). All academics were male, with a mean age of 61.4 years (range: 50–69). Birthdate was retrieved from the open access database of the College of Physicians (Ordine dei Medici). Elsevier’s Scopus was used as a database to search for information on scientific publications. The following procedure was followed to retrieve the publications: (i) each subject was identified on the basis of name and university in order to build a profile match. The profile match allowed discriminating between different individuals with the same name according to different pieces of information (affiliation, subject area, source title, citations, coauthors). In some cases, different profile matches were merged to obtain a unique profile for each subject; (ii) documents of each profile match were retrieved; (iii) documents of the period 2001–2010 were selected; (iv) documents were sorted according to the citation number; (v) citations and h-index of each profile match were automatically provided by the system; (vi) citations per paper and author mean and median per document were calculated for each academic; (vii) the h-index of each subject was calculated by selecting the documents with the author’s name in first or last position; (viii) the number of documents published in 7 journals of occupational medicine (American Journal of Industrial Medicine, Archives of Environmental & Occupational Health, International Archives of Occupational and Environmental Health, Journal of Occupational and Environmental Medicine, Occupational and Environmental Medicine, Occupational Medicine, Scandinavian Journal of Work, Environment and Health) was calculated. These journals were defined “core journals”14,15; and (ix) citations and h-index of papers published in core journals were calculated. The analysis was carried out in the period 2–8 May 2012. For each subject, the following indices were calculated: (i) number of papers; (ii) citations of papers published in the period 2001–2010; (iii) citations per paper; (iv) h-index of papers published in the period 2001–2010; (v) author mean and median; (vi) h-index of first-authored or last-authored papers (h-index f/l); (vii) number of documents published in core journals; (viii) citations of papers published in core journals (citation core); and (ix) h-index of papers published in core journals (h-index core). The measurement of the indices was based on the raw number of documents (including letters, editorials, commentaries, reviews). Neither the pertinence of the document to the discipline nor the target of hosting journals were considered. A free software package was used for descriptive statistical analysis (Wessa P. Free Statistics Software, Office for Research Development and Education, version 1.1.23-r7, 2012).
Comment Citation Analysis as A Proxy of Scientific Performance The progressive emphasis on the transparency and meritocracy in academia created a “culture of numbers.” Therefore, research institutions recognized that decisions on resources and appointments should rely on numerical information. Numerical information is derived from citation analysis by
112
Franco et al.
Table 1. Ten Years of Scientific Publications by Senior Italian Academics in Occupational Medicine (2001–2010) Documents
Citations
Authors
h-index
Subject
Total
Core
Total
Per document
Mean
Median
h-index
h-index f/l
h-index core
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Mean 95% CI Q1 Q2 Q3
150 139 128 91 90 90 80 77 69 66 61 54 50 49 47 46 46 45 43 41 41 41 39 27 25 24 15 14 1 58.2 45.1–71.4 41 75.5 69
10 7 7 6 5 20 2 5 0 26 8 16 5 3 1 3 0 3 3 5 1 3 0 0 0 0 2 0 0 4.9 2.6–7.1 0 5 5
841 835 1,472 683 971 508 299 398 1,563 381 520 227 244 177 101 178 96 421 113 263 147 127 276 3 151 40 82 32 1 384.5 236.1–532.9 113 421 421
5.6 6.0 11.5 7.5 10.8 5.6 3.7 5.2 22.7 5.8 8.5 4.2 4.9 3.6 2.1 3.9 2.1 9.4 2.6 6.4 3.6 3.1 7.1 0.1 6.0 1.7 5.5 2.3 1.0 5.6 4–7.2 3.1 3.3 6.4
7.4 7.1 8.2 7.5 9.5 7.8 9.3 8.2 8 5.3 7.5 5.1 6 9 5.2 7.4 6.4 4.6 7.6 6.9 7.4 6.3 6.2 5 2.6 6.8 5.1 5.8 8 6.8 6.2–7.4 5.8 7.7 7.8
7 7 8 9 8 7 9 8 7 4 7 2 6 8 5 7 6 3 7 5 7 6 6 5 2 6 6 6 8 6.3 5.6–6.9 6 7.5 7
18 16 26 15 21 14 11 12 21 13 12 9 11 8 4 8 5 12 7 9 9 4 9 1 6 4 4 3 1 10.1 7.8–12.4 5 9.5 12
13 10 19 13 8 12 7 7 14 10 9 9 6 5 3 7 3 10 5 7 7 4 7 1 6 2 3 3 1 7.3 5.7–8.8 4 7 9
5 3 7 4 2 9 2 4 0 9 5 8 3 3 0 3 0 3 3 2 1 2 0 0 0 0 2 0 0 2.8 1.8–3.8 0 2.5 3
Note. Subjects are ranked according to number of documents found on the database Scopus. The study includes the measurement of bibliometric indices (number of publications, number of citations, h-index, h-index first/last author, h-index of publications in core journals) established by the National Agency for Evaluation of University Research (ANVUR). Statistical data include mean, 95% confidence intervals, and quartiles.
means of simple and objective indices capturing the quality of publications.3 Although quantifying the quality of individual research is a complex task, there is an intuitive understanding that articles of high quality are highly cited and it is reasonably accepted that it is better to publish more than less and that the number of citations of a paper is related to its quality.16 Citation analysis for assessing the quality of scientific publications has a long history. However, several concerns were raised on its use in evaluating scientific productivity of researchers. Whereas the IF is the best-known citation-based index, other more recent indices were suggested and applied for ranking scientists. The most used is the h-index, defined as “an easily computable index, which gives an estimate of the importance, significance, and broad impact of a scientist’s cumulative research contributions.”5 This index was extensively used in scientific areas such as physics and chemistry,
where it shows a better capability of predicting future scientific achievement than indices such as publications number, total citation count, and citations per paper.17 In recent years, it was applied in different medical fields.8–11 This study provided a variety of findings derived from citation analysis. The productivity of senior academics was higher (mean documents per subject = 5.8 per year) than the already reported productivity of 40 academics in the period 2005–2008 (mean = 1.6 per year).18 However, the data of the 2 periods are not comparable because of the different source of information: the past 2005–2008 survey considered only journals listed in JCR, whereas the present study was based on Scopus, which includes a number of journals not included in JCR. The significant relationship between number of papers and h-index (r = .833) demonstrated that the overall productivity
Scientific Research of Senior Italian Academics is associated with the quality of publications. However, the hindex of Italian authors showed a great variability and its mean was lower in comparison with that of authors of other specialties.11 This observation is consistent with the observation that papers of other medical specialties show higher h-index.19 In fact, scientific papers of the category Public, Environmental and Occupational Health have less impact than papers of disciplines such as hematology, oncology, cardiology, and respiratory medicine.6 Finally, the current study showed that most indices derived from citation analysis are well associated with each other, suggesting that, in spite of some criticisms,20 they convey the same information. Therefore, they can be regarded as useful tools for assessing the scientific career of authors in the field of occupational medicine.
Teamwork and Multiple Authorship Because of its increasing complexity, research is a collaborative effort and teamwork is at the basis of knowledge advancement. Therefore, multiauthored papers are the norm. Hence, it is not surprising that scientific performance is associated with the number of researchers.21 An association between indices and number of authors was already observed in a previous survey, showing that numerous research groups are more productive, suggesting a relationship between human resources and publications.18 This study confirmed that the number of contributing authors is associated with overall productivity (number of documents and citation number) and is consistent with recent findings showing that productivity of occupational medicine structures is highly linked to the availability of human resources.22 However, no information was available on financial support needed to judge the efficiency of the research in terms of outcome.23 The upward trend in multiple-authorship makes it more and more difficult to assess the specific contribution of each author. In this respect, citation analysis shows some weaknesses. The author’s role remains unknown or uncertain if it is not clarified in some way. Therefore, if the authors’ role in multiauthored papers is not explicitly declared, their contribution might be overestimated. As expected, this study showed that most publications were multiauthored, with a mean number of authors ranging from 2.6 up to 9.5 per paper. This observation agrees with previous findings in the population of senior academics (including 11 individuals now retired), showing similar means (6.8 of this study vs 6.3 of the previous study) and an increase in median number of authors per paper (7.7 vs 6.3).18 These data are consistent with recent observations of an increasing number of authors in papers of journals of occupational medicine, a trend shared with other journals of medical and health science disciplines.24
Authorship Criteria, Seniority, and Individual Credit Authorship remains a critical issue in medical literature. Criteria for its definition and for the inclusion of researchers among the authors were well defined by the International Committee of Medical Journal Editors.25 However, inappropriate authorship involving honorary or gift authorship (authors not
113 meeting authorship criteria) was recently evidenced in 21% of papers published in major medical journals.26 This observation suggests that authorship criteria are not fully followed. Despite authorship appropriateness, it is uncertain how to apportion credit to each author of a paper. Coauthors usually do not equally contribute to a paper, and their position in the paper establishes credit as well as accountability. The authors in the first or last position are viewed as more creditworthy,27 whereas a large number of authors is thought to reduce the scientific credit of each participant, and the middle positions of a paper with a large number of authors make it difficult to appreciate individual contribution.28 This study showed that the h-index related to the byline of authors as first or last author (h-index f/l) was significantly associated with other indices (overall production, number of citations, h-index), suggesting that metric indices give the same information. These findings confirmed that h-index provides the right information about quantity and quality of papers within the cluster of academics, although it is difficult to extrapolate the contribution of each individual author unless it is clearly stated. It is usually recognized that senior academics are engaged mostly in administrative tasks, leaving a limited amount of time for research studies. In fact, it was observed that seniority (both age and career years) was negatively related to scientific production among academics of occupational medicine.21 However, the current study did not confirm these observations. In the present population of 29 subjects (a relatively younger population after the retirement of 11 members considered in the previous study), age was not associated with scientific production (r = .154). This might be explained with a persisting interest in research, although other factors cannot be excluded, such as gift authorship, mainly in papers with a large number of authors, especially when no information on the extent of the contribution is given. These results agree with other findings suggesting that productivity is not age related and that scholars of all ages can play a significant role in the process of teamwork research.29 Publishing in Occupational Medicine Journals The problem of choosing the right journal for publishing studies in the field of occupational medicine is not new.30 Occupational health and occupational medicine researchers are aware that journals of the discipline have a relatively small impact as compared with those of general medicine and other medical specialties.14 In fact, already in the past it was observed that occupational medicine journals are nearly 50 times more likely to cite the general medical literature than the converse.32 More recently, it was reported that only 0.48% of the articles published in The Lancet, New England Journal of Medicine, Journal of the American Medical Association, and British Medical Journal had occupational medicine as a main topic and that the discipline ranked 44th among 73 specialties.33 Although in recent years journals of occupational medicine benefited from a growing number of citations leading to an increased IF,31 this upward trend is similar for all medical disciplines. Therefore, it is not surprising that this study showed that no paper was published in the major journals of general and internal medicine, probably due to the limited importance or priority given to the topic by these journals. These reasons ac-
114 count for the preference of Italian academics to submit their papers to high-impact specialized journals, as suggested by our findings. In fact, Italian academics submitted a limited number (a percentage ranging from 0% to 39%) of papers to journals of occupational medicine (core journals), whereas most papers dealing with allergy and respiratory medicine, biochemistry and molecular biology, and toxicology were submitted to other journals. Our findings overlapped those published in 2009 showing that most Italian papers were published in journals not included in the Public, Environmental and Occupational Health category of the JCR.21 Authors likely believe that submitting their papers to specialized journals other than core could reach a more specialized target, could have more impact, and could deserve more citations from researchers of other medical areas. These choices could represent a challenge for most journals of the discipline aiming at attracting high-quality papers.34 Surprisingly, this study showed that the papers published in core journals were on average slightly more cited as compared with papers published in noncore journals (7.1 vs 5.3). However, this result is the consequence of the different journal coverage of the database used in this study. In fact, all core journals are included in the more selective JCR, whereas the coverage of Scopus includes, among a large number of JCR journals, also some journals of regional interest not included in JCR. These journals are of limited value for the international audience, and their papers rarely deserve citation. In fact, the overall h-index of noncore journals was higher than h-index of core journals (mean = 7.3 vs 2.8, median = 7 vs 2.5, respectively).
The Assessment of Research is a Multifaceted Process An effective system for research assessment should be based on different parameters, the first of which should be a proxy measure of scientific performance (publications, citations, metric indices).35 However, a second way of assessing scientific performance should be introduced beyond citation analysis through new measures of professional proficiency.36 In the field of occupational medicine, a variety of measures could be assessed and properly valued: (i) evidence of knowledge transfer, (ii) evidence of benefits of interventions, (iii) evidence of economic benefits, (iv) evidence of improved well-being of employees, (v) evidence of environmental and social improvement, and (vi) evidence of the overall quality of work. A final relevant aspect to consider is the financial performance expressed as the ratio of different outputs to outcome in terms of economics.36 An overall assessment of the scientific performance should take into account a variety of parameters (including originality of ideas and methods, availability of funds and staff, relevance to knowledge advancement, impact on social environment). Unfortunately, this study did not include information on professional activity and did not include financial information. In fact, funding information was available only for a very limited number of papers. In spite of these limits, the study showed that the assessment of research performance, even in a relatively homogeneous and small group of authors, is a complex task. There are several difficulties in detecting
Franco et al. the individual contribution of each author, and this would require a retrospective analysis to compare successful decisions of selecting committees for recruitment or academic promotion. Two points require attention: (i) authors need to better clarify their role in the research process, and (ii) methods and tools to assess creativity and innovation of research are needed. Conclusion This study found that there is a large difference in scientific performance in a homogeneous group of scholars. Whereas it is difficult to explain the reasons of metric differences, 2 aspects might have influenced the different scientific outcomes. The first aspect consists in the difficulty of comparing scientific activities with other, sometimes prevailing, commitments such as administrative responsibilities, teaching, and clinical duties. All these time-consuming activities are difficult to quantify so that much more credit is usually given to the objective scientific performance. This fact leads to an increasing difficulty in playing the double role of physician-scientist.37 The second aspect is related to the trend of research: the science-driven versus the society-driven research model. The former is mostly focused on developing and processing ideas on pathophysiology and mechanisms, the latter at applying knowledge for its practical use by occupational health practitioners. The different patterns of research are usually linked to individual choices and preferences but also to circumstances or even to chance. These aspects result in different bibliometrics: greater in science-driven research, lower in society-driven research. Different choices and different circumstances can therefore explain different metric outcomes. Although it was observed that researchers’ career cannot be synthesized by a number,38 citation data play a primary role in research evaluation and provide information about journals, papers, departments, and authors. Bibliometrics, however, should be properly used, cautiously interpreted, and represent only part of the evaluation process. A positive assessment of research outcomes may have multiple effects. For individual researchers, it undoubtedly has a positive effect on researchers’ career. For universities and departments, it represents a factor of success for obtaining human and financial resources. For occupational medicine, a positive assessment of its scholars can influence the trend and address the future of the discipline. This could make the difference between a thriving discipline and a languishing one.
References 1. Van Noorden R. Metrics: a profusion of measures. Nature. 2010;465:864–866. 2. Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295:90–93. 3. Adler R, Ewing J, Taylor P. Citation statistics. Stat Sci. 2009;24:1–14. 4. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314:498–502. 5. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102:16569–16572.
Scientific Research of Senior Italian Academics 6. Franco G. Research evaluation and competition for academic positions in occupational medicine. Arch Environ Occup Health. 2013;68:123–127. 7. Hunt GE, Cleary M, Walter G. Psychiatry and the Hirsch h-index: the relationship between journal impact factors and accrued citations. Harv Rev Psychiatry. 2010;18:207–219. 8. Benway BM, Kalidas P, Cabello JM, Bhayani SB. Does citation analysis reveal association between h-index and academic rank in urology? Urology. 2009;74:30–33. 9. Fuller CD, Choi M, Thomas CR Jr. Bibliometric analysis of radiation oncology departmental scholarly publication productivity at domestic residency training institutions. J Am Coll Radiol. 2009;6:112–118. 10. Lee J, Kraus KL, Couldwell WT. Use of the h index in neurosurgery. J Neurosurg. 2009;111:387–392. 11. Rad AE, Brinjikji W, Cloft HJ, et al. The H-index in academic radiology. Acad Radiol. 2010;17:817–821. 12. Legge 9 gennaio 2009, n. 1. Conversione in legge, con modificazioni, del decreto-legge 10 novembre 2008, n. 180, recante disposizioni urgenti per il diritto allo studio, la valorizzazione del merito e la qualit`a del sistema universitario e della ricerca. Gazzetta Ufficiale n. 6 del 9 gennaio 2009. 13. Agenzia Nazionale di Valutazione del sistema Universitario e della Ricerca (ANVUR). Criteri e parametri di valutazione dei candidati e dei commissari dell’abilitazione scientifica nazionale. Documento 1/2011. Available at: http://www. anvur.org/documenti.aspx?q=lista-documenti. Last accessed February 28, 2012. 14. Gehanno JF, Takahashi K, Darmoni S, et al. Citation classics in occupational medicine journals. Scand J Work Environ Health. 2007;33:245–251. 15. Smith DR. Identifying a set of ‘core’ journals in occupational health, part 2: lists derived by bibliometric techniques. Arch Environ Occup Health. 2010;65:173–175. 16. Lehmann S, Jackson AD, Lautrup BE. Measures for measures. Nature. 2006;444:1003–1004. 17. Hirsch JE. Does the h index have predictive power? Proc Natl Acad Sci U S A. 2007;104:19193–19198. 18. Franco G. Publish or perish: the scientific productivity of academics in the field of occupational medicine [Italian]. Med Lav. 2009;100:163–170. 19. Opthof T, Wilde AA. The Hirsch-index: a simple, new tool for the assessment of scientific output of individual scientists: the case of Dutch professors in clinical cardiology. Neth Heart J. 2009;17:145–154. 20. Thompson DF, Callen EC, Nahata MC. New indices in scholarship assessment. Am J Pharm Educ. 2009;73:111. 21. Itagaki MW, Pile-Spellman J. Factors associated with academic radiology research productivity. Radiology. 2005;237: 774–780.
115 22. Franco G. Scientific performance of occupational medicine research in the Italian Universities in the period 2001–2010 [Italian]. Med Lav. 2012;103:72–74. 23. Hendrix D. An analysis of bibliometric indicators, National Institutes of Health funding, and faculty size at Association of American Medical Colleges medical schools, 1997–2007. J Med Libr Assoc. 2008;96:324–334. 24. Shaban S, Aw TC. Trend towards multiple authorship in occupational medicine journals. J Occup Med Toxicol. 2009;4:3. 25. International Committee of Medical Journal Editors (ICMJE). Uniform requirements for manuscripts submitted to biomedical journals: ethical considerations in the conduct and reporting of research. Available at: http://www.icmje.org/ethical 1author.html. Published 2009. Last accessed July 10, 2012. 26. Wislar JS, Flanagin A, Fontanarosa PB, et al. Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey. BMJ. 2011;343:d6128. 27. Tscharntke T, Hochberg ME, Rand TA, et al. Author sequence and credit for contributions in multiauthored publications. PLoS Biol. 2007;5:e18. 28. Wren JD, Kozak KZ, Johnson KR, et al. The write position. A survey of perceived contributions to papers based on byline position and number of authors. EMBO Rep. 2007;8:988–991. 29. Gingras Y, Larivi´ere V, Macaluso B, et al. The effects of aging on researchers’ publication and citation patterns. PLoS ONE. 2008;3:e4048. 30. Viikari-Juntura E, Burdorf A. Focus and future of occupational health journals. Scand J Work Environ Health. 2007;33:241–243. 31. Smith DR. Citation analysis and impact factor trends of 5 core journals in occupational medicine, 1985–2006. Arch Environ Occup Health. 2008;63:114–122. 32. McCunney RJ, Harzbecker J. The influence of occupational medicine on general medicine: a look at the journals. J Occup Med 1992;34:279–286. 33. Gehanno JF, Rollin L, Ladner J, et al. How is occupational medicine represented in the major journals in general medicine? Occup Environ Med. 2012;69:603–605. 34. Smith DR. Highly cited articles in environmental and occupational health, 1919–1960. Arch Environ Occup Health. 2009;64(Suppl 1):32–42. 35. Lane J, Bertuzzi L. Measuring the results of science investments Science. 2011:331:678–680. 36. Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med Libr Assoc. 2010;98:17–23. 37. Zemlo TR, Garrison HH, Partridge NC, et al. The physicianscientist: career issues and challenges at the year 2000. FASEB J. 2000;14:221–230. 38. Abbott A, Cyranoski D, Jones N, et al. Do metrics matter? Nature. 2010;465:860–862.
Copyright of Archives of Environmental & Occupational Health is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
