Archives of Environmental & Occupational Health (2015) 70, 117–119 C Taylor & Francis Group, LLC Copyright ISSN: 1933-8244 print / 2154-4700 online DOI: 10.1080/19338244.2015.1042256
Emerging Topics in EOH Research
Assessing and Developing Early Careers in Academic Research: How Can We Improve the Current Systems? Fundamental to the success of any organization is the human capital upon which it is built. Consideration of the human aspects of research careers is critical therefore, if one utilizes quantitative measures for performance assessment in academic and scientific institutions. With this in mind, I recently proposed a refined version of the H-Index to help account for differences in career length versus the time needed to accrue citations in scientific research.1 This citation phase lag, up to 10 years or more in Environmental and Occupational Health (EOH),2 disadvantages those early in their careers if performance assessment is undertaken using raw metrics such as the number of publications, citation counts and the HIndex. There is also the relative disadvantage of publishing in a field that is not in itself, very highly cited. A previous study of over 15,000 EOH articles for example, revealed that only 85 studies had been cited more than 100 times each.3 To make matters worse, EOH as a discipline often gives these precious citations away to other fields. An early study of the occupational medicine literature, for example, found that journals in our field were 50 times more likely to cite the general medical literature than vice versa.4 A more recent investigation found that only 12% of EOH-related articles had actually published in the journals of occupational medicine.5 To help more fairly assess the published work of Early Career Researchers (ECRs), therefore, I suggested that we adjust the widely-used H-Index score for career length (as originally proposed by Hirsch, but not included in his final calculation)6 and multiply this by an author’s overall citation density.1 This represents one potential strategy arising from the increasingly common practice of examining and refining the main bibliometric components on which these metrics are based: namely, individual publications and article citations. Given that the H-Index is known to be more highly correlated with an individual’s total citations rather than their total number of publications,7 analyzing citation counts would appear to be a fairly promising start point for the revision process. Simply counting individual citations can incur major disadvantages for ECRs however, especially in fields with long citation lag times such as EOH.8 These conundrums arise because less experienced researchers have simply had less time to develop the extensive networks necessary for success in modern scientific enterprise. At the same time, most of their scientific output has only recently been published, and has therefore had a relatively limited opportunity to attract citations. In the current article, I propose a series of revisions to the bibliometric assessment framework often used in EOH, and
more broadly, across the wider sphere of health research. Aside from the potential H-Index limitations already described, there are three main aspects for research managers, staff mentors and supervisors to consider. Firstly, I suggest that any metrics-based techniques for assessing ECRs consider their initial 10 years to be an ‘early career’ phase and evaluated accordingly. This time frame is not without precedent, as other authors have also utilized the 10-year post PhD period as an ‘Early Career’ stage.9 Secondly, I propose that citation counts in the first 5 years of an article’s life (following publication), should be evaluated cautiously. Again, this time frame is not without precedent, with at least one scholar pointing out that the reliable measurement of citation impact cannot occur until around three years have passed following publication.10 As such, I suggest that we extend this 3-year time frame to 5 years in EOH, given the long citation lag time intrinsic to our discipline. It is worth noting that Thomson Reuters now calculates a 5-year journal impact factor, in addition to the original score which is derived from publications over a 2-year time period.11 Aside from the time lag between publication and citation, there are also some bigger picture issues to consider when developing research capacity in EOH. Firstly, it is important to recognize the value of human capital in research and accept that the continued upskilling of an organization’s own staff is usually in everyone’s best interests.12 It is also important to recognize that article citations and the scientific enterprise itself, does not occur in a vacuum. Because the entire bibliometrics paradigm revolves around citations made by human beings, it is reasonable to assume that a certain degree of behavioral and social characteristics are involved in the chain of events that ultimately lead to an article being cited. It is wellknown that the citation process itself is generally not random, as article citations reflect idiosyncrasies of the citer13 as well as the social networks of authors within a particular field.14 As with all social skills and human relationships, forming these connections, building networks and even understanding the game, simply takes time. Another issue when developing research careers is to consider the age of the individual themselves. On one hand, there is an argument that major discoveries tend to occur during a scientist’s early years. Isaac Newton supposedly undertook his first scientific experiment at the age of 16,15 while Albert Einstein was of the opinion that people who had not made a great contribution to science before the age of 30 would probably never do so.16 American researcher Harvey C. Lehman (1889–1965) devoted three decades of his life to studying this
118 issue in detail, publishing a landmark book titled: Age and Achievement, as well as a series of journal articles last century.17 Lehman’s research found that a scientist’s peak productivity most commonly occurred in their 30s, albeit with this value ranging somewhat between the disciplines.18 In the field of chemistry, for example, a scientist’s most creative years were found to be between the ages of 30 and 34.19 On the other hand, however, Thomas Edison was apparently still active in the laboratory at the age of 80.20 Similarly, an early study of mathematicians and scientists found that their annual citations did not peak until a mean age of 59.21 A more recent investigation found that the productivity of university academics increased as they grew older (into their 60s).22 Motivation probably has less to do with financial reasons by this age, however, with one study reporting that academics who worked beyond the age of 70, usually did so because of the intrinsic rewards of their work.23 Although establishing the most productive age among EOH researchers is a difficult task, a brief look at some early pioneers in our field suggests that outstanding performance was often evident at relatively young ages. The most highlycited author in EOH for example, John Christopher Wagner (1923–2000), was in his 30s when he first discovered mesothelioma in the lungs of a miner originally thought to have died from tuberculosis. This led Wagner (by then in his late 30s) and colleagues to write a groundbreaking article titled: ‘Diffuse pleural mesothelioma and asbestos exposure in the North Western Cape province’.24 Published in the British Journal of Industrial Medicine in 1960, it was one of the first studies to establish a link between asbestos exposure and pleural mesothelioma.25 By 1979 Wagner’s article had been recognized as a ‘Citation Classic’ with over 240 citations,26 and in 2007 became the most highly-cited article ever published in the field of occupational medicine, with over 900 citations.3 While high citation counts may help elucidate scientific achievement to some extent, in the days before bibliometric evaluation, the recognition of pioneering individuals usually took a little longer. Alice Hamilton (1869–1970), for example, was first appointed chief investigator for the Illinois State Commission on Industrial Diseases in her 40s and became the first female faculty member at Harvard University in her early 50s.27 Similarly, Carey P. McCord (1886–1979) was said to have reached his ‘apogee of international renown’ after 50 years in occupational medicine.28 Although there are clearly many different aspects to consider, some basic strategies to help improve the assessment and development of ECRs can be summarized as follows:
• Raw H-Index scores are less than optimal for assessing early career researchers and should (at least) be adjusted for career length and citation density • The 10-year time period following an author’s first publication should be recognized as their ‘Early Career’ phase and evaluated accordingly • The first 3 years of an article’s life (post publication) should not be evaluated using standard citation-based metrics (5 years is a more realistic time frame)
Archives of Environmental & Occupational Health • The age of the author is an important consideration when planning strategies for improving research performance in EOH, as elsewhere While the abovementioned list is not (and cannot) be an allinclusive inventory of where changes should be made, it is nevertheless the start of what will likely be a complicated revision process. One major issue confronting bibliometrics-based assessment centers around the fact that modern scientific research is usually not undertaken alone. Cooperative research and subsequent co-authorship, by its very nature, incurs a conundrum for those undertaking a citation-based assessment of individual researchers. This occurs because assigning author credit, establishing proportional intellectual contribution and simply figuring out who did what, is an inherent complication not well-accounted for by individual bibliometric measures such as the H-Index. Certainly, if every scientist published single-author papers, assessing their performance would be relatively straightforward from a citation-based perspective. This is rarely the case however, as the move towards multiple co-authors on scientific manuscripts intensifies and the average number of authors per article increases.29 Various solutions for evaluating multiple authorship have subsequently been proposed when considering H-Index scores. Essentially, one either counts article citations in a fractional way or one counts the ranks of these articles in a fractional way as credit for an author.30 Doing so is not straightforward, however. Although it is possible to simply divide the number of citations by the number of authors on a particular article so that the citations are shared between them,30 this would then require each individual author’s citation records to be rearranged according to their new fractional citation counts.31 An alternative solution is to rank and / or exclude potential authors at the front end (prior to publication). In 1991, for example, Hunt32 proposed an ‘Authorship Index’ that assigns authors a scaled series of points (0-25) across 5 categories: intellectual input, practical input (data-capture), practical input (beyond data-capture), specialist input and literary input. Such solutions may be appropriate for EOH. On the other hand, while the issue of self-citations and their effect on bibliometric measurements should not be ignored either, it may not be as critical as often believed, if only due to relative ubiquity across the spectrum. An early bibliometric study from 1977,33 for example, found that very few articles did not include any self-citations. Well-established authors most likely have a larger body of existing, published work from which to draw self-citations. In summary, it can be seen that the assessment of research performance and the development of academic careers will clearly require some changes to the current methods, if not the entire paradigm. While certain refinements will be needed in the intersecting areas of bibliometrics, researcher development and allied health, it is also critical that the basics are not overlooked. A few key points should be remembered here. Firstly, EOH researchers, especially those of the next generation, must be continually encouraged to publish in the dedicated journals of our field. According to some scholars, failing to do so risks a progressive loss of identity and a discontinuation of the historical values originally conveyed by Ramazzini
Emerging Topics in EOH Research over 300 years ago.34 Secondly, ECRs must continue to recognize and be reminded of, the value of perseverance in scientific endeavors. Chris Wagner’s aforementioned article describing mesothelioma in the Western Cape, for example, was originally rejected when pathologists stated that mesotheliomas did not exist.26 Wagner persevered, and with the support of some senior colleagues, his article was eventually accepted. Our world is now a safer place because of it. Derek R. Smith Deputy Editor-in-Chief Archives of Environmental & Occupational Health
References 1. Smith DR. ‘Platinum H’: Refining the H-Index to more realistically assess career trajectory and scientific publications. Arch Environ Occup Health. 2015; 70(2): 67–9. 2. Smith DR. A 30-year citation analysis of bibliometric trends at the Archives of Environmental Health, 1975–2004. Arch Environ Occup Health. 2009; 64(sup1): 43–54. 3. Gehanno JF, Takahashi K, Darmoni S, Weber J. Citation classics in occupational medicine journals. Scand J Work Environ Health. 2007; 33(4): 245–51. 4. McCunney RJ, Harzbecker J. The influence of occupational medicine on general medicine: a look at the journals. J Occup Med. 1992; 34(3): 279–86. 5. Franco G. Scientific research of senior Italian academics of occupational medicine: a citation analysis of products published during the decade 2001–2010. Arch Environ Occup Health. 2015; 70(2): 110–115. 6. Hirsch JE. An index to quantify an individual’s scientific research output. P Natl Acad Sci USA 2005; 102(46): 16569–72. 7. Franchignoni F, Munoz Lasa S, Ozcakar L, Ottonello M. Bibliometric indicators: a snapshot of the scientific productivity of leading European PRM researchers. Eur J Phys Rehabil Med. 2011; 47(3): 455–62. 8. Smith DR. Assessing productivity among university academics and scientific researchers. Arch Environ Occup Health. 2015; 70(1): 1–3. 9. Cronin B, Meho LI. Timelines of creativity: A study of intellectual innovators in information science. J Am Soc Inf Sci Tech. 2007; 58(13): 1948–59. 10. Bornmann L. The problem of citation impact assessments for recent publication years in institutional evaluations. J Informetr. 2013; 7(3): 722–9. 11. Thomson Reuters Webpage: Journal Evaluation & Highly Cited Research. Available at: http://wokinfo.com/media/pdf/JEHCR. Accessed March 26, 2015.
119 12. Hanssen T-ES, Jørgensen F. The value of experience in research. J Informetr. 2015; 9(1): 16–24. 13. Seglen PO. The skewness of science. J Am Soc Inf Sci. 1992; 43(9): 628–38. 14. Fu TJ, Song Q, Chiu D. The academic social network. Scientometrics. 2014; 101(1): 203–39. 15. Lehman HC. Young thinkers and memorable creative achievements. J Genet Psychol. 1964; 105(2): 237–55. 16. Stern N. Age and achievement in mathematics: a case-study in the sociology of science. Socl Stud Sci. 1978; 8(1): 127–40. 17. Simonton DK. Age and outstanding achievement: what do we know after a century of research? Psycholl Bull. 1988; 104(2): 251–67. 18. Lehman HC. Age and Achievement. Princeton: Princeton University Press; 1953. 19. Lehman HC. The chemist’s most creative years. Science (New York, NY 1958; 127(3308): 1213–22. 20. When is old? Ind Med Surg. 1952; 21(10): 495. 21. Diamond AM. The life-cycle research productivity of mathematicians and scientists. J Gerontol. 1986; 41(4): 520–5. ¨ Taidre E. High work ability in the scientific activity of 22. Kristjuhan U, older and experienced academics. Work: J Prev Assess Rehab. 2012; 41(0): 313–5. 23. Dorfman LT. Stayers and leavers: professors in an era of no mandatory retirement. Educ Gerontol. 2002; 28(1): 15–33. 24. Wagner JC, Sleggs CA, Marchand P. Diffuse pleural mesothelioma and asbestos exposure in the North Western Cape Province. Br J Ind Med. 1960; 17(4): 260–71. 25. International Mesothelioma Interest Group Webpage: J. Christopher Wagner Biography. Available at: http://imig.org/about/wagner-award-recipients-2/j-christopher-wagner-biography. Accessed March 23, 2015. 26. This week’s citation classic. Current Contents. 1979; 32(August 6): 317. 27. Mayers MR, Alice Hamilton. Am Ind Hyg Assoc J. 1958; 19(6): 449–52. 28. O’Connor RB, Carey P, McCord MD. J Occup Med. 1969; 11(12): 693–4. 29. Smith DR. Authorship, scholarship and ergonomics. Trav Humain. 2009; 72(4): 397–403. 30. Egghe L. Mathematical theory of the h- and g-index in case of fractional counting of authorship. J Am Soc Inf Sci Tech. 2008; 59(10): 1608–16. 31. Schreiber M. A case study of the modified Hirsch index hm accounting for multiple coauthors. J Am Soc Inf Sci Tech. 2009; 60(6): 1274–82. 32. Hunt R. Trying an authorship index. Nature 1991; 352: 187. 33. Tagliacozzo R. Self-citations in scientific literature. J Doc. 1977; 33(4): 251–65. 34. Franco G. Research evaluation and competition for academic positions in occupational medicine. Arch Environ Occup Health. 2013; 68(2): 123–7.
Copyright of Archives of Environmental & Occupational Health is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.