European Journal of Neuroscience, Vol. 41, pp. 1111–1112, 2015
doi:10.1111/ejn.12886
Neuro opinion: reforming the academic system is a joint responsibility €ls,1,* Monica Di Luca2,† and Barry J. Everitt3,‡ Marian Joe 1
Department of Translational Neuroscience, Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, The Netherlands 2 Department of Pharmacological Sciences, University of Milano, Milan, Italy 3 Department of Psychology, University of Cambridge, Cambridge, UK There has perhaps never been a more exciting time to be involved in neuroscience research. The huge challenge of understanding the workings of the brain and mind are increasingly tractable because of an explosion in new and powerful methodologies that can be used in animals or humans or both. Over 15% of all scientific papers published worldwide are related to the brain or behaviour (http:// www.elsevier.com/__data/assets/pdf_file/0003/236730/ElsevierBrainScienceReport2014-web.pdf). Highly intelligent and capable young researchers are eager to be involved in neuroscience research and many are undoubtedly attracted by the possibility of answering questions that could barely be asked even 20 years ago. But there is also a dark side to our field that we need to understand and limit if we are to avoid losing talented young researchers at such an important moment. In an ideal scientific world, bright ideas lead to hypotheses that are tested by performing carefully designed, well-controlled and rigorous experiments. These lead to exciting results that form the basis of a paper that is written and submitted for publication, followed by the rapid receipt of a letter of acceptance. But life is rarely like that. Why? The bright ideas come for free, although they require long-term individual investment in hard thinking about the research to be undertaken based on a comprehensive understanding of what is known and what is not. Invest in this process and cherish your brainwaves! They are a rare commodity that emanate from your own personal engagement with your field of research. The first real hurdle then to be faced is to acquire the funds, in the form of fellowships and grants, in order to undertake the research. Although occasionally a single pair of hands can complete a major study, in order to answer big questions several pairs of hands and a collaborative group of researchers are often required. While universities and institutes may provide some of the necessary lab infrastructure, financial support for human resources, both researchers and technicians, as well as equipment and consumables must be obtained from funding agencies. This has become increasingly difficult. Success rates for European Research Council starting grants hover well below 10% (http:// erc.europa.eu/sites/default/files/document/file/erc_2013_stg_statistics. pdf). The first rounds of Horizon 2020 grants saw a 3% success rate. This is not just confined to Europe. Recent data from NIH show that the average age at which American scientists receive their
Correspondence: Marian Jo€els, as above. E-mail: [email protected] *MJ is Past-President of FENS. † MDL is currently President of FENS. ‡ BJE served as co-Editor-in-Chief of EJN from 1997 to 2008 and is President-Elect of FENS.
first RO1 grant (a substantial personal grant by NIH) has increased from 36 to 42 (http://grants.nih.gov/grants/new_investigators/). Alberts et al. (2014) have recently argued that the system is dysfunctional and increasingly out of control. Despite the good news that, since 1996, 1.7 million active researchers worldwide were investigating the brain (http://www.elsevier.com/__data/assets/ pdf_file/0003/236730/ElsevierBrainScienceReport2014-web.pdf), the inevitable downside is that there are simply too many neuroscientists competing for limited funds. Under such conditions, self-interest unfortunately often overrides efforts towards a common good. Many are so focused on their own success that the important qualities of working together, of team spirit and helping others may have somewhat lost their appeal. This may be an adaptive and profitable course of action in the short term, but it is counter-productive in the long run, not least because we must reach out and help the next generation of researchers, our own successors. It is therefore important that teaching is properly valued and not seen, as so often it is, as a distraction from research. We agree entirely with Alberts et al. (2014) that the system that has evolved needs to change, and the only way this can happen is through the concerted action of (neuro) scientists themselves. Those of our colleagues who have made a deliberate choice to invest in the neuroscience community as a whole through their work on university, grant and scientific society committees, sometimes with a cost to their scientific output, should be valued in a reformed system. There are wonderful examples in Europe of exceptional neuroscientists who have opted to play a role in both European and national politics or who through personal perseverance have secured large amounts of funding either at the European level or for their national neuroscience communities. Let us assume that the research funding is available, bright young scientists have been recruited to undertake groundbreaking studies and original and compelling data are collected. The next obstacle to be faced is publishing the results. In the electronic era in which we live it might be assumed that it makes little difference where the papers are published, provided of course that the work is subject to careful peer review. After all, the power of search engines and the right keywords will enable any publication to be found, no matter where it is published. This simple truth is negated by a very influential factor: prestige. Some journals have kept the number of published articles low by rejecting as many as 90% of submitted articles in order to keep the Impact Factor high. It seems counter-intuitive that even some Society journals have adopted this path, to the detriment of their own members to whom the message appears to be that only a very limited number of them, or only a limited proportion of their work, is worthy of publication in the journal that represents their field. This element of exclusivity has proven to be so attractive that we continue to submit our papers to
© 2015 Federation of European Neuroscience Societies and John Wiley & Sons Ltd
1112 M. Jo€els et al. these journals in order to boost our visibility and prestige. For younger researchers, the perception, instigated by (among others) their advisers, institutions and funding agencies, is that job prospects in neuroscience and the ability to secure funding are bleak unless they can display a ‘high-impact paper’ early in their curriculum vitae. We play the game and perpetuate the Impact Factor hegemony, and one consequence is that the ‘elite’ journals exert an extraordinary degree of control over which papers will be selected, or even sent out for review. This frequently leads to what feels like capricious decision-making that can have a devastating impact on the morale and motivation of young scientists who have spent 2 or 3 years on a single project conducting research of genuine quality and value. Often the printed paper is only a fraction of the online supplementary material (which may not have been carefully reviewed) and any number of additional experiments can be asked for by reviewers, sometimes misleadingly dressed up as a request for ‘essential controls’, before a paper will be considered for publication. We have made ourselves hostages to this system and, as with the increasingly challenging peer-review grant system, the people who need to decide that this must end are the scientists themselves. Unless influential scientists, including neuroscientists, stand up and organize an orchestrated rejection of the status quo, the current situation will endure or worsen. The point we wish to make here is that precisely the situation we now find ourselves in is counter-productive to scientific careers and more importantly to scientific progress. It wastes much precious time and puts everyone under unnecessary pressure, even to the point where occasionally some are lured into the dark world of scientific fraud, while others become so discouraged that they reject a research career, even though it is something they intrinsically love doing and at which they are likely to excel. This is the crux of the matter: society at large faces enormous challenges related to the brain, including the cost of brain diseases, and we need as many bright and talented researchers as possible to tackle these challenges. We simply cannot afford to lose any for the wrong reasons. Instead of playing this increasingly toxic game, we plead for a resetting of the criteria for effective and rapid publication by placing emphasis on what matters, namely scientific rigor based on methodological thoroughness and appropriate statistical evaluation. We do not of course need bad science presented in bad papers. But nor do we need catchy little studies that are judged on their likelihood of being picked up by the media for simple newsworthiness. We make a plea to journals not to shy away from publishing replication studies, or negative results that have arisen from rigorous research but which fail to replicate earlier published data or which refute a
prevailing view or hypothesis. This is how science progresses. The (probably unjustified) fear that a journal’s Impact Factor might take a dip when negative findings are reported should not be a leading principle; it is not the Impact Factor that matters, it is instead the intrinsic scientific value, quality and reproducibility of the research and thereby the degree to which it underlies progress in the field. We encourage journals to publish such studies and decrease the current publication and outcome bias towards positive findings. In clinical research the publication of replication studies has become more acceptable, and for good reason: it has already led to important readjustments in clinical practice. Animal studies would equally benefit from such a course, allowing meta-analyses to be conducted and thereby improving their overall power. Finally, we believe that Society journals in particular should serve their community of scientists by publishing rigorous research from across their discipline, and not selecting only those papers subjectively judged in terms of their likely contribution to the upwards drift of the Impact Factor. Moreover, while general neuroscience journals publish papers that reflect the rich diversity of the field, from genes and molecules to systems, behavior, cognition and neurological and neuropsychiatric diseases, there may be a tendency to look less favorably on papers from fields viewed as ‘small’ because citations of them may be few, thereby detracting from the drive to increase the Impact Factor. We argue instead that the negative effect of excluding them, even on relatively small sections of our community, has been and remains a strong reason for publishing them. Few scientists, or indeed Editors of journals, will argue that Impact Factor is the best indicator of research quality and rigor; there are other available measures such as citation half-life that are perhaps more informative. Let us try to change the dominant influence of Impact Factor and realign the motivation it generates among scientists to publish only in a restricted group of journals. There is much to be said for supporting Society journals, such as the European Journal of Neuroscience and eNeuro, as they stand for both scientific rigor and breadth of discipline. Together with eLife, they are also committed to repairing a broken peer-review system that has evolved, and they wish to serve the inclusivity, rather than the selectivity, of neuroscience publishing.
Reference Alberts, B., Kirschner, M.W., Tilghman, S. & Varmus, H. (2014) Rescuing US biomedical research from its systematic flaws. Proc. Natl. Acad. Sci. USA, 111, 5773–5777.
© 2015 Federation of European Neuroscience Societies and John Wiley & Sons Ltd European Journal of Neuroscience, 41, 1111–1112
doi:10.1111/ejn.12886
Neuro opinion: reforming the academic system is a joint responsibility €ls,1,* Monica Di Luca2,† and Barry J. Everitt3,‡ Marian Joe 1
Department of Translational Neuroscience, Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, The Netherlands 2 Department of Pharmacological Sciences, University of Milano, Milan, Italy 3 Department of Psychology, University of Cambridge, Cambridge, UK There has perhaps never been a more exciting time to be involved in neuroscience research. The huge challenge of understanding the workings of the brain and mind are increasingly tractable because of an explosion in new and powerful methodologies that can be used in animals or humans or both. Over 15% of all scientific papers published worldwide are related to the brain or behaviour (http:// www.elsevier.com/__data/assets/pdf_file/0003/236730/ElsevierBrainScienceReport2014-web.pdf). Highly intelligent and capable young researchers are eager to be involved in neuroscience research and many are undoubtedly attracted by the possibility of answering questions that could barely be asked even 20 years ago. But there is also a dark side to our field that we need to understand and limit if we are to avoid losing talented young researchers at such an important moment. In an ideal scientific world, bright ideas lead to hypotheses that are tested by performing carefully designed, well-controlled and rigorous experiments. These lead to exciting results that form the basis of a paper that is written and submitted for publication, followed by the rapid receipt of a letter of acceptance. But life is rarely like that. Why? The bright ideas come for free, although they require long-term individual investment in hard thinking about the research to be undertaken based on a comprehensive understanding of what is known and what is not. Invest in this process and cherish your brainwaves! They are a rare commodity that emanate from your own personal engagement with your field of research. The first real hurdle then to be faced is to acquire the funds, in the form of fellowships and grants, in order to undertake the research. Although occasionally a single pair of hands can complete a major study, in order to answer big questions several pairs of hands and a collaborative group of researchers are often required. While universities and institutes may provide some of the necessary lab infrastructure, financial support for human resources, both researchers and technicians, as well as equipment and consumables must be obtained from funding agencies. This has become increasingly difficult. Success rates for European Research Council starting grants hover well below 10% (http:// erc.europa.eu/sites/default/files/document/file/erc_2013_stg_statistics. pdf). The first rounds of Horizon 2020 grants saw a 3% success rate. This is not just confined to Europe. Recent data from NIH show that the average age at which American scientists receive their
Correspondence: Marian Jo€els, as above. E-mail: [email protected] *MJ is Past-President of FENS. † MDL is currently President of FENS. ‡ BJE served as co-Editor-in-Chief of EJN from 1997 to 2008 and is President-Elect of FENS.
first RO1 grant (a substantial personal grant by NIH) has increased from 36 to 42 (http://grants.nih.gov/grants/new_investigators/). Alberts et al. (2014) have recently argued that the system is dysfunctional and increasingly out of control. Despite the good news that, since 1996, 1.7 million active researchers worldwide were investigating the brain (http://www.elsevier.com/__data/assets/ pdf_file/0003/236730/ElsevierBrainScienceReport2014-web.pdf), the inevitable downside is that there are simply too many neuroscientists competing for limited funds. Under such conditions, self-interest unfortunately often overrides efforts towards a common good. Many are so focused on their own success that the important qualities of working together, of team spirit and helping others may have somewhat lost their appeal. This may be an adaptive and profitable course of action in the short term, but it is counter-productive in the long run, not least because we must reach out and help the next generation of researchers, our own successors. It is therefore important that teaching is properly valued and not seen, as so often it is, as a distraction from research. We agree entirely with Alberts et al. (2014) that the system that has evolved needs to change, and the only way this can happen is through the concerted action of (neuro) scientists themselves. Those of our colleagues who have made a deliberate choice to invest in the neuroscience community as a whole through their work on university, grant and scientific society committees, sometimes with a cost to their scientific output, should be valued in a reformed system. There are wonderful examples in Europe of exceptional neuroscientists who have opted to play a role in both European and national politics or who through personal perseverance have secured large amounts of funding either at the European level or for their national neuroscience communities. Let us assume that the research funding is available, bright young scientists have been recruited to undertake groundbreaking studies and original and compelling data are collected. The next obstacle to be faced is publishing the results. In the electronic era in which we live it might be assumed that it makes little difference where the papers are published, provided of course that the work is subject to careful peer review. After all, the power of search engines and the right keywords will enable any publication to be found, no matter where it is published. This simple truth is negated by a very influential factor: prestige. Some journals have kept the number of published articles low by rejecting as many as 90% of submitted articles in order to keep the Impact Factor high. It seems counter-intuitive that even some Society journals have adopted this path, to the detriment of their own members to whom the message appears to be that only a very limited number of them, or only a limited proportion of their work, is worthy of publication in the journal that represents their field. This element of exclusivity has proven to be so attractive that we continue to submit our papers to
© 2015 Federation of European Neuroscience Societies and John Wiley & Sons Ltd
1112 M. Jo€els et al. these journals in order to boost our visibility and prestige. For younger researchers, the perception, instigated by (among others) their advisers, institutions and funding agencies, is that job prospects in neuroscience and the ability to secure funding are bleak unless they can display a ‘high-impact paper’ early in their curriculum vitae. We play the game and perpetuate the Impact Factor hegemony, and one consequence is that the ‘elite’ journals exert an extraordinary degree of control over which papers will be selected, or even sent out for review. This frequently leads to what feels like capricious decision-making that can have a devastating impact on the morale and motivation of young scientists who have spent 2 or 3 years on a single project conducting research of genuine quality and value. Often the printed paper is only a fraction of the online supplementary material (which may not have been carefully reviewed) and any number of additional experiments can be asked for by reviewers, sometimes misleadingly dressed up as a request for ‘essential controls’, before a paper will be considered for publication. We have made ourselves hostages to this system and, as with the increasingly challenging peer-review grant system, the people who need to decide that this must end are the scientists themselves. Unless influential scientists, including neuroscientists, stand up and organize an orchestrated rejection of the status quo, the current situation will endure or worsen. The point we wish to make here is that precisely the situation we now find ourselves in is counter-productive to scientific careers and more importantly to scientific progress. It wastes much precious time and puts everyone under unnecessary pressure, even to the point where occasionally some are lured into the dark world of scientific fraud, while others become so discouraged that they reject a research career, even though it is something they intrinsically love doing and at which they are likely to excel. This is the crux of the matter: society at large faces enormous challenges related to the brain, including the cost of brain diseases, and we need as many bright and talented researchers as possible to tackle these challenges. We simply cannot afford to lose any for the wrong reasons. Instead of playing this increasingly toxic game, we plead for a resetting of the criteria for effective and rapid publication by placing emphasis on what matters, namely scientific rigor based on methodological thoroughness and appropriate statistical evaluation. We do not of course need bad science presented in bad papers. But nor do we need catchy little studies that are judged on their likelihood of being picked up by the media for simple newsworthiness. We make a plea to journals not to shy away from publishing replication studies, or negative results that have arisen from rigorous research but which fail to replicate earlier published data or which refute a
prevailing view or hypothesis. This is how science progresses. The (probably unjustified) fear that a journal’s Impact Factor might take a dip when negative findings are reported should not be a leading principle; it is not the Impact Factor that matters, it is instead the intrinsic scientific value, quality and reproducibility of the research and thereby the degree to which it underlies progress in the field. We encourage journals to publish such studies and decrease the current publication and outcome bias towards positive findings. In clinical research the publication of replication studies has become more acceptable, and for good reason: it has already led to important readjustments in clinical practice. Animal studies would equally benefit from such a course, allowing meta-analyses to be conducted and thereby improving their overall power. Finally, we believe that Society journals in particular should serve their community of scientists by publishing rigorous research from across their discipline, and not selecting only those papers subjectively judged in terms of their likely contribution to the upwards drift of the Impact Factor. Moreover, while general neuroscience journals publish papers that reflect the rich diversity of the field, from genes and molecules to systems, behavior, cognition and neurological and neuropsychiatric diseases, there may be a tendency to look less favorably on papers from fields viewed as ‘small’ because citations of them may be few, thereby detracting from the drive to increase the Impact Factor. We argue instead that the negative effect of excluding them, even on relatively small sections of our community, has been and remains a strong reason for publishing them. Few scientists, or indeed Editors of journals, will argue that Impact Factor is the best indicator of research quality and rigor; there are other available measures such as citation half-life that are perhaps more informative. Let us try to change the dominant influence of Impact Factor and realign the motivation it generates among scientists to publish only in a restricted group of journals. There is much to be said for supporting Society journals, such as the European Journal of Neuroscience and eNeuro, as they stand for both scientific rigor and breadth of discipline. Together with eLife, they are also committed to repairing a broken peer-review system that has evolved, and they wish to serve the inclusivity, rather than the selectivity, of neuroscience publishing.
Reference Alberts, B., Kirschner, M.W., Tilghman, S. & Varmus, H. (2014) Rescuing US biomedical research from its systematic flaws. Proc. Natl. Acad. Sci. USA, 111, 5773–5777.
© 2015 Federation of European Neuroscience Societies and John Wiley & Sons Ltd European Journal of Neuroscience, 41, 1111–1112
