INSIGHTS
LET TERS Edited by Jennifer Sills
Evaluating Science’s open-data policy
654
Downloaded from http://science.sciencemag.org/ on August 24, 2017
Students could benefit from STEM programs structured to help them succeed.
a complete and understandable data set that is open to reviewers and, ultimately, becomes open to scientists and the public. Dominique G. Roche Institute of Biology, University of Neuchâtel, 2000, Neuchâtel, Switzerland. Email: [email protected] REFERENCES
1. Science, Editorial policies (www.sciencemag.org/authors/ science-editorial-policies). 2. C. J. Savage, A. J. Vickers, PLOS ONE 4, e7078 (2009). 3. T. Vines et al., Curr. Biol. 24, 94 (2014). 4. D. G. Roche, L. E. B. Kruuk, R. Lanfear, S. A. Binning, PLOS Biol. 13, e1002295 (2015).
10.1126/science.aan8158
Editor’s note Before acceptance of a paper at Science, all authors must sign a statement affirming that “all data necessary for a reader to understand and evaluate the conclusions of the paper are included in the paper or its supplement or will be archived in an approved database and made available to any reader.” In the case of the retracted Report to which Roche refers in his Letter, both authors signed the statement. It became apparent only after publication that the data in the supplementary material were incomplete, that they had not been archived, and that the only copy of the raw observations on which the findings were based was allegedly stored on a computer that had been stolen. Our policies are intended to ensure that all appropriate data are available to our readers, and we are actively examining ways to make these policies more effective. Jeremy Berg Editor-in-Chief
10.1126/science.aao5141
Helping less-prepared students excel Many minority-serving institutions have had great success in graduating students in science, technology, engineering, and math (STEM) disciplines, but this hasn’t made a substantial dent in the numbers of minorities in these fields. Making a real difference will require universities to change their approach. STEM programs often try to weed out students who are deemed “less prepared” or who have not demonstrated that they are capable of succeeding in the most demanding courses. Instead of providing support, university advisers may encourage these students to seek less demanding majors. Faculty members lack incentives to expend extra effort to help less-prepared students catch up to their peers. It is perceived by many to be much easier, and more prestigious, to work with students who immediately excel in their coursework. Many less-prepared students, including a large percentage of minority STEM students, would benefit instead from policies that have as their goal keeping them in STEM. For example, some primarily undergraduate institutions with small departments [e.g., (1)] encourage all students to initially engage in research projects that not only expose them to “real” science, but also provide close mentoring by full-time faculty. Some students are allowed to take lighter course loads if they are actively engaged in research, thus extending the time to attain a degree. The Engineering Augmented Degree Program (ENGAGE) at the School of Engineering at the University of Pretoria sciencemag.org SCIENCE
18 AUGUST 2017 • VOL 357 ISSUE 6352
Published by AAAS
PHOTO: FATCAMERA/ISTOCKPHOTO
Science must re-evaluate its open-data policy after retracting a controversial study on microplastics and fish in May (“Editorial Retraction,” J. Berg, 26 May, p. 812 and “Fishy business,” M. Enserink, News Feature, 24 March, p. 1254). The only computer containing the study’s raw data was allegedly stolen and no backups existed on another machine or an online repository. Many are left wondering how this could happen in an era of cloud computing and open data. Science currently has a two-step editorial policy to promote data availability and research transparency (1). First, “before publication, large data sets must be deposited in an approved database and an accession number…included in the published paper.” Second, “after publication, all data [and code] necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science.” Unfortunately, two loopholes weaken the effectiveness of this policy. First, it is unclear why “small data sets” are exempt from archival before publication given that they are simple to share and equally important in verifying research results (as the above case demonstrates). Second, research shows that individual scientists cannot be trusted to reliably preserve their data and share them upon request (2, 3). Compounding this problem is the fact that publishers seldom state the consequences of breaching their policies. For example, does failure to provide raw data post-publication lead to the automatic retraction of the paper? The solution to avoid data disappearing is simple: Journals must mandate and enforce data archival on a recognized, online repository at the time of submission. Only editors and reviewers would have access during peer review; the data would be made generally accessible upon publication. Publishing verifiable research is a tenet of scientific progress and, ultimately, journals are responsible for guaranteeing compliance with their open-data policy. At a minimum, this responsibility involves a cursory check of the underlying data and ensuring that all data are available for reviewers to assess (4). Science publishes many papers describing major breakthroughs, but these extraordinary claims must be supported by extraordinary evidence. This includes, first and foremost,
in South Africa (2) is one example of an inclusive program that welcomes all but strives to help those who are less prepared. The ENGAGE curriculum gradually increases the volume of work over five years to help students adjust to life at the university. The students are provided with mentoring and other forms of academic and social support, including peer-to-peer interactions. This program has documented stunning success for students from underrepresented groups (3), such as black students from poor townships, and the concept may merit serious consideration by other colleges and universities. Allen Sessoms The Hollins Group, Washington, DC 20011, USA. Email: [email protected] REFERENCES
10.1126/science.aao4525
R. Revesz,1 M. Greenstone,2 M. Hanemann,3 M. Livermore, 4 T. Sterner,5 D. Grab,1 P. Howard, 1 J. Schwartz1* 1
Best cost estimate of greenhouse gases In March, President Trump’s Executive Order 13783 disbanded the Interagency Working Group on the Social Cost of Greenhouse Gases (IWG) (1). IWG developed estimates for federal agencies to use in cost-benefit analyses of climate policies. IWG’s most recent central estimate was $50 in global damages per ton of carbon dioxide, based on year 2020 emissions, converted from 2007 to 2017 dollars (2). Trump’s Executive Order withdrew IWG’s official valuations and instead instructed agencies to monetize climate effects using “the best available science and economics” (1). Yet IWG’s estimates already are the product of the most widely peer-reviewed models and best available data (3). The Executive Order asks agencies to reconsider “appropriate discount rates” (the factor for converting future costs and benefits into present-day values) and “domestic versus international impacts” (1). These instructions implicitly question IWG’s choices to base central estimates on a 3% instead of a 7% discount rate (higher discount rates place less value on avoiding future damages) and to value global damages rather than ignore climate effects beyond U.S. borders. However, scientists and economists widely endorse these methodological choices. The National
Institute for Policy Integrity, New York University School of Law, New York, NY 10012, USA. 2 Department of Economics, University of Chicago, Chicago, IL 60637, USA. 3Department of Agriculture and Resource Economics, University of California– Berkeley, Berkeley, CA 94720-3310, USA. 4University of Virginia School of Law, Charlottesville, VA 229031738, USA. 5Department of Economics, University of Gothenburg, Gothenburg, SE-405 30, Sweden. *Corresponding author. Email: [email protected] REFERENCES
1. Executive Office of the President, “Executive Order 13783: Promoting energy independence and economic growth,” Federal Register 82, 16093 (2017). 2. U.S. Interagency Working Group on the Social Cost of Greenhouse Gases (IWG), “Technical Support Document: Technical Update of the Social Cost of Carbon for Regulatory Impact Analysis Under Executive Order 12866” (2016); https://obamawhitehouse.archives. gov/sites/default/files/omb/inforeg/scc-tsd-finaljuly-2015.pdf. 3. M. Greenstone, et al., Rev. Environ. Econ. Pol. 7, 23 (2013). 4. National Academies of Sciences, “Valuing Climate Damages: Updating Estimation of the Social Cost of Carbon Dioxide” (National Academies Press, 2017); www.nap.edu/download/24651. 5. U.S. Council of Economic Advisers, “Discounting for public policy: Theory and recent evidence on the merits of updating the discount rate” (2017); https://obamawhitehouse.archives.gov/sites/default/files/page/ files/201701_cea_discounting_issue_brief.pdf. 6. P. Howard, D. Sylvan, “The Wisdom of the Economic Crowd: Calibrating Integrated Assessment Models Using Consensus” (2016); http://ageconsearch.umn.edu/ bitstream/235639/2/HowardSylvan_AAEA2016.pdf. 7. M. Greenstone, “What Financial Markets Can Teach Us About Managing Climate Risks,” New York Times (4 April 2017). 8. A. Fraas et al., Science 351, 569 (2016). 9. P. Howard, J. Schwartz, Columbia J. Environ. Law 42, 203 (2017). 10. A. Barreca et al., J. Polit. Econ. 124 , 105 (2016).
10.1126/science.aao4322
SCIENCE sciencemag.org
Published by AAAS
Downloaded from http://science.sciencemag.org/ on August 24, 2017
1. Lafayette, Student Research (https://ece.lafayette.edu/ opportunities/student-research/). 2. University of Pretoria, Engineering Augmented Degree Program (www.up.ac.za/engage-program). 3. D. Grayson, in Proceedings of the First Biennial Conference of the South African Society for Engineering Education, B. Collier-Reed, Ed. (Stellenbosch South Africa, 2011), pp. 69–79.
Academies of Sciences and the U.S. Council of Economic Advisers (4, 5) strongly support a 3% or lower discount rate for intergenerational effects. A 7% rate based on private capital returns is considered inappropriate because the risk profiles of climate effects differ from private investments (6, 7). Most economists and climate policy experts [though not all (8)] also defend valuing the full global externalities of U.S. emissions to reinforce reciprocal climate policies in other countries (3, 4, 9). Moreover, current models cannot accurately estimate a domestic-only share of the social cost of greenhouse gases (4, 9). The social cost of greenhouse gases should be regularly updated, especially to reflect the latest evidence about damage functions (10). Meanwhile, government and private sector analysts should continue using IWG’s central estimate of $50 per ton of carbon dioxide with confidence that it is still the best estimate of the social cost of greenhouse gases.
Evaluating Science's open-data policy Dominique G. Roche
Science 357 (6352), 654. DOI: 10.1126/science.aan8158
http://science.sciencemag.org/content/357/6352/654.1
RELATED CONTENT
http://science.sciencemag.org/content/sci/357/6352/654.2.full
REFERENCES
This article cites 3 articles, 0 of which you can access for free http://science.sciencemag.org/content/357/6352/654.1#BIBL
PERMISSIONS
http://www.sciencemag.org/help/reprints-and-permissions
Use of this article is subject to the Terms of Service Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science is a registered trademark of AAAS.
Downloaded from http://science.sciencemag.org/ on August 24, 2017
ARTICLE TOOLS
LET TERS Edited by Jennifer Sills
Evaluating Science’s open-data policy
654
Downloaded from http://science.sciencemag.org/ on August 24, 2017
Students could benefit from STEM programs structured to help them succeed.
a complete and understandable data set that is open to reviewers and, ultimately, becomes open to scientists and the public. Dominique G. Roche Institute of Biology, University of Neuchâtel, 2000, Neuchâtel, Switzerland. Email: [email protected] REFERENCES
1. Science, Editorial policies (www.sciencemag.org/authors/ science-editorial-policies). 2. C. J. Savage, A. J. Vickers, PLOS ONE 4, e7078 (2009). 3. T. Vines et al., Curr. Biol. 24, 94 (2014). 4. D. G. Roche, L. E. B. Kruuk, R. Lanfear, S. A. Binning, PLOS Biol. 13, e1002295 (2015).
10.1126/science.aan8158
Editor’s note Before acceptance of a paper at Science, all authors must sign a statement affirming that “all data necessary for a reader to understand and evaluate the conclusions of the paper are included in the paper or its supplement or will be archived in an approved database and made available to any reader.” In the case of the retracted Report to which Roche refers in his Letter, both authors signed the statement. It became apparent only after publication that the data in the supplementary material were incomplete, that they had not been archived, and that the only copy of the raw observations on which the findings were based was allegedly stored on a computer that had been stolen. Our policies are intended to ensure that all appropriate data are available to our readers, and we are actively examining ways to make these policies more effective. Jeremy Berg Editor-in-Chief
10.1126/science.aao5141
Helping less-prepared students excel Many minority-serving institutions have had great success in graduating students in science, technology, engineering, and math (STEM) disciplines, but this hasn’t made a substantial dent in the numbers of minorities in these fields. Making a real difference will require universities to change their approach. STEM programs often try to weed out students who are deemed “less prepared” or who have not demonstrated that they are capable of succeeding in the most demanding courses. Instead of providing support, university advisers may encourage these students to seek less demanding majors. Faculty members lack incentives to expend extra effort to help less-prepared students catch up to their peers. It is perceived by many to be much easier, and more prestigious, to work with students who immediately excel in their coursework. Many less-prepared students, including a large percentage of minority STEM students, would benefit instead from policies that have as their goal keeping them in STEM. For example, some primarily undergraduate institutions with small departments [e.g., (1)] encourage all students to initially engage in research projects that not only expose them to “real” science, but also provide close mentoring by full-time faculty. Some students are allowed to take lighter course loads if they are actively engaged in research, thus extending the time to attain a degree. The Engineering Augmented Degree Program (ENGAGE) at the School of Engineering at the University of Pretoria sciencemag.org SCIENCE
18 AUGUST 2017 • VOL 357 ISSUE 6352
Published by AAAS
PHOTO: FATCAMERA/ISTOCKPHOTO
Science must re-evaluate its open-data policy after retracting a controversial study on microplastics and fish in May (“Editorial Retraction,” J. Berg, 26 May, p. 812 and “Fishy business,” M. Enserink, News Feature, 24 March, p. 1254). The only computer containing the study’s raw data was allegedly stolen and no backups existed on another machine or an online repository. Many are left wondering how this could happen in an era of cloud computing and open data. Science currently has a two-step editorial policy to promote data availability and research transparency (1). First, “before publication, large data sets must be deposited in an approved database and an accession number…included in the published paper.” Second, “after publication, all data [and code] necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science.” Unfortunately, two loopholes weaken the effectiveness of this policy. First, it is unclear why “small data sets” are exempt from archival before publication given that they are simple to share and equally important in verifying research results (as the above case demonstrates). Second, research shows that individual scientists cannot be trusted to reliably preserve their data and share them upon request (2, 3). Compounding this problem is the fact that publishers seldom state the consequences of breaching their policies. For example, does failure to provide raw data post-publication lead to the automatic retraction of the paper? The solution to avoid data disappearing is simple: Journals must mandate and enforce data archival on a recognized, online repository at the time of submission. Only editors and reviewers would have access during peer review; the data would be made generally accessible upon publication. Publishing verifiable research is a tenet of scientific progress and, ultimately, journals are responsible for guaranteeing compliance with their open-data policy. At a minimum, this responsibility involves a cursory check of the underlying data and ensuring that all data are available for reviewers to assess (4). Science publishes many papers describing major breakthroughs, but these extraordinary claims must be supported by extraordinary evidence. This includes, first and foremost,
in South Africa (2) is one example of an inclusive program that welcomes all but strives to help those who are less prepared. The ENGAGE curriculum gradually increases the volume of work over five years to help students adjust to life at the university. The students are provided with mentoring and other forms of academic and social support, including peer-to-peer interactions. This program has documented stunning success for students from underrepresented groups (3), such as black students from poor townships, and the concept may merit serious consideration by other colleges and universities. Allen Sessoms The Hollins Group, Washington, DC 20011, USA. Email: [email protected] REFERENCES
10.1126/science.aao4525
R. Revesz,1 M. Greenstone,2 M. Hanemann,3 M. Livermore, 4 T. Sterner,5 D. Grab,1 P. Howard, 1 J. Schwartz1* 1
Best cost estimate of greenhouse gases In March, President Trump’s Executive Order 13783 disbanded the Interagency Working Group on the Social Cost of Greenhouse Gases (IWG) (1). IWG developed estimates for federal agencies to use in cost-benefit analyses of climate policies. IWG’s most recent central estimate was $50 in global damages per ton of carbon dioxide, based on year 2020 emissions, converted from 2007 to 2017 dollars (2). Trump’s Executive Order withdrew IWG’s official valuations and instead instructed agencies to monetize climate effects using “the best available science and economics” (1). Yet IWG’s estimates already are the product of the most widely peer-reviewed models and best available data (3). The Executive Order asks agencies to reconsider “appropriate discount rates” (the factor for converting future costs and benefits into present-day values) and “domestic versus international impacts” (1). These instructions implicitly question IWG’s choices to base central estimates on a 3% instead of a 7% discount rate (higher discount rates place less value on avoiding future damages) and to value global damages rather than ignore climate effects beyond U.S. borders. However, scientists and economists widely endorse these methodological choices. The National
Institute for Policy Integrity, New York University School of Law, New York, NY 10012, USA. 2 Department of Economics, University of Chicago, Chicago, IL 60637, USA. 3Department of Agriculture and Resource Economics, University of California– Berkeley, Berkeley, CA 94720-3310, USA. 4University of Virginia School of Law, Charlottesville, VA 229031738, USA. 5Department of Economics, University of Gothenburg, Gothenburg, SE-405 30, Sweden. *Corresponding author. Email: [email protected] REFERENCES
1. Executive Office of the President, “Executive Order 13783: Promoting energy independence and economic growth,” Federal Register 82, 16093 (2017). 2. U.S. Interagency Working Group on the Social Cost of Greenhouse Gases (IWG), “Technical Support Document: Technical Update of the Social Cost of Carbon for Regulatory Impact Analysis Under Executive Order 12866” (2016); https://obamawhitehouse.archives. gov/sites/default/files/omb/inforeg/scc-tsd-finaljuly-2015.pdf. 3. M. Greenstone, et al., Rev. Environ. Econ. Pol. 7, 23 (2013). 4. National Academies of Sciences, “Valuing Climate Damages: Updating Estimation of the Social Cost of Carbon Dioxide” (National Academies Press, 2017); www.nap.edu/download/24651. 5. U.S. Council of Economic Advisers, “Discounting for public policy: Theory and recent evidence on the merits of updating the discount rate” (2017); https://obamawhitehouse.archives.gov/sites/default/files/page/ files/201701_cea_discounting_issue_brief.pdf. 6. P. Howard, D. Sylvan, “The Wisdom of the Economic Crowd: Calibrating Integrated Assessment Models Using Consensus” (2016); http://ageconsearch.umn.edu/ bitstream/235639/2/HowardSylvan_AAEA2016.pdf. 7. M. Greenstone, “What Financial Markets Can Teach Us About Managing Climate Risks,” New York Times (4 April 2017). 8. A. Fraas et al., Science 351, 569 (2016). 9. P. Howard, J. Schwartz, Columbia J. Environ. Law 42, 203 (2017). 10. A. Barreca et al., J. Polit. Econ. 124 , 105 (2016).
10.1126/science.aao4322
SCIENCE sciencemag.org
Published by AAAS
Downloaded from http://science.sciencemag.org/ on August 24, 2017
1. Lafayette, Student Research (https://ece.lafayette.edu/ opportunities/student-research/). 2. University of Pretoria, Engineering Augmented Degree Program (www.up.ac.za/engage-program). 3. D. Grayson, in Proceedings of the First Biennial Conference of the South African Society for Engineering Education, B. Collier-Reed, Ed. (Stellenbosch South Africa, 2011), pp. 69–79.
Academies of Sciences and the U.S. Council of Economic Advisers (4, 5) strongly support a 3% or lower discount rate for intergenerational effects. A 7% rate based on private capital returns is considered inappropriate because the risk profiles of climate effects differ from private investments (6, 7). Most economists and climate policy experts [though not all (8)] also defend valuing the full global externalities of U.S. emissions to reinforce reciprocal climate policies in other countries (3, 4, 9). Moreover, current models cannot accurately estimate a domestic-only share of the social cost of greenhouse gases (4, 9). The social cost of greenhouse gases should be regularly updated, especially to reflect the latest evidence about damage functions (10). Meanwhile, government and private sector analysts should continue using IWG’s central estimate of $50 per ton of carbon dioxide with confidence that it is still the best estimate of the social cost of greenhouse gases.
Evaluating Science's open-data policy Dominique G. Roche
Science 357 (6352), 654. DOI: 10.1126/science.aan8158
http://science.sciencemag.org/content/357/6352/654.1
RELATED CONTENT
http://science.sciencemag.org/content/sci/357/6352/654.2.full
REFERENCES
This article cites 3 articles, 0 of which you can access for free http://science.sciencemag.org/content/357/6352/654.1#BIBL
PERMISSIONS
http://www.sciencemag.org/help/reprints-and-permissions
Use of this article is subject to the Terms of Service Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science is a registered trademark of AAAS.
Downloaded from http://science.sciencemag.org/ on August 24, 2017
ARTICLE TOOLS
