This article was downloaded by: [University of Louisville] On: 27 January 2015, At: 07:27 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The Journal of Social Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vsoc20
Announcing Open Science Badges and Reaching for the Sky Jon E. Grahe PhD
a
a
Executive Editor, JSP Published online: 13 Dec 2013.
Click for updates To cite this article: Jon E. Grahe PhD (2014) Announcing Open Science Badges and Reaching for the Sky, The Journal of Social Psychology, 154:1, 1-3, DOI: 10.1080/00224545.2014.853582 To link to this article: http://dx.doi.org/10.1080/00224545.2014.853582
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [University of Louisville] at 07:27 27 January 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
The Journal of Social Psychology, 154: 1–3, 2014 Copyright © Taylor & Francis Group, LLC ISSN: 0022-4545 print / 1940-1183 online DOI: 10.1080/00224545.2014.853582
Downloaded by [University of Louisville] at 07:27 27 January 2015
EDITORIAL
Announcing Open Science Badges and Reaching for the Sky The Journal of Social Psychology is the longest running journal publishing research focused on this sub-discipline. JSP’s mission has always been to provide empirically based theoretical contributions to the discipline of the highest quality. Though our tradition is rich, we have not always met these idealistic standards. Beginning in 2000 with the significant changes to the executive editorial and consulting editorial review boards, we have worked toward the goal of increasing the publication standards of the Journal and, thus, the quality and impact of research published in JSP. We are ready to take another leap forward. As you hold this issue in your hands, you will notice that it has a new size and layout. The timing of this external change also marks an announcement aimed at taking another leap forward in our goal toward again being a leading social psychological journal. We invite you to leap with us, and maybe together we can grab a piece of sky. In the past two years, a culminating chorus of voices have called for new ways of keeping science accountable. Some of these calls were driven by concerns that there are too many published studies with Type I errors across all fields of science (Ioannidis, 2005; Simmons, Nelson, & Simonson, 2011). Simmons et al. argued that the reason may lie in there being too much flexibility in how we collect and analyze our data. They included suggestions for minimum numbers of participants per cell and clarity in decisions regarding hypothesis testing. Other voices of concern followed when several social psychologists were accused of falsifying data and fears escalated that unethical behavior was more common than previously anticipated. A growing number of failures to replicate some well-known findings led to concerns about priming research, prompting prominent psychologist and Nobel laureate Daniel Kahneman to suggest novel methods of validating research findings (Yong, 2012). In November 2012, a special issue of Perspectives on Psychological Science was dedicated to addressing this “crisis of confidence” in the discipline. This collection of articles reviewed causes of the crisis and potential solutions and the Association for Psychological Science quickly declared the issue free so that access was not a barrier for researchers to start applying the suggestions. In one article Nosek, Spies, and Moytl (2012) discussed both the causes of impediments to good scientific practices, such as interest in novel papers and the academic’s eternal need to publish, as well as a number of idealistic actions and goals to improve them, such as rewarding novel research paradigms and changing publication practices for authors, reviewers, and editors. Their “Ultimate Solution” to these problems was to make scientific practices more open.
Downloaded by [University of Louisville] at 07:27 27 January 2015
2
THE JOURNAL OF SOCIAL PSYCHOLOGY
Making good on their suggestions, the Center for Open Science (COS) was formed with the goal of providing tools to make our practices more open. The COS asserts, as we do, that scientists are overwhelmingly earnest and ethical. Rather than focus on the minority of events that alarmed the field, we focus instead on the potential improvements to science made possible by novel technology. The COS harnessed this technology for their Open Science Framework software platform that allows researchers to easily upload files, such as materials and data, and to time stamp their work. Further, the platform enables researchers to freely share these with colleagues, privately or publicly, to help support a further function which is to facilitate collaboration amongst researchers. For example, the Open Science Collaboration (OSC) involves over 150 researchers attempting to replicate all of the studies published in the first three months of 2008 in three journals as part of the Replication Project (Open Science Collaboration, 2012). The OSC through the support of the COS has also created a new initiative to offer Open Science Badges (https:// openscienceframework.org/project/TVyXZ/wiki/home). These badges are intended to encourage researchers to follow idealistic standards in their research. The Journal of Social Psychology has decided to become a charter adopter and we will begin to offer three badges in the future issues: Open Data, Open Materials, and Preregistered. Though the badges are more fully described at the COS website, a brief summary follows: • the Open Data Badge involves making publicly available the dataset necessary to reproduce reported results; • the Open Materials Badge involves making publicly available the materials necessary for the study to be directly replicated; and • the Preregistered Badge involves making publicly available a preregistered design and analysis plan that can be compared with the published results. We will at first offer “self-disclosure badges” which require only a simple check of the public availability of relevant files. Details are available at the following website: https:// openscienceframework.org/project/TVyXZ/wiki\home. Although we firmly believe in the value of open science practices, we would like to stress that any decisions to publish or not to publish a particular article in The Journal of Social Psychology will remain independent of the authors’ decision to apply for any of the Open Science Badges, which are voluntary and not required. We hope that the majority of our authors will share our desire for open science practices, however we understand that there may be practical reasons why badges are not feasible for all authors or studies. We want the papers published in this journal to be impactful and we believe that published studies carrying these badges will meet this expectation. Finally, let us restate the main purpose for introducing Open Science Badges. We do not believe that fraud and deceptive practices are widespread in social psychology and our goal is not to further police the field. Instead, we wish to encourage the sharing of more complete information about studies, including information that will allow readers to draw more or less confidence about the findings presented. For example, pre-registration can help readers to differentiate a priori predictions from those that might be more exploratory (and in need of replication). Sharing data and materials help our science to become more cumulative, allowing for more meta-analyses and systematic replications. This will help to ensure that our science is self-correcting and that failures to replicate become fewer (or at least more widely known). We believe that it is possible that, upon looking back at this moment in 50 or 100 years, we will note that the practices of sharing data, materials, and a priori predictions marked the start of a distinct change in our
GRAHE
3
field’s ability to make good on its clear potential to make contributions to science and to society. We hope that researchers agree and will help us to achieve these laudable goals. Jon E. Grahe, PhD Executive Editor, JSP
Downloaded by [University of Louisville] at 07:27 27 January 2015
REFERENCES Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Med 2(8), e124. Nosek, B. A., Spies, J. R., & Moytl, M. (2012). Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7, 615–631. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366 doi:10.1177/0956797611417632 Yong, E. (2012, October 3). Nobel laureate challenges psychologists to clean up their act: Social-priming research needs “daisy chain” of replication. Nature. Retrieved from http://www.nature.com/news/. doi:10.1038/nature.2012.11535
The Journal of Social Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vsoc20
Announcing Open Science Badges and Reaching for the Sky Jon E. Grahe PhD
a
a
Executive Editor, JSP Published online: 13 Dec 2013.
Click for updates To cite this article: Jon E. Grahe PhD (2014) Announcing Open Science Badges and Reaching for the Sky, The Journal of Social Psychology, 154:1, 1-3, DOI: 10.1080/00224545.2014.853582 To link to this article: http://dx.doi.org/10.1080/00224545.2014.853582
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [University of Louisville] at 07:27 27 January 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
The Journal of Social Psychology, 154: 1–3, 2014 Copyright © Taylor & Francis Group, LLC ISSN: 0022-4545 print / 1940-1183 online DOI: 10.1080/00224545.2014.853582
Downloaded by [University of Louisville] at 07:27 27 January 2015
EDITORIAL
Announcing Open Science Badges and Reaching for the Sky The Journal of Social Psychology is the longest running journal publishing research focused on this sub-discipline. JSP’s mission has always been to provide empirically based theoretical contributions to the discipline of the highest quality. Though our tradition is rich, we have not always met these idealistic standards. Beginning in 2000 with the significant changes to the executive editorial and consulting editorial review boards, we have worked toward the goal of increasing the publication standards of the Journal and, thus, the quality and impact of research published in JSP. We are ready to take another leap forward. As you hold this issue in your hands, you will notice that it has a new size and layout. The timing of this external change also marks an announcement aimed at taking another leap forward in our goal toward again being a leading social psychological journal. We invite you to leap with us, and maybe together we can grab a piece of sky. In the past two years, a culminating chorus of voices have called for new ways of keeping science accountable. Some of these calls were driven by concerns that there are too many published studies with Type I errors across all fields of science (Ioannidis, 2005; Simmons, Nelson, & Simonson, 2011). Simmons et al. argued that the reason may lie in there being too much flexibility in how we collect and analyze our data. They included suggestions for minimum numbers of participants per cell and clarity in decisions regarding hypothesis testing. Other voices of concern followed when several social psychologists were accused of falsifying data and fears escalated that unethical behavior was more common than previously anticipated. A growing number of failures to replicate some well-known findings led to concerns about priming research, prompting prominent psychologist and Nobel laureate Daniel Kahneman to suggest novel methods of validating research findings (Yong, 2012). In November 2012, a special issue of Perspectives on Psychological Science was dedicated to addressing this “crisis of confidence” in the discipline. This collection of articles reviewed causes of the crisis and potential solutions and the Association for Psychological Science quickly declared the issue free so that access was not a barrier for researchers to start applying the suggestions. In one article Nosek, Spies, and Moytl (2012) discussed both the causes of impediments to good scientific practices, such as interest in novel papers and the academic’s eternal need to publish, as well as a number of idealistic actions and goals to improve them, such as rewarding novel research paradigms and changing publication practices for authors, reviewers, and editors. Their “Ultimate Solution” to these problems was to make scientific practices more open.
Downloaded by [University of Louisville] at 07:27 27 January 2015
2
THE JOURNAL OF SOCIAL PSYCHOLOGY
Making good on their suggestions, the Center for Open Science (COS) was formed with the goal of providing tools to make our practices more open. The COS asserts, as we do, that scientists are overwhelmingly earnest and ethical. Rather than focus on the minority of events that alarmed the field, we focus instead on the potential improvements to science made possible by novel technology. The COS harnessed this technology for their Open Science Framework software platform that allows researchers to easily upload files, such as materials and data, and to time stamp their work. Further, the platform enables researchers to freely share these with colleagues, privately or publicly, to help support a further function which is to facilitate collaboration amongst researchers. For example, the Open Science Collaboration (OSC) involves over 150 researchers attempting to replicate all of the studies published in the first three months of 2008 in three journals as part of the Replication Project (Open Science Collaboration, 2012). The OSC through the support of the COS has also created a new initiative to offer Open Science Badges (https:// openscienceframework.org/project/TVyXZ/wiki/home). These badges are intended to encourage researchers to follow idealistic standards in their research. The Journal of Social Psychology has decided to become a charter adopter and we will begin to offer three badges in the future issues: Open Data, Open Materials, and Preregistered. Though the badges are more fully described at the COS website, a brief summary follows: • the Open Data Badge involves making publicly available the dataset necessary to reproduce reported results; • the Open Materials Badge involves making publicly available the materials necessary for the study to be directly replicated; and • the Preregistered Badge involves making publicly available a preregistered design and analysis plan that can be compared with the published results. We will at first offer “self-disclosure badges” which require only a simple check of the public availability of relevant files. Details are available at the following website: https:// openscienceframework.org/project/TVyXZ/wiki\home. Although we firmly believe in the value of open science practices, we would like to stress that any decisions to publish or not to publish a particular article in The Journal of Social Psychology will remain independent of the authors’ decision to apply for any of the Open Science Badges, which are voluntary and not required. We hope that the majority of our authors will share our desire for open science practices, however we understand that there may be practical reasons why badges are not feasible for all authors or studies. We want the papers published in this journal to be impactful and we believe that published studies carrying these badges will meet this expectation. Finally, let us restate the main purpose for introducing Open Science Badges. We do not believe that fraud and deceptive practices are widespread in social psychology and our goal is not to further police the field. Instead, we wish to encourage the sharing of more complete information about studies, including information that will allow readers to draw more or less confidence about the findings presented. For example, pre-registration can help readers to differentiate a priori predictions from those that might be more exploratory (and in need of replication). Sharing data and materials help our science to become more cumulative, allowing for more meta-analyses and systematic replications. This will help to ensure that our science is self-correcting and that failures to replicate become fewer (or at least more widely known). We believe that it is possible that, upon looking back at this moment in 50 or 100 years, we will note that the practices of sharing data, materials, and a priori predictions marked the start of a distinct change in our
GRAHE
3
field’s ability to make good on its clear potential to make contributions to science and to society. We hope that researchers agree and will help us to achieve these laudable goals. Jon E. Grahe, PhD Executive Editor, JSP
Downloaded by [University of Louisville] at 07:27 27 January 2015
REFERENCES Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Med 2(8), e124. Nosek, B. A., Spies, J. R., & Moytl, M. (2012). Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7, 615–631. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366 doi:10.1177/0956797611417632 Yong, E. (2012, October 3). Nobel laureate challenges psychologists to clean up their act: Social-priming research needs “daisy chain” of replication. Nature. Retrieved from http://www.nature.com/news/. doi:10.1038/nature.2012.11535
