Recommendations for accelerating open preprint peer review to improve the culture of science.
Ontology highlight
ABSTRACT:
SUBMITTER: Avissar-Whiting M
PROVIDER: S-EPMC10903809 | biostudies-literature | 2024 Feb
REPOSITORIES: biostudies-literature
Publications
Recommendations for accelerating open preprint peer review to improve the culture of science.
Avissar-Whiting Michele M Belliard Frédérique F Bertozzi Stefano M SM Brand Amy A Brown Katherine K Clément-Stoneham Géraldine G Dawson Stephanie S Dey Gautam G Ecer Daniel D Edmunds Scott C SC Farley Ashley A Fischer Tara D TD Franko Maryrose M Fraser James S JS Funk Kathryn K Ganier Clarisse C Harrison Melissa M Hatch Anna A Hazlett Haley H Hindle Samantha S Hook Daniel W DW Hurst Phil P Kamoun Sophien S Kiley Robert R Lacy Michael M MM LaFlamme Marcel M Lawrence Rebecca R Lemberger Thomas T Leptin Maria M Lumb Elliott E MacCallum Catriona J CJ Marcum Christopher Steven CS Marinello Gabriele G Mendonça Alex A Monaco Sara S Neves Kleber K Pattinson Damian D Polka Jessica K JK Puebla Iratxe I Rittman Martyn M Royle Stephen J SJ Saderi Daniela D Sever Richard R Shearer Kathleen K Spiro John E JE Stern Bodo B Taraborelli Dario D Vale Ron R Vasquez Claudia G CG Waltman Ludo L Watt Fiona M FM Weinberg Zara Y ZY Williams Mark M
PLoS biology 20240229 2
Peer review is an important part of the scientific process, but traditional peer review at journals is coming under increased scrutiny for its inefficiency and lack of transparency. As preprints become more widely used and accepted, they raise the possibility of rethinking the peer-review process. Preprints are enabling new forms of peer review that have the potential to be more thorough, inclusive, and collegial than traditional journal peer review, and to thus fundamentally shift the culture o ...[more]
Similar Datasets
Project description:One of the foundations of the scientific method is to be able to reproduce experiments and corroborate the results of research that has been done before. However, with the increasing complexities of new technologies and techniques, coupled with the specialisation of experiments, reproducing research findings has become a growing challenge. Clearly, scientific methods must be conveyed succinctly, and with clarity and rigour, in order for research to be reproducible. Here, we propose steps to help increase the transparency of the scientific method and the reproducibility of research results: specifically, we introduce a peer-review oath and accompanying manifesto. These have been designed to offer guidelines to enable reviewers (with the minimum friction or bias) to follow and apply open science principles, and support the ideas of transparency, reproducibility and ultimately greater societal impact. Introducing the oath and manifesto at the stage of peer review will help to check that the research being published includes everything that other researchers would need to successfully repeat the work. Peer review is the lynchpin of the publishing system: encouraging the community to consciously (and conscientiously) uphold these principles should help to improve published papers, increase confidence in the reproducibility of the work and, ultimately, provide strategic benefits to authors and their institutions.
| S-EPMC4304228 | biostudies-literature
Project description:Science is undergoing rapid change with the movement to improve science focused largely on reproducibility/replicability and open science practices. This moment of change-in which science turns inward to examine its methods and practices-provides an opportunity to address its historic lack of diversity and noninclusive culture. Through network modeling and semantic analysis, we provide an initial exploration of the structure, cultural frames, and women's participation in the open science and reproducibility literatures (n = 2,926 articles and conference proceedings). Network analyses suggest that the open science and reproducibility literatures are emerging relatively independently of each other, sharing few common papers or authors. We next examine whether the literatures differentially incorporate collaborative, prosocial ideals that are known to engage members of underrepresented groups more than independent, winner-takes-all approaches. We find that open science has a more connected, collaborative structure than does reproducibility. Semantic analyses of paper abstracts reveal that these literatures have adopted different cultural frames: open science includes more explicitly communal and prosocial language than does reproducibility. Finally, consistent with literature suggesting the diversity benefits of communal and prosocial purposes, we find that women publish more frequently in high-status author positions (first or last) within open science (vs. reproducibility). Furthermore, this finding is further patterned by team size and time. Women are more represented in larger teams within reproducibility, and women's participation is increasing in open science over time and decreasing in reproducibility. We conclude with actionable suggestions for cultivating a more prosocial and diverse culture of science.
| S-EPMC7533847 | biostudies-literature
Project description:Mounting evidence indicates that worldwide, innovation systems are increasing unsustainable. Equally, concerns about inequities in the science and innovation process, and in access to its benefits, continue. Against a backdrop of growing health, economic and scientific challenges global stakeholders are urgently seeking to spur innovation and maximize the just distribution of benefits for all. Open Science collaboration (OS) - comprising a variety of approaches to increase open, public, and rapid mobilization of scientific knowledge - is seen to be one of the most promising ways forward. Yet, many decision-makers hesitate to construct policy to support the adoption and implementation of OS without access to substantive, clear and reliable evidence. In October 2017, international thought-leaders gathered at an Open Science Leadership Forum in the Washington DC offices of the Bill and Melinda Gates Foundation to share their views on what successful Open Science looks like. Delegates from developed and developing nations, national governments, science agencies and funding bodies, philanthropy, researchers, patient organizations and the biotechnology, pharma and artificial intelligence (AI) industries discussed the outcomes that would rally them to invest in OS, as well as wider issues of policy and implementation. This first of two reports, summarizes delegates' views on what they believe OS will deliver in terms of research, innovation and social impact in the life sciences. Through open and collaborative process over the next months, we will translate these success outcomes into a toolkit of quantitative and qualitative indicators to assess when, where and how open science collaborations best advance research, innovation and social benefit. Ultimately, this work aims to develop and openly share tools to allow stakeholders to evaluate and re-invent their innovation ecosystems, to maximize value for the global public and patients, and address long-standing questions about the mechanics of innovation.
| S-EPMC7839829 | biostudies-literature
Project description:The traditional scientific paper falls short of effectively communicating computational research. To help improve this situation, we propose a system by which the computational workflows underlying research articles are checked. The CODECHECK system uses open infrastructure and tools and can be integrated into review and publication processes in multiple ways. We describe these integrations along multiple dimensions (importance, who, openness, when). In collaboration with academic publishers and conferences, we demonstrate CODECHECK with 25 reproductions of diverse scientific publications. These CODECHECKs show that asking for reproducible workflows during a collaborative review can effectively improve executability. While CODECHECK has clear limitations, it may represent a building block in Open Science and publishing ecosystems for improving the reproducibility, appreciation, and, potentially, the quality of non-textual research artefacts. The CODECHECK website can be accessed here: https://codecheck.org.uk/.
| S-EPMC8311796 | biostudies-literature
Project description:Clear and findable publishing policies are important for authors to choose appropriate journals for publication. We investigated the clarity of policies of 171 major academic journals across disciplines regarding peer review and preprinting. 31.6% of journals surveyed do not provide information on the type of peer review they use. Information on whether preprints can be posted or not is unclear in 39.2% of journals. 58.5% of journals offer no clear information on whether reviewer identities are revealed to authors. Around 75% of journals have no clear policy on co-reviewing, citation of preprints, and publication of reviewer identities. Information regarding practices of open peer review is even more scarce, with <20% of journals providing clear information. Having found a lack of clear information, we conclude by examining the implications this has for researchers (especially early career) and the spread of open research practices.
| S-EPMC7577440 | biostudies-literature
Project description:Areas of open science (OS) policy and practice are already relatively well-advanced in several countries and sectors through the initiatives of some governments, funders, philanthropy, researchers and the community. Nevertheless, the current research and innovation system, including in the focus of this report, the life sciences, remains weighted against OS. In October 2017, thought-leaders from across the world gathered at an Open Science Leadership Forum in the Washington DC office of the Bill and Melinda Gates Foundation to share their views on what successful OS looks like. We focused on OS partnerships as this is an emerging model that aims to accelerate science and innovation. These outcomes are captured in a first meeting report: Defining Success in Open Science. On several occasions, these conversations turned to the challenges that must be addressed and new policies required to effectively and sustainably advance OS practice. Thereupon, in this report, we describe the concerns raised and what is needed to address them supplemented by our review of the literature, and suggest the stakeholder groups that may be best placed to begin to take action. It emerges that to be successful, OS will require the active engagement of all stakeholders: while the research community must develop research questions, identify partners and networks, policy communities need to create an environment that is supportive of experimentation by removing barriers. This report aims to contribute to ongoing discussions about OS and its implementation. It is also part of a step-wise process to develop and mobilize a toolkit of quantitative and qualitative indicators to assist global stakeholders in implementing high value OS collaborations. Currently in co-development through an open and international process, this set of measures will allow the generation of needed evidence on the influence of OS partnerships on research, innovation, and critical social and economic goals.
| S-EPMC7845503 | biostudies-literature
Project description:Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes. But what is meant by `open peer review', and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for? Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues. We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.
| S-EPMC6073088 | biostudies-other
Project description:Over the past 20 years, a series of reports written by groups of senior researchers and administrators have recommended changes to improve the training environments for graduate students and postdoctoral researchers in the United States. However, academic institutions and departments have largely failed to implement these recommendations, which has exacerbated the problems faced by these trainees. Here, based on input from trainees at different career stages, we outline seven practical changes that academic institutions and departments can make to improve their training environments.
| S-EPMC7413740 | biostudies-literature
Project description:The Draft National Open Science Policy, which was shared by the South African government with stakeholders in 2022, is an encouraging step forward as it aims to promote the practice of open science in South Africa through a system of incentives. Since South Africa is constitutionally committed to be an open and democratic society, this approach is preferable to the approach of state control that characterizes the Draft National Policy on Data and Cloud-another data-related policy initiative by the South African government. However, there is room for improvement in the Draft National Open Science Policy. In particular, it should: (a) rely on the right to freedom of scientific research to strengthen the policy; (b) rectify the omission of ownership from its policy analysis; and (c) retain a clear differentiation between human and non-human genetic data. This will ensure that the final policy is clearly anchored in the South African Constitution, and that the principle of "as open as possible, as closed as necessary" can be applied to human genetic data in a legally well informed and accountable way.
| S-EPMC10577200 | biostudies-literature
Project description:BackgroundAs the production of scientific manuscripts and journal options both increase, the peer review process remains at the center of quality control. Recent advances in understanding reviewer biases and behaviors along with electronic manuscript handling records have allowed unprecedented investigations into the peer review process.MethodsWe examined a sample of six journals within the field of fisheries science (and all published by the American Fisheries Society) specifically looking for changes in reviewer invitation rates, review time, patterns of reviewer agreements, and rejection rates relative to different forms of blinding.ResultsData from 6,606 manuscripts from 2011-2021 showed significant increases in reviewer invitations. Specifically, four journals showed statistically significant increases in reviewer invitations while two showed no change. Review times changed relatively little (± 2 weeks), and we found no concerning patterns in reviewer agreement. However, we documented a consistently higher rejection rate-around 20% higher-of double-blinded manuscripts when compared to single-blinded manuscripts.ConclusionsOur findings likely represent broader trends across fisheries science publications, and possibly extend to other life science disciplines. Because peer review remains a primary tool for scientific quality control, authors and editors are encouraged to understand the process and evaluate its performance at whatever level can help in the creation of trusted science. Minimally, our findings can help the six journals we investigated to better understand and improve their peer review processes.
| S-EPMC11197202 | biostudies-literature