Why publishing everything is more effective than selective publishing of statistically significant results.
ABSTRACT: BACKGROUND: De Winter and Happee examined whether science based on selective publishing of significant results may be effective in accurate estimation of population effects, and whether this is even more effective than a science in which all results are published (i.e., a science without publication bias). Based on their simulation study they concluded that "selective publishing yields a more accurate meta-analytic estimation of the true effect than publishing everything, (and that) publishing nonreplicable results while placing null results in the file drawer can be beneficial for the scientific collective" (p.4). METHODS AND FINDINGS: Using their scenario with a small to medium population effect size, we show that publishing everything is more effective for the scientific collective than selective publishing of significant results. Additionally, we examined a scenario with a null effect, which provides a more dramatic illustration of the superiority of publishing everything over selective publishing. CONCLUSION: Publishing everything is more effective than only reporting significant outcomes.
Project description:Concerns exist within the medical and psychological sciences that many published research findings are not replicable. Guidelines accordingly recommend that the file drawer effect should be eliminated and that statistical significance should not be a criterion in the decision to submit and publish scientific results. By means of a simulation study, we show that selectively publishing effects that differ significantly from the cumulative meta-analytic effect evokes the Proteus phenomenon of poorly replicable and alternating findings. However, the simulation also shows that the selective publication approach yields a scientific record that is content rich as compared to publishing everything, in the sense that fewer publications are needed for obtaining an accurate meta-analytic estimation of the true effect. We conclude that, under the assumption of self-correcting science, the file drawer effect can be beneficial for the scientific collective.
Project description:Major changes are afoot in the world of academic publishing, exemplified by innovations in publishing platforms, new approaches to metrics, improvements in our approach to peer review, and a focus on developing and encouraging open access to scientific literature and data. The FAIR acronym recommends that authors and publishers should aim to make their output Findable, Accessible, Interoperable and Reusable. In this opinion article, I explore the parallel view that we should take a collective stance on making the dissemination of scientific data fair in the conventional sense, by being mindful of equity and justice for patients, clinicians, academics, publishers, funders and academic institutions. The views I represent are founded on oral and written dialogue with clinicians, academics and the publishing industry. Further progress is needed to improve collaboration and dialogue between these groups, to reduce misinterpretation of metrics, to minimise inequity that arises as a consequence of geographic setting, to improve economic sustainability, and to broaden the spectrum, scope, and diversity of scientific publication.
Project description:Scientific societies provide numerous services to the scientific enterprise, including convening meetings, publishing journals, developing scientific programs, advocating for science, promoting education, providing cohesion and direction for the discipline, and more. For most scientific societies, publishing provides revenues that support these important activities. In recent decades, the proportion of papers on microbiology published in scientific society journals has declined. This is largely due to two competing pressures: authors' drive to publish in "glam journals"-those with high journal impact factors-and the availability of "mega journals," which offer speedy publication of articles regardless of their potential impact. The decline in submissions to scientific society journals and the lack of enthusiasm on the part of many scientists to publish in them should be matters of serious concern to all scientists because they impact the service that scientific societies can provide to their members and to science.
Project description:To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees' willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.
Project description:The way science and research is done is rapidly becoming more open and collaborative. The traditional way of publishing new findings in journals is becoming increasingly outdated and no longer serves the needs of much of science. Whilst preprints can bring significant benefits of removing delay and selection, they do not go far enough if simply implemented alongside the existing journal system. We propose that we need a new approach, an Open Science Platform, that takes the benefits of preprints but adds formal, invited, and transparent post-publication peer review. This bypasses the problems of the current journal system and, in doing so, moves the evaluation of research and researchers away from the journal-based Impact Factor and towards a fairer system of article-based qualitative and quantitative indicators. In the long term, it should be irrelevant where a researcher publishes their findings. What is important is that research is shared and made available without delay within a framework that encourages quality standards and requires all players in the research community to work as collaborators.
Project description:Advances in the health sciences rely on sharing research and data through publication. As information professionals are often asked to contribute their knowledge to assist clinicians and researchers in selecting journals for publication, the authors recognized an opportunity to build a decision support tool, SPI-Hub: Scholarly Publishing Information Hub™, to capture the team's collective publishing industry knowledge, while carefully retaining the quality of service. SPI-Hub's decision support functionality relies on a data framework that describes journal publication policies and practices through a newly designed metadata structure, the Knowledge Management Journal Record™. Metadata fields are populated through a semi-automated process that uses custom programming to access content from multiple sources. Each record includes 25 metadata fields representing best publishing practices. Currently, the database includes more than 24,000 health sciences journal records. To correctly capture the resources needed for both completion and future maintenance of the project, the team conducted an internal study to assess time requirements for completing records through different stages of automation. The journal decision support tool, SPI-Hub, provides an opportunity to assess publication practices by compiling data from a variety of sources in a single location. Automated and semi-automated approaches have effectively reduced the time needed for data collection. Through a comprehensive knowledge management framework and the incorporation of multiple quality points specific to each journal, SPI-Hub provides prospective users with both recommendations for publication and holistic assessment of the trustworthiness of journals in which to publish research and acquire trusted knowledge.
Project description:Publishing peer review materials alongside research articles promises to make the peer review process more transparent as well as making it easier to recognise these contributions and give credit to peer reviewers. Traditionally, the peer review reports, editors letters and author responses are only shared between the small number of people in those roles prior to publication, but there is a growing interest in making some or all of these materials available. A small number of journals have been publishing peer review materials for some time, others have begun this practice more recently, and significantly more are now considering how they might begin. This article outlines the outcomes from a recent workshop among journals with experience in publishing peer review materials, in which the specific operation of these workflows, and the challenges, were discussed. Here, we provide a draft as to how to represent these materials in the JATS and Crossref data models to facilitate the coordination and discoverability of peer review materials, and seek feedback on these initial recommendations.
Project description:Introduction:Open access (OA) publishing rates have risen dramatically in the biomedical sciences in the past decade. However, few studies have focused on the publishing activities and attitudes of early career researchers. The aim of this study was to examine current publishing activities of clinical and research fellows and their perceptions of OA publishing and public access. Methods:This study employed a mixed methods approach. Data on publications authored by Memorial Sloan Kettering Cancer Center fellows between 2013 and 2018 were collected via an in-house author profile system and citation indexes. Journals were categorized according to SHERPA/RoMEO classifications. In-person and telephone interviews were conducted with fifteen fellows to discern their perceptions of OA publishing. Results:The total percentage of fellows' publications that were freely available OA was 28.6%, with a relatively flat rate between 2013 and 2018. Publications with fellows as first authors were significantly more likely to be OA. Fellows cited high article processing charges (APCs) and perceived lack of journal quality or prestige as barriers to OA publishing. Fellows generally expressed support for the National Institutes of Health (NIH) public access policy. Conclusions:While the fellows in this study acknowledged the potential of OA to aid in research dissemination, they also expressed hesitation to publish OA related to confusion surrounding legitimate OA and predatory publications and frustration with APCs. Fellows supported the NIH public access policy and accepted it as part of their research process. Health sciences information professionals could potentially leverage this acceptance of public access to advocate for OA publishing.
Project description:The success of a scientist depends on their production of scientific papers and the impact factor of the journal in which they publish. Because most major scientific journals are published in English, success is related to publishing in this language. Currently, 98% of publications in science are written in English, including researchers from English as a Foreign Language (EFL) countries. Colombia is among the countries with the lowest English proficiency in the world. Thus, understanding the disadvantages that Colombians face in publishing is crucial to reducing global inequality in science. This paper quantifies the disadvantages that result from the language hegemony in scientific publishing by examining the additional costs that communicating in English creates in the production of articles. It was identified that more than 90% of the scientific articles published by Colombian researchers are in English, and that publishing in a second language creates additional financial costs to Colombian doctoral students and results in problems with reading comprehension, writing ease and time, and anxiety. Rejection or revision of their articles because of the English grammar was reported by 43.5% of the doctoral students, and 33% elected not to attend international conferences and meetings due to the mandatory use of English in oral presentations. Finally, among the translation/editing services reviewed, the cost per article is between one-quarter and one-half of a doctoral monthly salary in Colombia. Of particular note, we identified a positive correlation between English proficiency and higher socioeconomic origin of the researcher. Overall, this study exhibits the negative consequences of hegemony of English that preserves the global gap in science. Although having a common language is important for science communication, generating multilinguistic alternatives would promote diversity while conserving a communication channel. Such an effort should come from different actors and should not fall solely on EFL researchers.
Project description:Experience plays a critical role in crafting high-impact scientific work. This is particularly evident in top multidisciplinary journals, where a scientist is unlikely to appear as senior author if he or she has not previously published within the same journal. Here, we develop a quantitative understanding of author order by quantifying this "chaperone effect," capturing how scientists transition into senior status within a particular publication venue. We illustrate that the chaperone effect has a different magnitude for journals in different branches of science, being more pronounced in medical and biological sciences and weaker in natural sciences. Finally, we show that in the case of high-impact venues, the chaperone effect has significant implications, specifically resulting in a higher average impact relative to papers authored by new principal investigators (PIs). Our findings shed light on the role played by experience in publishing within specific scientific journals, on the paths toward acquiring the necessary experience and expertise, and on the skills required to publish in prestigious venues.