Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues
ABSTRACT: COVID-19-related (vs. non-related) articles appear to be more expeditiously processed and published in peer-reviewed journals. We aimed to evaluate: (i) whether COVID-19-related preprints were favored for publication, (ii) preprinting trends and public discussion of the preprints, and (iii) the relationship between the publication topic (COVID-19-related or not) and quality issues. Manuscripts deposited at bioRxiv and medRxiv between January 1 and September 27 2020 were assessed for the probability of publishing in peer-reviewed journals, and those published were evaluated for submission-to-acceptance time. The extent of public discussion was assessed based on Altmetric and Disqus data. The Retraction Watch Database and PubMed were used to explore the retraction of COVID-19 and non-COVID-19 articles and preprints. With adjustment for the preprinting server and number of deposited versions, COVID-19-related preprints were more likely to be published within 120 days since the deposition of the first version (OR = 1.96, 95% CI: 1.80–2.14) as well as over the entire observed period (OR = 1.39, 95% CI: 1.31–1.48). Submission-to-acceptance was by 35.85 days (95% CI: 32.25–39.45) shorter for COVID-19 articles. Public discussion of preprints was modest and COVID-19 articles were overrepresented in the pool of retracted articles in 2020. Current data suggest a preference for publication of COVID-19-related preprints over the observed period.
The online version contains supplementary material available at 10.1007/s11192-021-04249-7.
Project description:<h4>Background</h4>Preprints are preliminary reports that have not been peer-reviewed. In December 2019, a novel coronavirus appeared in China, and since then, scientific production, including preprints, has drastically increased. In this study, we intend to evaluate how often preprints about COVID-19 were published in scholarly journals and cited.<h4>Methods</h4>We searched the iSearch COVID-19 portfolio to identify all preprints related to COVID-19 posted on bioRxiv, medRxiv, and Research Square from January 1, 2020, to May 31, 2020. We used a custom-designed program to obtain metadata using the Crossref public API. After that, we determined the publication rate and made comparisons based on citation counts using non-parametric methods. Also, we compared the publication rate, citation counts, and time interval from posting on a preprint server to publication in a scholarly journal among the three different preprint servers.<h4>Results</h4>Our sample included 5,061 preprints, out of which 288 were published in scholarly journals and 4,773 remained unpublished (publication rate of 5.7%). We found that articles published in scholarly journals had a significantly higher total citation count than unpublished preprints within our sample (<i>p</i> < 0.001), and that preprints that were eventually published had a higher citation count as preprints when compared to unpublished preprints (<i>p</i> < 0.001). As well, we found that published preprints had a significantly higher citation count after publication in a scholarly journal compared to as a preprint (<i>p</i> < 0.001). Our results also show that medRxiv had the highest publication rate, while bioRxiv had the highest citation count and shortest time interval from posting on a preprint server to publication in a scholarly journal.<h4>Conclusions</h4>We found a remarkably low publication rate for preprints within our sample, despite accelerated time to publication by multiple scholarly journals. These findings could be partially attributed to the unprecedented surge in scientific production observed during the COVID-19 pandemic, which might saturate reviewing and editing processes in scholarly journals. However, our findings show that preprints had a significantly lower scientific impact, which might suggest that some preprints have lower quality and will not be able to endure peer-reviewing processes to be published in a peer-reviewed journal.
Project description:Amid the Coronavirus Disease 2019 (COVID-19) pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press, and policymakers for the first time. This phenomenon has sharpened long-standing questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19–related and 17.2% of COVID-19–related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper. During the COVID-19 pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, but can we trust them? This study reveals that the majority of preprints published within the first 4 months of the pandemic are comparable to their published version.
Project description:The COVID-19 pandemic has unleashed a deluge of publications. For this cross-sectional study we compared the amount and reporting characteristics of COVID-19-related academic articles and preprints and the number of ongoing clinical trials and systematic reviews. To do this, we searched the PubMed database of citations and abstracts for published life science journals by using appropriate combinations of medical subject headings (MeSH terms), and the COVID-19 section of the MedRxiv and BioRxiv archives up to 20 May 2020 (21 weeks). In addition, we searched Clinicaltrial.gov, Chinese Clinical Trial Registry, EU Clinical Trials Register, and 15 other trial registers, as well as PROSPERO, the international prospective register of systematic reviews. The characteristics of each publication were extracted. Regression analyses and Z tests were used to detect publication trends and their relative proportions. A total of 3635 academic publications and 3805 preprints were retrieved. Only 8.6% (n = 329) of the preprints were already published in indexed journals. The number of academic and preprint publications increased significantly over time (p<0.001). Case reports (6% academic vs 0.9% preprints; p<0.001) and letters (17.4% academic vs 0.5% preprints; p<0.001) accounted for a greater share of academic compared to preprint publications. Differently, randomized controlled trials (0.22% vs 0.63%; p<0.001) and systematic reviews (0.08% vs 5%) made up a greater share of the preprints. The relative proportion of clinical studies registered at Clinicaltrials.gov, Chinese Clinical Trial Registry, and EU Clinical Trials Register was 57.9%, 49.5%, and 98.9%, respectively, most of which were still "recruiting". PROSPERO listed 962 systematic review protocols. Preprints were slightly more prevalent than academic articles but both were increasing in number. The void left by the lack of primary studies was filled by an outpour of immediate opinions (i.e., letters to the editor) published in PubMed-indexed journals. Summarizing, preprints have gained traction as a publishing response to the demand for prompt access to empirical, albeit not peer-reviewed, findings during the present pandemic.
Project description:Here we present the results of a bibliometric survey of peer-reviewed and pre-print papers published in the English language on issues related to COVID-19 within the first three months since a cluster of a severe acute respiratory disease of unknown etiology was officially confirmed by the Chinese Center for Disease Control and Prevention on 31 December 2019. A systematic search using PubMed/Medline and Scopus databases and preprint servers was performed. The articles were classified according to their type, subject and country of origin. Up to 31 March 2020, a total of 2062 papers published in 578 peer-reviewed journals and 1425 preprints posted mostly on medRxiv (55.4 %), were identified. The mean number of published journal papers and preprints per day in the considered period was 27 and 12, respectively, and reached a maximum of 51 and 46 per day in March, respectively. The identified articles, journal papers and preprints, mostly covered the epidemiology of COVID-19 (35.7 %), clinical aspects of infection (21.0 %), preventative measures (12.8 %), treatment options (12.5 %), diagnostics (12.2 %), mathematical modeling of disease transmission and mitigation (9.6 %), and molecular biology and pathogenesis of SARS-CoV-2 (8.7 %). The majority of the journal papers were commentaries (38.5 %), reviews (33.6 %) and original research (21.3 %), while preprints predominantly presented original results (89.8 %). Chinese scientists contributed the highest share of original research and were responsible for 32.9 % journal papers and 43.9 % preprints published in the considered period. A high number of contributions was also seen from the United States, the United Kingdom, and Italy. The benefits and potential risks of such a massive publication output are discussed. The scientific response seen during the first 3 months of the COVID-19 outbreak is a demonstration of the capabilities of modern science to react rapidly to emerging global health threats by providing and discussing the essential information for understanding the etiological factor, its spread, preventative measures, and mitigation strategies.
Project description:This study explores the response to COVID-19 from investigators, editors, and publishers and seeks to define challenges during the early stages of the pandemic. A cross-sectional bibliometric review of COVID-19 literature was undertaken between 1 November 2019 and 24 March 2020, along with a comparative review of Middle East respiratory syndrome (MERS) literature. Investigator responsiveness was assessed by measuring the volume and type of research published. Editorial responsiveness was assessed by measuring the submission-to-acceptance time and availability of original data. Publisher-responsiveness was assessed by measuring the acceptance-to-publication time and the provision of open access. Three hundred and ninety-eight of 2,835 COVID-19 and 55 of 1,513 MERS search results were eligible. Most COVID-19 studies were clinical reports (<i>n</i> = 242; 60.8%). The submission-to-acceptance [median: 5 days (IQR: 3-11) versus 71.5 days (38-106); <i>P</i> < .001] and acceptance-to-publication [median: 5 days (IQR: 2-8) versus 22.5 days (4-48·5-; <i>P</i> < .001] times were strikingly shorter for COVID-19. Almost all COVID-19 (<i>n</i> = 396; 99.5%) and MERS (<i>n</i> = 55; 100%) studies were open-access. Data sharing was infrequent, with original data available for 104 (26.1%) COVID-19 and 10 (18.2%) MERS studies (<i>P</i> = .203). The early academic response was characterized by investigators aiming to define the disease. Studies were made rapidly and openly available. Only one-in-four were published alongside original data, which is a key target for improvement.<h4>Key points</h4>COVID-19 publications show rapid response from investigators, specifically aiming to define the disease.Median time between submission and acceptance of COVID-19 articles is 5 days demonstrating rapid decision-making compared with the median of 71.5 days for MERS articles.Median time from acceptance to publication of COVID-19 articles is 5 days, confirming the ability to introduce rapid increases at times of crisis, such as during the SARS outbreak.The majority of both COVID-19 and MERS articles are available open-access.
Project description:<h4>Background</h4>Since the start of the COVID-19 outbreak, a large number of COVID-19-related papers have been published. However, concerns about the risk of expedited science have been raised. We aimed at reviewing and categorizing COVID-19-related medical research and to critically appraise peer-reviewed original articles.<h4>Methods</h4>The data sources were Pubmed, Cochrane COVID-19 register study, arXiv, medRxiv and bioRxiv, from 01/11/2019 to 01/05/2020. Peer-reviewed and preprints publications related to COVID-19 were included, written in English or Chinese. No limitations were placed on study design. Reviewers screened and categorized studies according to i) publication type, ii) country of publication, and iii) topics covered. Original articles were critically appraised using validated quality assessment tools.<h4>Results</h4>Among the 11,452 publications identified, 10,516 met the inclusion criteria, among which 7468 (71.0%) were peer-reviewed articles. Among these, 4190 publications (56.1%) did not include any data or analytics (comprising expert opinion pieces). Overall, the most represented topics were infectious disease (n = 2326, 22.1%), epidemiology (n = 1802, 17.1%), and global health (n = 1602, 15.2%). The top five publishing countries were China (25.8%), United States (22.3%), United Kingdom (8.8%), Italy (8.1%) and India (3.4%). The dynamic of publication showed that the exponential growth of COVID-19 peer-reviewed articles was mainly driven by publications without original data (mean 261.5 articles ± 51.1 per week) as compared with original articles (mean of 69.3 ± 22.3 articles per week). Original articles including patient data accounted for 713 (9.5%) of peer-reviewed studies. A total of 576 original articles (80.8%) showed intermediate to high risk of bias. Last, except for simulation studies that mainly used large-scale open data, the median number of patients enrolled was of 102 (IQR = 37-337).<h4>Conclusions</h4>Since the beginning of the COVID-19 pandemic, the majority of research is composed by publications without original data. Peer-reviewed original articles with data showed a high risk of bias and included a limited number of patients. Together, these findings underscore the urgent need to strike a balance between the velocity and quality of research, and to cautiously consider medical information and clinical applicability in a pressing, pandemic context. SYSTEMATIC REVIEW REGISTRATION: https://osf.io/5zjyx/.
Project description:The time it takes for a completed manuscript to be published traditionally can be extremely lengthy. Article publication delay, which occurs in part due to constraints associated with peer review, can prevent the timely dissemination of critical and actionable data associated with new information on rare diseases or developing health concerns such as Zika virus. Preprint servers are open access online repositories housing preprint research articles that enable authors (1) to make their research immediately and freely available and (2) to receive commentary and peer review prior to journal submission. There is a growing movement of preprint advocates aiming to change the current journal publication and peer review system, proposing that preprints catalyze biomedical discovery, support career advancement, and improve scientific communication. While the number of articles submitted to and hosted by preprint servers are gradually increasing, there has been no simple way to identify biomedical research published in a preprint format, as they are not typically indexed and are only discoverable by directly searching the specific preprint server websites. To address this issue, we created a search engine that quickly compiles preprints from disparate host repositories and provides a one-stop search solution. Additionally, we developed a web application that bolsters the discovery of preprints by enabling each and every word or phrase appearing on any web site to be integrated with articles from preprint servers. This tool, search.bioPreprint, is publicly available at http://www.hsls.pitt.edu/resources/preprint.
Project description:<h4>Background</h4>The COVID-19 pandemic has highlighted the importance of rapid dissemination of scientific and medical discoveries. Current platforms available for the distribution of scientific and clinical research data and information include preprint repositories and traditional peer-reviewed journals. In recent times, social media has emerged as a helpful platform to share scientific and medical discoveries.<h4>Objective</h4>This study aimed to comparatively analyze activity on social media (specifically, Twitter) and that related to publications in the form of preprint and peer-reviewed journal articles in the context of COVID-19 and gastroenterology during the early stages of the COVID-19 pandemic.<h4>Methods</h4>COVID-19-related data from Twitter (tweets and user data) and articles published in preprint servers (bioRxiv and medRxiv) as well as in the PubMed database were collected and analyzed during the first 6 months of the pandemic, from December 2019 through May 2020. Global and regional geographic and gastrointestinal organ-specific social media trends were compared to preprint and publication activity. Any relationship between Twitter activity and preprint articles published and that between Twitter activity and PubMed articles published overall, by organ system, and by geographic location were identified using Spearman's rank-order correlation.<h4>Results</h4>Over the 6-month period, 73,079 tweets from 44,609 users, 7164 journal publications, and 4702 preprint publications were retrieved. Twitter activity (ie, number of tweets) peaked in March 2020, whereas preprint and publication activity (ie, number of articles published) peaked in April 2020. Overall, strong correlations were identified between trends in Twitter activity and preprint and publication activity (P<.001 for both). COVID-19 data across the three platforms mainly concentrated on pulmonology or critical care, but when analyzing the field of gastroenterology specifically, most tweets pertained to pancreatology, most publications focused on hepatology, and most preprints covered hepatology and luminal gastroenterology. Furthermore, there were significant positive associations between trends in Twitter and publication activity for all gastroenterology topics (luminal gastroenterology: P=.009; hepatology and inflammatory bowel disease: P=.006; gastrointestinal endoscopy: P=.007), except pancreatology (P=.20), suggesting that Twitter activity did not correlate with publication activity for this topic. Finally, Twitter activity was the highest in the United States (7331 tweets), whereas PubMed activity was the highest in China (1768 publications).<h4>Conclusions</h4>The COVID-19 pandemic has highlighted the potential of social media as a vehicle for disseminating scientific information during a public health crisis. Sharing and spreading information on COVID-19 in a timely manner during the pandemic has been paramount; this was achieved at a much faster pace on social media, particularly on Twitter. Future investigation could demonstrate how social media can be used to augment and promote scholarly activity, especially as the world begins to increasingly rely on digital or virtual platforms. Scientists and clinicians should consider the use of social media in augmenting public awareness regarding their scholarly pursuits.
Project description:<h4>Background</h4>Flawed or misleading articles may be retracted because of either honest scientific errors or scientific misconduct. This study explored the characteristics of retractions in medical journals published in Korea through the KoreaMed database.<h4>Methods</h4>We retrieved retraction articles indexed in the KoreaMed database from January 1990 to January 2016. Three authors each reviewed the details of the retractions including the reason for retraction, adherence to retraction guidelines, and appropriateness of retraction. Points of disagreement were reconciled by discussion among the three.<h4>Results</h4>Out of 217,839 articles in KoreaMed published from 1990 to January 2016, the publication type of 111 articles was retraction (0.051%). Of the 111 articles (addressing the retraction of 114 papers), 58.8% were issued by the authors, 17.5% were jointly issued (author, editor, and publisher), 15.8% came from editors, and 4.4% were dispatched by institutions; in 5.3% of the instances, the issuer was unstated. The reasons for retraction included duplicate publication (57.0%), plagiarism (8.8%), scientific error (4.4%), author dispute (3.5%), and other (5.3%); the reasons were unstated or unclear in 20.2%. The degree of adherence to COPE's retraction guidelines varied (79.8%-100%), and some retractions were inappropriate by COPE standards. These were categorized as follows: retraction of the first published article in the case of duplicate publication (69.2%), authorship dispute (15.4%), errata (7.7%), and other (7.7%).<h4>Conclusion</h4>The major reason for retraction in Korean medical journals is duplicate publication. Some retractions resulted from overreaction by the editors. Therefore, editors of Korean medical journals should take careful note of the COPE retraction guidelines and should undergo training on appropriate retraction practices.
Project description:<h4>Objective</h4>To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.<h4>Design</h4>Cross-sectional study.<h4>Setting</h4>International medical literature.<h4>Participants</h4>Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020.<h4>Main outcome measures</h4>Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably).<h4>Results</h4>Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study.<h4>Conclusions</h4>The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.