Measuring the quality of scientific references in Wikipedia: an analysis of more than 115M citations to over 800 000 scientific articles.
Ontology highlight
ABSTRACT:
SUBMITTER: Nicholson JM
PROVIDER: S-EPMC8060352 | biostudies-literature | 2021 Jul
REPOSITORIES: biostudies-literature
Publications
Measuring the quality of scientific references in Wikipedia: an analysis of more than 115M citations to over 800 000 scientific articles.
Nicholson Joshua M JM Uppala Ashish A Sieber Matthias M Grabitz Peter P Mordaunt Milo M Rife Sean C SC
The FEBS journal 20201119 14
Wikipedia is a widely used online reference work which cites hundreds of thousands of scientific articles across its entries. The quality of these citations has not been previously measured, and such measurements have a bearing on the reliability and quality of the scientific portions of this reference work. Using a novel technique, a massive database of qualitatively described citations, and machine learning algorithms, we analyzed 1 923 575 Wikipedia articles which cited a total of 824 298 sci ...[more]
Similar Datasets
Project description:Referencing scholarly documents as information sources on Wikipedia is important because it supports or improves the quality of Wikipedia content. Several studies have been conducted regarding scholarly references on Wikipedia; however, little is known of the editors and their edits contributing to add the scholarly references on Wikipedia. In this study, we develop a methodology to detect the oldest scholarly reference added to Wikipedia articles by which a certain paper is uniquely identifiable as the "first appearance of the scholarly reference." We identified the first appearances of 923,894 scholarly references (611,119 unique DOIs) in 180,795 unique pages on English Wikipedia as of March 1, 2017 and stored them in the dataset. Moreover, we assessed the precision of the dataset, which was highly precise regardless of the research field. Finally, we demonstrate the potential of our dataset. This dataset is unique and attracts those who are interested in how the scholarly references on Wikipedia grew and which editors added them.
| S-EPMC8921307 | biostudies-literature
Project description:ObjectiveWikipedia is the most frequently accessed online health information resource and is well positioned as a valuable tool for public health communication and knowledge translation. The authors aimed to explore their institution's health and medical research reach by analyzing its presence in Wikipedia articles.MethodsIn October 2022, a comprehensive database search was constructed in PubMed to retrieve clinical evidence syntheses published by at least one author affiliated with McMaster University from 2017 to 2022, inclusive. Altmetric Explorer was queried using PubMed Identifiers and article titles to access metadata and Wikipedia citation data. 3,582 health evidence syntheses from at least one McMaster University affiliated author were analyzed.ResultsSix percent (n=219) of health evidence syntheses from the authors' institution were cited 568 times in 524 unique Wikipedia articles across 28 different language editions. 45% of citations appeared in English Wikipedia, suggesting a broad global reach for the institutions' research outputs. When adjusted for open access publications, 8% of McMaster University's health evidence syntheses appear in Wikipedia.ConclusionAltmetric Explorer is a valuable tool for exploring the reach of an institution's research outputs. Isolating Altmetric data to focus on Wikipedia citations has value for any institution wishing to gain more insight into the global, community-level reach of its contributions to the latest health and medical evidence.
| S-EPMC11305477 | biostudies-literature
Project description:BACKGROUND:Social media coverage is increasingly used to spread the message of scientific publications. Traditionally, the scientific impact of an article is measured by the number of citations. At a journal level, this conventionally matures over a 2-year period, and it is challenging to gauge impact around the time of publication. OBJECTIVE:We, therefore, aimed to assess whether Web-based attention is associated with citations and to develop a predictive model that assigns relative importance to different elements of social media coverage: the #SoME_Impact score. METHODS:We included all original articles published in 2015 in a selection of the highest impact journals: The New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, Nature, Cell, and Science. We first characterized the change in Altmetric score over time by taking a single month's sample of recently published articles from the same journals and gathered Altmetric data daily from the time of publication to create a mixed effects spline model. We then obtained the overall weighted Altmetric score for all articles from 2015, the unweighted data for each Altmetric component, and the 2-year citation count from Scopus for each of these articles from 2016 to 2017. We created a stepwise multivariable linear regression model to develop a #SoME_Score that was predictive of 2-year citations. The score was validated using a dataset of articles from the same journals published in 2016. RESULTS:In our unselected sample of 145 recently published articles, social media coverage appeared to plateau approximately 14 days after publication. A total of 3150 articles with a median citation count of 16 (IQR 5-33) and Altmetric score of 72 (IQR 28-169) were included for analysis. On multivariable regression, compared with articles in the lowest quantile of #SoME_Score, articles in the second, third, and upper quantiles had 0.81, 15.20, and 87.67 more citations, respectively. On the validation dataset, #SoME_Score model outperformed the Altmetric score (adjusted R2 0.19 vs 0.09; P<.001). Articles in the upper quantile of #SoME_Score were more than 5 times more likely to be among the upper quantile of those cites (odds ratio 5.61, 95% CI 4.70-6.73). CONCLUSIONS:Social media attention predicts citations and could be used as an early surrogate measure of scientific impact. Owing to the cross-sectional study design, we cannot determine whether correlation relates to causation.
| S-EPMC7195668 | biostudies-literature
Project description:The existence of a shared classification system is essential to knowledge production, transfer, and sharing. Studies of knowledge classification, however, rarely consider the fact that knowledge categories exist within hierarchical information systems designed to facilitate knowledge search and discovery. This neglect is problematic whenever information about categorical membership is itself used to evaluate the quality of the items that the category contains. The main objective of this paper is to show that the effects of category membership depend on the position that a category occupies in the hierarchical knowledge classification system of Wikipedia-an open knowledge production and sharing platform taking the form of a freely accessible on-line encyclopedia. Using data on all English-language Wikipedia articles, we examine how the position that a category occupies in the classification hierarchy affects the attention that articles in that category attract from Wikipedia editors, and their evaluation of quality of the Wikipedia articles. Specifically, we show that Wikipedia articles assigned to coarse-grained categories (i. e., categories that occupy higher positions in the hierarchical knowledge classification system) garner more attention from Wikipedia editors (i. e., attract a higher volume of text editing activity), but receive lower evaluations (i. e., they are considered to be of lower quality). The negative relation between attention and quality implied by this result is consistent with current theories of social categorization, but it also goes beyond available results by showing that the effects of categorization on evaluation depend on the position that a category occupies in a hierarchical knowledge classification system.
| S-EPMC5749832 | biostudies-literature
Project description:In this work we study how people navigate the information network of Wikipedia and investigate (i) free-form navigation by studying all clicks within the English Wikipedia over an entire month and (ii) goal-directed Wikipedia navigation by analyzing wikigames, where users are challenged to retrieve articles by following links. To study how the organization of Wikipedia articles in terms of layout and links affects navigation behavior, we first investigate the characteristics of the structural organization and of hyperlinks in Wikipedia and then evaluate link selection models based on article structure and other potential influences in navigation, such as the generality of an article's topic. In free-form Wikipedia navigation, covering all Wikipedia usage scenarios, we find that click choices can be best modeled by a bias towards article structure, such as a tendency to click links located in the lead section. For the goal-directed navigation of wikigames, our findings confirm the zoom-out and the homing-in phases identified by previous work, where users are guided by generality at first and textual similarity to the target later. However, our interpretation of the link selection models accentuates that article structure is the best explanation for the navigation paths in all except these initial and final stages. Overall, we find evidence that users more frequently click on links that are located close to the top of an article. The structure of Wikipedia articles, which places links to more general concepts near the top, supports navigation by allowing users to quickly find the better-connected articles that facilitate navigation. Our results highlight the importance of article structure and link position in Wikipedia navigation and suggest that better organization of information can help make information networks more navigable.
| S-EPMC5468769 | biostudies-literature
Project description:Wikipedia is a gateway to knowledge. However, the extent to which this gateway ends at Wikipedia or continues via supporting citations is unknown. Wikipedia's gateway functionality has implications for information design and education, notably in medicine. This study aims to establish benchmarks for the relative distribution and referral (click) rate of citations-as indicated by presence of a Digital Object Identifier (DOI)-from Wikipedia, with a focus on medical citations. DOIs referred from the English Wikipedia in August 2016 were obtained from Crossref.org. Next, based on a DOI's presence on a WikiProject Medicine page, all DOIs in Wikipedia were categorized as medical (WP:MED) or non-medical (non-WP:MED). Using this categorization, referred DOIs were classified as WP:MED, non-WP:MED, or BOTH, meaning the DOI may have been referred from either category. Data were analyzed using descriptive and inferential statistics. Out of 5.2 million Wikipedia pages, 4.42% (n = 229,857) included at least one DOI. 68,870 were identified as WP:MED, with 22.14% (n = 15,250) featuring one or more DOIs. WP:MED pages featured on average 8.88 DOI citations per page, whereas non-WP:MED pages had on average 4.28 DOI citations. For DOIs only on WP:MED pages, a DOI was referred every 2,283 pageviews and for non-WP:MED pages every 2,467 pageviews. DOIs from BOTH pages accounted for 12% (n = 58,475). The referral of DOI citations found in BOTH could not be assigned to WP:MED or non-WP:MED, as the page from which the referral was made was not provided with the data. While these results cannot provide evidence of greater citation referral from WP:MED than non-WP:MED, they do provide benchmarks to assess strategies for changing referral patterns. These changes might include editors adopting new methods for designing and presenting citations or the introduction of teaching strategies that address the value of consulting citations as a tool for extending learning.
| S-EPMC5739466 | biostudies-other
Project description:ImportanceWomen are less likely to be promoted and hold leadership positions in academic medicine. How often academic articles are cited is a key measure of scholarly impact and frequently assessed for professional advancement; however, it is unknown whether peer-reviewed articles written by men and women are cited differently.ObjectiveTo evaluate whether academic articles from high-impact medical journals written by men and women are cited differently.Design, setting, and participantsIn this cross-sectional study of all original research and commentary articles from 5 high-impact medical journals (Annals of Internal Medicine, British Medical Journal, JAMA, JAMA Internal Medicine, and The New England Journal of Medicine) from 2015 to 2018, the gender of the primary and senior authors of each article were identified using an online database, and the number of times each article has been cited was identified using Web of Science. The number of citations by primary and senior author gender were then compared. Data were analyzed from July 2020 to April 2021.ExposuresPrimary and senior authors' genders.Main outcomes and measuresNumber of citations per article.ResultsAmong 5554 articles, women wrote 1975 (35.6%) as primary author and 1273 of 4940 (25.8%) as senior author. Original research articles written by women as primary authors had fewer median (interquartile range) citations than articles written by men as primary authors (36 [17-82] citations vs 54 [22-141] citations; P < .001) and senior authors (37 [17-93] citations vs 51 [20-128] citations; P < .001). Articles written by women as both primary and senior authors had approximately half as many median (interquartile range) citations as those authored by men as both primary and senior authors (33 [15-68] citations vs 59 [23-149] citations; P < .001). Differences in citations remained in each year of the study and were less pronounced among commentary articles.Conclusions and relevanceIn this study, articles written by women in high-impact medical journals had fewer citations than those written by men, particularly when women wrote together as primary and senior authors. These differences may have important consequences for the professional success of women and achieving gender equity in academic medicine.
| S-EPMC8254129 | biostudies-literature
Project description:How different cultures evaluate a person? Is an important person in one culture is also important in the other culture? We address these questions via ranking of multilingual Wikipedia articles. With three ranking algorithms based on network structure of Wikipedia, we assign ranking to all articles in 9 multilingual editions of Wikipedia and investigate general ranking structure of PageRank, CheiRank and 2DRank. In particular, we focus on articles related to persons, identify top 30 persons for each rank among different editions and analyze distinctions of their distributions over activity fields such as politics, art, science, religion, sport for each edition. We find that local heroes are dominant but also global heroes exist and create an effective network representing entanglement of cultures. The Google matrix analysis of network of cultures shows signs of the Zipf law distribution. This approach allows to examine diversity and shared characteristics of knowledge organization between cultures. The developed computational, data driven approach highlights cultural interconnections in a new perspective. Dated: June 26, 2013.
| S-EPMC3789750 | biostudies-literature
Project description:The effective evaluation of the impact of a scholarly article is a significant endeavor; for this reason, it has garnered attention. From the perspective of knowledge flow, this paper extracted various knowledge flow patterns concealed in articles citation counts to describe the citation impact of the articles. First, the intensity characteristic of knowledge flow was investigated to distinguish the different citation vitality of articles. Second, the knowledge diffusion capacity was examined to differentiate the size of the scope of articles' influences on the academic environment. Finally, the knowledge transfer capacity was discussed to investigate the support degree of articles on the follow-up research. Experimental results show that articles got more citations recently have a higher knowledge flow intensity. The articles have various impacts on the academic environment and have different supporting effects on the follow-up research, representing the differences in their knowledge diffusion and knowledge transfer capabilities. Compared with the single quantitative index of citation frequency, these knowledge flow patterns can carefully explore the citation value of articles. By integrating the three knowledge flow patterns to examine the total citation impact of articles, we found that the articles exhibit distinct value of citation impact even if they were published in the same field, in the same year, and with similar citation frequencies.
| S-EPMC6872168 | biostudies-literature
Project description:ImportanceRetraction is a tool that journals can use to deter research misconduct and alert their audience to erroneous content published in the journals. However, retracted articles may continue to damage science if they are still cited as legitimate articles.ObjectiveTo characterize patterns of postretraction citations, particularly in microRNA biomarker research, a field with one of the highest rates of retraction.Evidence reviewRetracted scientific articles on microRNAs were retrieved from PubMed, Web of Science, and Retraction Watch between database inception and July 17, 2021, according to preestablished search strategies. Control articles with characteristics in common with retracted articles (ie, published in the same journals in the same years and months and with the same number of authors) were matched and retrieved from PubMed. Citation metrics of retractions and control articles were collected from Web of Science. PubPeer was referenced to examine the public response or comments on included retractions. Data were analyzed from September 2021 through March 2023.FindingsA total of 10 461 articles were analyzed, with 887 retractions and 9574 articles as controls. Among retracted articles, which were published from 1999 to 2021, there were 756 articles (85.23%) written by researchers affiliated with Chinese institutions. Retracted articles were cited 6327 times after retraction. Of 792 retracted articles that were cited, 621 articles (78.41%) were cited at least once after retraction and 238 articles (30.05%) were cited more often after retraction than before retraction. Overall citations (comprising citations before and after retraction) and postretraction citations accumulated over time (eg, the median [IQR] number of postretraction citations was 1 [1-2] and 23 [9-44] citations at the first 6 and 66 months, respectively, between retraction and citation retrieval). A random sample of 87 retracted articles (9.81%) recorded 478 citations after retraction, with 208 citations (43.51%) in articles published 12 months or longer after retraction. Of these citing articles, 19 articles (3.97%) mentioned the retractions. Compared with the control group of 1620 nonretracted articles, no significant differences were found in overall number of citations or citations after retraction. Among 478 articles citing retracted articles, 414 articles were found on PubMed and had matched control articles; these articles had higher odds of being subsequently retracted than 7954 matched control articles (odds ratio, 6.57; 95% CI, 3.39-12.72).Conclusions and relevanceIn this study, retraction was not associated with a reduction in citations of retracted articles, but articles that cited retracted publications had higher odds of later retraction. These findings suggest that journals may need to implement mechanisms for detection of postretraction citations.
| S-EPMC10958238 | biostudies-literature