Dataset Information

Standardized benchmarking in the quest for orthologs.

ABSTRACT: Achieving high accuracy in orthology inference is essential for many comparative, evolutionary and functional genomic analyses, yet the true evolutionary history of genes is generally unknown and orthologs are used for very different applications across phyla, requiring different precision-recall trade-offs. As a result, it is difficult to assess the performance of orthology inference methods. Here, we present a community effort to establish standards and an automated web-based service to facilitate orthology benchmarking. Using this service, we characterize 15 well-established inference methods and resources on a battery of 20 different benchmarks. Standardized benchmarking provides a way for users to identify the most effective methods for the problem at hand, sets a minimum requirement for new tools and resources, and guides the development of more accurate orthology inference methods.

SUBMITTER: Altenhoff AM

PROVIDER: S-EPMC4827703 | biostudies-literature | 2016 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Standardized benchmarking in the quest for orthologs.

Altenhoff Adrian M AM Boeckmann Brigitte B Capella-Gutierrez Salvador S Dalquen Daniel A DA DeLuca Todd T Forslund Kristoffer K Huerta-Cepas Jaime J Linard Benjamin B Pereira Cécile C Pryszcz Leszek P LP Schreiber Fabian F da Silva Alan Sousa AS Szklarczyk Damian D Train Clément-Marie CM Bork Peer P Lecompte Odile O von Mering Christian C Xenarios Ioannis I Sjölander Kimmen K Jensen Lars Juhl LJ Martin Maria J MJ Muffato Matthieu M Gabaldón Toni T Lewis Suzanna E SE Thomas Paul D PD Sonnhammer Erik E Dessimoz Christophe C

Nature methods 20160404 5

Achieving high accuracy in orthology inference is essential for many comparative, evolutionary and functional genomic analyses, yet the true evolutionary history of genes is generally unknown and orthologs are used for very different applications across phyla, requiring different precision-recall trade-offs. As a result, it is difficult to assess the performance of orthology inference methods. Here, we present a community effort to establish standards and an automated web-based service to facili ...[more]

PMID: 27043882

Similar Datasets

Project description:BackgroundDiversity, equity, and inclusion pertaining to race, ethnicity, and related concepts have historically been underrepresented in clinical trials for pharmaceutical drug development, although this is an increasing topic for regulators, payers, and patient advocacy groups. We aimed to develop a summary statistical measure to assess such representativeness.MethodsA statistical measure using population demographic parameters derived from performance metrics through verbal autopsy research was proposed for using population frameworks in the UK. The summary measure, R-index, was demonstrated using simulation data with population frameworks from the UK (116 Roche UK clinical trials 2013-2022) and then using published clinical trial results (NCT02366143 [March 1, 2015-September 15, 2017], NCT04368728 [July 27, 2020-October 9, 2020], and NCT04470427 [July 27, 2020-November 25, 2020]). R-index was further proposed for use with benchmarking performance in representative trial development for internal processes, external benchmarking, and performance tracking in clinical trial development.FindingsR-index was derived from a standardized statistical measure called the L1 norm, or Manhattan distance, and then normalized to the maximum theoretical error observed in some populations using population framework or ontology for reporting concepts such as race, ethnicity, and other dimensions of diversity used to characterize patient cohorts. R-index demonstrated desirable qualities in demonstration simulations, including a range of 0-1, ease of calculation and use, and interpretability and flexibility, as data standards in the space of inclusive research continue to develop.InterpretationR-index is an interpretable, accessible summary statistic that may be useful for tracking and benchmarking representativeness in inclusive research and related domains. R-index is adaptable to different population frameworks and ontologies across different settings and considerations in terms of underlying population variables.FundingF. Hoffmann-La Roche Ltd/Genentech, Inc.

Project description:The adaptive immune responses of humans and of other jawed vertebrate species (gnasthostomata) are characterized by the B and T cells and their specific antigen receptors, the immunoglobulins (IG) or antibodies and the T cell receptors (TR) (up to 2.1012 different IG and TR per individual). IMGT, the international ImMunoGeneTics information system (http://www.imgt.org), was created in 1989 by Marie-Paule Lefranc (Montpellier University and CNRS) to manage the huge and complex diversity of these antigen receptors. IMGT built on IMGT-ONTOLOGY concepts of identification (keywords), description (labels), classification (gene and allele nomenclature) and numerotation (IMGT unique numbering), is at the origin of immunoinformatics, a science at the interface between immunogenetics and bioinformatics. IMGT/HighV-QUEST, the first web portal, and so far the only one, for the next generation sequencing (NGS) analysis of IG and TR, is the paradigm for immune repertoire standardized outputs and immunoprofiles of the adaptive immune responses. It provides the identification of the variable (V), diversity (D) and joining (J) genes and alleles, analysis of the V-(D)-J junction and complementarity determining region 3 (CDR3) and the characterization of the 'IMGT clonotype (AA)' (AA for amino acid) diversity and expression. IMGT/HighV-QUEST compares outputs of different batches, up to one million nucleotide sequencesfor the statistical module. These high throughput IG and TR repertoire immunoprofiles are of prime importance in vaccination, cancer, infectious diseases, autoimmunity and lymphoproliferative disorders, however their comparative statistical analysis still remains a challenge. We present a standardized statistical procedure to analyze IMGT/HighV-QUEST outputs for the evaluation of the significance of the IMGT clonotype (AA) diversity differences in proportions, per gene of a given group, between NGS IG and TR repertoire immunoprofiles. The procedure is generic and suitable for evaluating significance of the IMGT clonotype (AA) diversity and expression per gene, and for any IG and TR immunoprofiles of any species.

Dataset Information

Standardized benchmarking in the quest for orthologs.

Publications

Standardized benchmarking in the quest for orthologs.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets