Dataset Information

The transferability limits of static benchmarks.

ABSTRACT: Every practical method to solve the Schrödinger equation for interacting many-particle systems introduces approximations. Such methods are therefore plagued by systematic errors. For computational chemistry, it is decisive to quantify the specific error for some system under consideration. Traditionally, the primary way for such an error assessment has been benchmarking data, usually taken from the literature. However, their transferability to a specific molecular system, and hence, the reliability of the traditional approach always remains uncertain to some degree. In this communication, we elaborate on the shortcomings of this traditional way of static benchmarking by exploiting statistical analyses using one of the largest quantum chemical benchmark sets available. We demonstrate the uncertainty of error estimates in the light of the choice of reference data selected for a benchmark study. To alleviate the issues with static benchmarks, we advocate to rely instead on a rolling and system-focused approach for rigorously quantifying the uncertainty of a quantum chemical result.

SUBMITTER: Weymuth T

PROVIDER: S-EPMC9215699 | biostudies-literature | 2022 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The transferability limits of static benchmarks.

Weymuth Thomas T Reiher Markus M

Physical chemistry chemical physics : PCCP 20220622 24

Every practical method to solve the Schrödinger equation for interacting many-particle systems introduces approximations. Such methods are therefore plagued by systematic errors. For computational chemistry, it is decisive to quantify the specific error for some system under consideration. Traditionally, the primary way for such an error assessment has been benchmarking data, usually taken from the literature. However, their transferability to a specific molecular system, and hence, the reliabil ...[more]

PMID: 35700515

Similar Datasets

Project description:Background For 50 years, static cold storage (SCS) has been the gold standard for solid organ preservation in transplantation. Although logistically convenient, this preservation method presents important constraints in terms of duration and cold ischemia-induced lesions. We aimed to develop a machine perfusion (MP) protocol for recovery of vascularized composite allografts (VCA) after static cold preservation and determine its effects in a rat limb transplantation model.Methods Partial hindlimbs were procured from Lewis rats and subjected to SCS in Histidine-Tryptophan-Ketoglutarate solution for 0, 12, 18, 24, and 48 hours. They were then either transplanted (Txp), subjected to subnormothermic machine perfusion (SNMP) for 3 hours with a modified Steen solution, or to SNMP + Txp. Perfusion parameters were assessed for blood gas and electrolytes measurement, and flow rate and arterial pressures were monitored continuously. Histology was assessed at the end of perfusion. For select SCS durations, graft survival and clinical outcomes after transplantation were compared between groups at 21 days.Results Transplantation of limbs preserved for 0, 12, 18, and 24-hour SCS resulted in similar survival rates at postoperative day 21. Grafts cold-stored for 48 hours presented delayed graft failure (p = 0.0032). SNMP of limbs after 12-hour SCS recovered the vascular resistance, potassium, and lactate levels to values similar to limbs that were not subjected to SCS. However, 18-hour SCS grafts developed significant edema during SNMP recovery. Transplantation of grafts that had undergone a mixed preservation method (12-hour SCS + SNMP + Txp) resulted in better clinical outcomes based on skin clinical scores at day 21 post-transplantation when compared to the SCS + Txp group (p = 0.01613).Conclusion To date, VCA MP is still limited to animal models and no protocols are yet developed for graft recovery. Our study suggests that ex vivo SNMP could help increase the preservation duration and limit cold ischemia-induced injury in VCA transplantation.

Project description:BackgroundAn important step in annotation of sequenced genomes is the identification of transcription factor binding sites. More than a hundred different computational methods have been proposed, and it is difficult to make an informed choice. Therefore, robust assessment of motif discovery methods becomes important, both for validation of existing tools and for identification of promising directions for future research.ResultsWe use a machine learning perspective to analyze collections of transcription factors with known binding sites. Algorithms are presented for finding position weight matrices (PWMs), IUPAC-type motifs and mismatch motifs with optimal discrimination of binding sites from remaining sequence. We show that for many data sets in a recently proposed benchmark suite for motif discovery, none of the common motif models can accurately discriminate the binding sites from remaining sequence. This may obscure the distinction between the potential performance of the motif discovery tool itself versus the intrinsic complexity of the problem we are trying to solve. Synthetic data sets may avoid this problem, but we show on some previously proposed benchmarks that there may be a strong bias towards a presupposed motif model. We also propose a new approach to benchmark data set construction. This approach is based on collections of binding site fragments that are ranked according to the optimal level of discrimination achieved with our algorithms. This allows us to select subsets with specific properties. We present one benchmark suite with data sets that allow good discrimination between positive and negative instances with the common motif models. These data sets are suitable for evaluating algorithms for motif discovery that rely on these models. We present another benchmark suite where PWM, IUPAC and mismatch motif models are not able to discriminate reliably between positive and negative instances. This suite could be used for evaluating more powerful motif models.ConclusionOur improved benchmark suites have been designed to differentiate between the performance of motif discovery algorithms and the power of motif models. We provide a web server where users can download our benchmark suites, submit predictions and visualize scores on the benchmarks.

Dataset Information

The transferability limits of static benchmarks.

Publications

The transferability limits of static benchmarks.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets