Unknown

Dataset Information

Evaluating generalizability of artificial intelligence models for molecular datasets.


ABSTRACT: Deep learning has made rapid advances in modeling molecular sequencing data. Despite achieving high performance on benchmarks, it remains unclear to what extent deep learning models learn general principles and generalize to previously unseen sequences. Benchmarks traditionally interrogate model generalizability by generating metadata based (MB) or sequence-similarity based (SB) train and test splits of input data before assessing model performance. Here, we show that this approach mischaracterizes model generalizability by failing to consider the full spectrum of cross-split overlap, i.e., similarity between train and test splits. We introduce Spectra, a spectral framework for comprehensive model evaluation. For a given model and input data, Spectra plots model performance as a fun

SUBMITTER: Ektefaie Y 

PROVIDER: S-EPMC10925170 | biostudies-literature | 2024 Feb

REPOSITORIES: biostudies-literature