Heimdall: A Modular Framework for Tokenization in Single-Cell Foundation Models.

ABSTRACT: Foundation models trained on single-cell RNA-sequencing (scRNA-seq) data have rapidly become powerful tools for single-cell analysis. Their performance, however, depends critically on how cells are tokenized into model inputs - a design space that remains poorly understood. Here, we present Heimdall, a comprehensive framework and open-source toolkit for systematically evaluating tokenization strategies in single-cell foundation models (scFMs). Heimdall decomposes each scFM into modular components: a gene identity encoder (F _G), an expression encoder (F _E), and a "cell sentence" constructor (F _C) with submodules (order, sequence, and reduce) enabling fine-grained control and attribution. Using a transformer trained from scratch, we evalu

SUBMITTER: Haber E 

PROVIDER: S-EPMC12642435 | biostudies-literature | 2025 Nov

REPOSITORIES: biostudies-literature