Generative language reconstruction from brain recordings.

ABSTRACT: Language reconstruction from non-invasive brain recordings has been a long-standing challenge. Existing research has addressed this challenge with a classification setup, where a set of language candidates are pre-constructed and then matched with the representation decoded from brain recordings. Here, we propose a method that addresses language reconstruction through auto-regressive generation, which directly uses the representation decoded from functional magnetic resonance imaging (fMRI) as the input for a large language model (LLM), mitigating the need for pre-constructed candidates. While an LLM can already generate high-quality content, our approach produces results more closely aligned with the visual or auditory language stimuli in response to which brain recordings are sampled, es

SUBMITTER: Ye Z 

PROVIDER: S-EPMC11873242 | biostudies-literature | 2025 Mar

REPOSITORIES: biostudies-literature