Dataset Information

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS.

ABSTRACT: Recent studies have shown that speech can be reconstructed and synthesized using only brain activity recorded with intracranial electrodes, but until now this has only been done using retrospective analyses of recordings from able-bodied patients temporarily implanted with electrodes for epilepsy surgery. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a clinical trial participant (ClinicalTrials.gov, NCT03567213) with dysarthria due to amyotrophic lateral sclerosis (ALS). We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the user from a vocabulary of 6 keywords originally designed to allow intuitive selection of items on a communication board. Our results show for the first time that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words that are intelligible to human listeners while preserving the participants voice profile.

SUBMITTER: Angrick M

PROVIDER: S-EPMC10327279 | biostudies-literature | 2023 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS.

Angrick Miguel M Luo Shiyu S Rabbani Qinwan Q Candrea Daniel N DN Shah Samyak S Milsap Griffin W GW Anderson William S WS Gordon Chad R CR Rosenblatt Kathryn R KR Clawson Lora L Maragakis Nicholas N Tenore Francesco V FV Fifer Matthew S MS Hermansky Hynek H Ramsey Nick F NF Crone Nathan E NE

medRxiv : the preprint server for health sciences 20230701

Recent studies have shown that speech can be reconstructed and synthesized using only brain activity recorded with intracranial electrodes, but until now this has only been done using retrospective analyses of recordings from able-bodied patients temporarily implanted with electrodes for epilepsy surgery. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a clinical trial participant (ClinicalTrials.gov, NCT03567213) with dysart ...[more]

PMID: 37425721

Similar Datasets

Project description:Damage or degeneration of motor pathways necessary for speech and other movements, as in brainstem strokes or amyotrophic lateral sclerosis (ALS), can interfere with efficient communication without affecting brain structures responsible for language or cognition. In the worst-case scenario, this can result in the locked in syndrome (LIS), a condition in which individuals cannot initiate communication and can only express themselves by answering yes/no questions with eye blinks or other rudimentary movements. Existing augmentative and alternative communication (AAC) devices that rely on eye tracking can improve the quality of life for people with this condition, but brain-computer interfaces (BCIs) are also increasingly being investigated as AAC devices, particularly when eye tracking is too slow or unreliable. Moreover, with recent and ongoing advances in machine learning and neural recording technologies, BCIs may offer the only means to go beyond cursor control and text generation on a computer, to allow real-time synthesis of speech, which would arguably offer the most efficient and expressive channel for communication. The potential for BCI speech synthesis has only recently been realized because of seminal studies of the neuroanatomical and neurophysiological underpinnings of speech production using intracranial electrocorticographic (ECoG) recordings in patients undergoing epilepsy surgery. These studies have shown that cortical areas responsible for vocalization and articulation are distributed over a large area of ventral sensorimotor cortex, and that it is possible to decode speech and reconstruct its acoustics from ECoG if these areas are recorded with sufficiently dense and comprehensive electrode arrays. In this article, we review these advances, including the latest neural decoding strategies that range from deep learning models to the direct concatenation of speech units. We also discuss state-of-the-art vocoders that are integral in constructing natural-sounding audio waveforms for speech BCIs. Finally, this review outlines some of the challenges ahead in directly synthesizing speech for patients with LIS.

Project description:Background and objectivesBrain-computer interfaces (BCIs) hold promise as augmentative and alternative communication technology for people with severe motor and speech impairment (locked-in syndrome) due to neural disease or injury. Although such BCIs should be available 24/7, to enable communication at all times, feasibility of nocturnal BCI use has not been investigated. Here, we addressed this question using data from an individual with amyotrophic lateral sclerosis (ALS) who was implanted with an electrocorticography-based BCI that enabled the generation of click-commands for spelling words and call-caregiver signals.MethodsWe investigated nocturnal dynamics of neural signal features used for BCI control, namely low (LFB: 10-30Hz) and high frequency band power (HFB: 65-95Hz). Additionally, we assessed the nocturnal performance of a BCI decoder that was trained on daytime data by quantifying the number of unintentional BCI activations at night. Finally, we developed and implemented a nightmode decoder that allowed the participant to call a caregiver at night, and assessed its performance.ResultsPower and variance in HFB and LFB were significantly higher at night than during the day in the majority of the nights, with HFB variance being higher in 88% of nights. Daytime decoders caused 245 unintended selection-clicks and 13 unintended caregiver-calls per hour when applied to night data. The developed nightmode decoder functioned error-free in 79% of nights over a period of ±1.5 years, allowing the user to reliably call the caregiver, with unintended activations occurring only once every 12 nights.DiscussionReliable nighttime use of a BCI requires decoders that are adjusted to sleep-related signal changes. This demonstration of a reliable BCI nightmode and its long-term use by an individual with advanced ALS underscores the importance of 24/7 BCI reliability.Trial registrationThis trial is registered in clinicaltrials.gov under number NCT02224469 (https://clinicaltrials.gov/study/NCT02224469?term=NCT02224469&rank=1). Date of submission to registry: August 21, 2014. Enrollment of first participant: September 7, 2015.

Dataset Information

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS.

Publications

Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets