Identifying psychiatric manifestations in schizophrenia and depression from audio-visual behavioural indicators through a machine-learning approach.
Ontology highlight
ABSTRACT:
SUBMITTER: Xu S
PROVIDER: S-EPMC9640655 | biostudies-literature | 2022 Nov
REPOSITORIES: biostudies-literature
Publications
Identifying psychiatric manifestations in schizophrenia and depression from audio-visual behavioural indicators through a machine-learning approach.
Xu Shihao S Yang Zixu Z Chakraborty Debsubhra D Chua Yi Han Victoria YHV Tolomeo Serenella S Winkler Stefan S Birnbaum Michel M Tan Bhing-Leet BL Lee Jimmy J Dauwels Justin J
Schizophrenia (Heidelberg, Germany) 20221107 1
Schizophrenia (SCZ) and depression (MDD) are two chronic mental disorders that seriously affect the quality of life of millions of people worldwide. We aim to develop machine-learning methods with objective linguistic, speech, facial, and motor behavioral cues to reliably predict the severity of psychopathology or cognitive function, and distinguish diagnosis groups. We collected and analyzed the speech, facial expressions, and body movement recordings of 228 participants (103 SCZ, 50 MDD, and 7 ...[more]
Similar Datasets
Project description:People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants were more tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of "synchronous" responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception.
| S-EPMC7930005 | biostudies-literature
Project description:We observed a case over 25 years of relapsing-remitting schizophrenic spectrum disorder, varying regarding the main symptomatology between more depressive or more schizoaffective or rather typical schizophrenic syndrome. Diseased phases were repeatedly accompanied by minor skin lesions, which were initially classified as mixed tissue disorder. Psychotic phases were waxing-waning over years. During one later relapse, skin involvement was severe, classified to likely represent an allergic reaction to psychopharmaca; this generalized exanthema remitted rapidly with cortisone treatment and azathioprine. Under continued azathioprine and low dose neuroleptics, the patient remitted completely, appearing psychiatrically healthy for 16 years. When azathioprine was set off due to pregnancy, an extraordinary severe relapse of schizophrenia like psychosis accompanied by most severe skin lesions developed within a few weeks, then requiring 2 years of psychiatric inpatient treatment. Finally, a diagnosis of systemic lupus erythematodes plus neuropsychiatric lupus was made. A single CSF sample in 2013 showed suspicious biomarkers, matching with CSF cytokine profiling in schizophrenic and affective spectrum disorder patients and indicated mild neuroinflammation. Complex immune suppressive treatment was reinitiated short after relapse, but was only partially successful. However, surprisingly the psychosis and skin lesions remitted (in parallel) when belimumab was given (add-on). The very details of this complicated, long-term disease course are discussed also with regard to general ideas, in particular with respect to the question if this case of seemingly comorbid schizophrenia with minor autoimmunity signs represented a case of one emerging autoimmune disorder with variant manifestations systemically and within the CNS, though atypically with predominant appearance as a schizophrenia spectrum disorder.
| S-EPMC5530032 | biostudies-other
Project description:ObjectiveEncephalitis is a serious and potentially life-threatening condition of infectious or autoimmune cause. We aim to characterize the frequency and clinical spectrum of presenting psychiatric symptoms in encephalitis in order to inform earlier recognition and initiation of treatment.MethodsThis was a retrospective study of adult patients who met the 2013 International Encephalitis Consortium (IEC) and/or 2016 Graus criteria between February 2005 and February 2023. The study included two hospital systems in Houston, Texas, and Baltimore, Maryland and included a total of 642 patients. Psychiatric manifestations were grouped into five high-level categories: behavior, psychosis, mood, sleep disturbances, and catatonia.ResultsIn our cohort of 642 patients, 318 (49.6%) had psychiatric symptoms at the time of initial presentation, including 78.2% with autoimmune etiologies and 35.2% with viral etiologies (P < 0.001). Those with psychiatric symptoms were younger (median age 47.5 vs. 51.5; P < 0.001), and more likely to have a history of documented psychiatric disorders, as well as longer lengths of hospital stay, and poorer discharge outcomes. Of patients initially admitted to a psychiatric service (n = 28), most had autoimmune causes, although 3 out of 28 (10.7%) had herpes viral infections; admission to a psychiatric service was associated with substantially longer interval to initiation of antivirals and immunotherapy. Autoimmune and infectious etiologies differed in the spectrum and frequency of psychiatric manifestations.InterpretationPsychiatric symptoms are common across etiologies of encephalitis and are associated with longer lengths of hospital stay and worse clinical outcomes. Specific patterns and dimensionality of psychiatric symptoms distinguish autoimmune from infectious causes.
| S-EPMC11822797 | biostudies-literature
Project description:In order to parse the world around us, we must constantly determine which sensory inputs arise from the same physical source and should therefore be perceptually integrated. Temporal coherence between auditory and visual stimuli drives audio-visual (AV) integration, but the role played by AV spatial alignment is less well understood. Here, we manipulated AV spatial alignment and collected electroencephalography (EEG) data while human subjects performed a free-field variant of the "pip and pop" AV search task. In this paradigm, visual search is aided by a spatially uninformative auditory tone, the onsets of which are synchronized to changes in the visual target. In Experiment 1, tones were either spatially aligned or spatially misaligned with the visual display. Regardless of AV spatial alignment, we replicated the key pip and pop result of improved AV search times. Mirroring the behavioral results, we found an enhancement of early event-related potentials (ERPs), particularly the auditory N1 component, in both AV conditions. We demonstrate that both top-down and bottom-up attention contribute to these N1 enhancements. In Experiment 2, we tested whether spatial alignment influences AV integration in a more challenging context with competing multisensory stimuli. An AV foil was added that visually resembled the target and was synchronized to its own stream of synchronous tones. The visual components of the AV target and AV foil occurred in opposite hemifields; the two auditory components were also in opposite hemifields and were either spatially aligned or spatially misaligned with the visual components to which they were synchronized. Search was fastest when the auditory and visual components of the AV target (and the foil) were spatially aligned. Attention modulated ERPs in both spatial conditions, but importantly, the scalp topography of early evoked responses shifted only when stimulus components were spatially aligned, signaling the recruitment of different neural generators likely related to multisensory integration. These results suggest that AV integration depends on AV spatial alignment when stimuli in both modalities compete for selective integration, a common scenario in real-world perception.
| S-EPMC8016516 | biostudies-literature
Project description:BackgroundDifferent sources of sensory information can interact, often shaping what we think we have seen or heard. This can enhance the precision of perceptual decisions relative to those made on the basis of a single source of information. From a computational perspective, there are multiple reasons why this might happen, and each predicts a different degree of enhanced precision. Relatively slight improvements can arise when perceptual decisions are made on the basis of multiple independent sensory estimates, as opposed to just one. These improvements can arise as a consequence of probability summation. Greater improvements can occur if two initially independent estimates are summated to form a single integrated code, especially if the summation is weighted in accordance with the variance associated with each independent estimate. This form of combination is often described as a Bayesian maximum likelihood estimate. Still greater improvements are possible if the two sources of information are encoded via a common physiological process.Principal findingsHere we show that the provision of simultaneous audio and visual speech cues can result in substantial sensitivity improvements, relative to single sensory modality based decisions. The magnitude of the improvements is greater than can be predicted on the basis of either a Bayesian maximum likelihood estimate or a probability summation.ConclusionOur data suggest that primary estimates of speech content are determined by a physiological process that takes input from both visual and auditory processing, resulting in greater sensitivity than would be possible if initially independent audio and visual estimates were formed and then subsequently combined.
| S-EPMC2855706 | biostudies-literature
Project description:The ability to actively follow a moving auditory target with our heads remains unexplored even though it is a common behavioral response. Previous studies of auditory motion perception have focused on the condition where the subjects are passive. The current study examined head tracking behavior to a moving auditory target along a horizontal 100° arc in the frontal hemisphere, with velocities ranging from 20 to 110°/s. By integrating high fidelity virtual auditory space with a high-speed visual presentation we compared tracking responses of auditory targets against visual-only and audio-visual "bisensory" stimuli. Three metrics were measured-onset, RMS, and gain error. The results showed that tracking accuracy (RMS error) varied linearly with target velocity, with a significantly higher rate in audition. Also, when the target moved faster than 80°/s, onset and RMS error were significantly worst in audition the other modalities while responses in the visual and bisensory conditions were statistically identical for all metrics measured. Lastly, audio-visual facilitation was not observed when tracking bisensory targets.
| S-EPMC4701932 | biostudies-literature
Project description:Cerebrotendinous xanthomatosis (CTX) is a rare and severe, but treatable, inborn disorder of bile acid biosynthesis and sterol storage with autosomal recessive inheritance and variable clinical presentation. CTX treatment consists of chenodeoxycholic acid and must be started as early as possible to prevent permanent disability. Psychiatric manifestations are rare and non-specific, and often lead to significant diagnostic and treatment delay. Therefore, better recognition of the gamut of psychiatric manifestations in CTX can diminish the risk of misdiagnosis and irreversible neurological deterioration. We hereby describe the psychiatric features in CTX. A complete review of all published cases of CTX in the medical literature was undertaken and the case reports with psychiatric presentation were collected and analyzed. We also describe the psychiatric features in relation to the neurological semeiology in six patients with CTX diagnosed at the La Salpêtrière Hospital. We conclude that psychiatric manifestations in CTX follow a bimodal/bitemporal pattern, appearing early in the disease course in the form of a behavioral/personality disorder associated with learning difficulties or mental retardation, or manifesting in advanced disease in the setting of dementia as rich neuropsychiatric syndromes, such as frontal, orbitofrontal or frontotemporal syndromes of cortico-subcortical dementia encompassing behavioral/personality disturbance, affective/mood disorders or psychotic disorders. Behavioral/personality disturbance in childhood or adolescence, especially when accompanied by learning difficulties, should therefore lead to further investigation to exclude CTX, as early diagnosis and treatment is critical for prognosis.
| S-EPMC3784765 | biostudies-literature
Project description:In an ever-changing environment, crossmodal recalibration is crucial to maintain precise and coherent spatial estimates across different sensory modalities. Accordingly, it has been found that perceived auditory space is recalibrated toward vision after consistent exposure to spatially misaligned audio-visual stimuli (VS). While this so-called ventriloquism aftereffect (VAE) yields internal consistency between vision and audition, it does not necessarily lead to consistency between the perceptual representation of space and the actual environment. For this purpose, feedback about the true state of the external world might be necessary. Here, we tested whether the size of the VAE is modulated by external feedback and reward. During adaptation audio-VS with a fixed spatial discrepancy were presented. Participants had to localize the sound and received feedback about the magnitude of their localization error. In half of the sessions the feedback was based on the position of the VS and in the other half it was based on the position of the auditory stimulus. An additional monetary reward was given if the localization error fell below a certain threshold that was based on participants' performance in the pretest. As expected, when error feedback was based on the position of the VS, auditory localization during adaptation trials shifted toward the position of the VS. Conversely, feedback based on the position of the auditory stimuli reduced the visual influence on auditory localization (i.e., the ventriloquism effect) and improved sound localization accuracy. After adaptation with error feedback based on the VS position, a typical auditory VAE (but no visual aftereffect) was observed in subsequent unimodal localization tests. By contrast, when feedback was based on the position of the auditory stimuli during adaptation, no auditory VAE was observed in subsequent unimodal auditory trials. Importantly, in this situation no visual aftereffect was found either. As feedback did not change the physical attributes of the audio-visual stimulation during adaptation, the present findings suggest that crossmodal recalibration is subject to top-down influences. Such top-down influences might help prevent miscalibration of audition toward conflicting visual stimulation in situations in which external feedback indicates that visual information is inaccurate.
| S-EPMC6979315 | biostudies-literature
Project description:Auditory and visual signals are two primary perception modalities that are usually present together and correlate with each other, not only in natural environments but also in clinical settings. However, audio-visual modelling in the latter case can be more challenging, due to the different sources of audio/video signals and the noise (both signal-level and semantic-level) in auditory signals-usually speech audio. In this study, we consider audio-visual modelling in a clinical setting, providing a solution to learn medical representations that benefit various clinical tasks, without relying on dense supervisory annotations from human experts for the model training. A simple yet effective multi-modal self-supervised learning framework is presented for this purpose. The proposed approach is able to help find standard anatomical planes, predict the focusing position of sonographer's eyes, and localise anatomical regions of interest during ultrasound imaging. Experimental analysis on a large-scale clinical multi-modal ultrasound video dataset show that the proposed novel representation learning method provides good transferable anatomical representations that boost the performance of automated downstream clinical tasks, even outperforming fully-supervised solutions. Being able to learn such medical representations in a self-supervised manner will contribute to several aspects including a better understanding of obstetric imaging, training new sonographers, more effective assistive tools for human experts, and enhancement of the clinical workflow.
| S-EPMC11227581 | biostudies-literature
Project description:Learned associations between stimuli in different sensory modalities can shape the way we perceive these stimuli. However, it is not well understood how these interactions are mediated or at what level of the processing hierarchy they occur. Here we describe a neural mechanism by which an auditory input can shape visual representations of behaviorally relevant stimuli through direct interactions between auditory and visual cortices in mice. We show that the association of an auditory stimulus with a visual stimulus in a behaviorally relevant context leads to experience-dependent suppression of visual responses in primary visual cortex (V1). Auditory cortex axons carry a mixture of auditory and retinotopically matched visual input to V1, and optogenetic stimulation of these axons selectively suppresses V1 neurons that are responsive to the associated visual stimulus after, but not before, learning. Our results suggest that cross-modal associations can be communicated by long-range cortical connections and that, with learning, these cross-modal connections function to suppress responses to predictable input.
| S-EPMC8737331 | biostudies-literature