Multimodal Integration of Brain Images for MRI-Based Diagnosis in Schizophrenia.
ABSTRACT: Magnetic resonance imaging (MRI) has been proposed as a source of information for automatic prediction of individual diagnosis in schizophrenia. Optimal integration of data from different MRI modalities is an active area of research aimed at increasing diagnostic accuracy. Based on a sample of 96 patients with schizophrenia and a matched sample of 115 healthy controls that had undergone a single multimodal MRI session, we generated individual brain maps of gray matter vbm, 1back, and 2back levels of activation (nback fMRI), maps of amplitude of low-frequency fluctuations (resting-state fMRI), and maps of weighted global brain connectivity (resting-state fMRI). Four unimodal classifiers (Ridge, Lasso, Random Forests, and Gradient boosting) were applied to these maps to evaluate their classification accuracies. Based on the assignments made by the algorithms on test individuals, we quantified the amount of predictive information shared between maps (what we call redundancy analysis). Finally, we explored the added accuracy provided by a set of multimodal strategies that included post-classification integration based on probabilities, two-step sequential integration, and voxel-level multimodal integration through one-dimensional-convolutional neural networks (1D-CNNs). All four unimodal classifiers showed the highest test accuracies with the 2back maps (80% on average) achieving a maximum of 84% with the Lasso. Redundancy levels between brain maps were generally low (overall mean redundancy score of 0.14 in a 0-1 range), indicating that each brain map contained differential predictive information. The highest multimodal accuracy was delivered by the two-step Ridge classifier (87%) followed by the Ridge maximum and mean probability classifiers (both with 85% accuracy) and by the 1D-CNN, which achieved the same accuracy as the best unimodal classifier (84%). From these results, we conclude that from all MRI modalities evaluated task-based fMRI may be the best unimodal diagnostic option in schizophrenia. Low redundancy values point to ample potential for accuracy improvements through multimodal integration, with the two-step Ridge emerging as a suitable strategy.
Project description:One of the central questions in cognitive neuroscience is the precise neural representation, or brain pattern, associated with a semantic category. In this study, we explored the influence of audiovisual stimuli on the brain patterns of concepts or semantic categories through a functional magnetic resonance imaging (fMRI) experiment. We used a pattern search method to extract brain patterns corresponding to two semantic categories: "old people" and "young people." These brain patterns were elicited by semantically congruent audiovisual, semantically incongruent audiovisual, unimodal visual, and unimodal auditory stimuli belonging to the two semantic categories. We calculated the reproducibility index, which measures the similarity of the patterns within the same category. We also decoded the semantic categories from these brain patterns. The decoding accuracy reflects the discriminability of the brain patterns between two categories. The results showed that both the reproducibility index of brain patterns and the decoding accuracy were significantly higher for semantically congruent audiovisual stimuli than for unimodal visual and unimodal auditory stimuli, while the semantically incongruent stimuli did not elicit brain patterns with significantly higher reproducibility index or decoding accuracy. Thus, the semantically congruent audiovisual stimuli enhanced the within-class reproducibility of brain patterns and the between-class discriminability of brain patterns, and facilitate neural representations of semantic categories or concepts. Furthermore, we analyzed the brain activity in superior temporal sulcus and middle temporal gyrus (STS/MTG). The strength of the fMRI signal and the reproducibility index were enhanced by the semantically congruent audiovisual stimuli. Our results support the use of the reproducibility index as a potential tool to supplement the fMRI signal amplitude for evaluating multimodal integration.
Project description:Invasive Brain-Computer Interfaces (BCIs) require surgeries with high health-risks. The risk-to-benefit ratio of the procedure could potentially be improved by pre-surgically identifying the ideal locations for mental strategy classification. We recorded high-spatiotemporal resolution blood-oxygenation-level-dependent (BOLD) signals using functional MRI at 7 Tesla in eleven healthy participants during two motor imagery tasks. BCI diagnostic task isolated the intent to imagine movements, while BCI simulation task simulated the neural states that may be yielded in a real-life BCI-operation scenario. Imagination of movements were classified from the BOLD signals in sub-regions of activation within a single or multiple dorsal motor network regions. Then, the participant's decoding performance during the BCI simulation task was predicted from the BCI diagnostic task. The results revealed that drawing information from multiple regions compared to a single region increased the classification accuracy of imagined movements. Importantly, systematic unimodal and multimodal classification revealed the ideal combination of regions that yielded the best classification accuracy at the individual-level. Lastly, a given participant's decoding performance achieved during the BCI simulation task could be predicted from the BCI diagnostic task. These results show the feasibility of 7T-fMRI with unimodal and multimodal classification being utilized for identifying ideal sites for mental strategy classification.
Project description:Dysconnectivity between brain regions is thought to underlie the cognitive abnormalities that characterise schizophrenia (SZ). Consistent with this notion functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) studies in SZ have reliably provided evidence of abnormalities in functional integration and in white matter connectivity. Yet little is known about how alterations at the functional level related to abnormalities in anatomical connectivity.We obtained fMRI data during the 2-back working memory task from 25 patients with SZ and 19 healthy controls matched for age, sex and IQ. DTI data were also acquired in the same session. In addition to conventional unimodal analyses we extracted "features" [contrast maps for fMRI and fractional anisotropy (FA) for DTI] that were subjected to joint independent component analysis (JICA) in order to examine interactions between fMRI and DTI data sources.Conventional unimodal analyses revealed both functional and structural deficits in patients with SZ. The JICA identified regions of joint, multimodal brain sources that differed in patients and controls. The fMRI source implicated regions within the anterior cingulate and ventrolateral prefrontal cortex and in the cuneus where patients showed relative hypoactivation and within the frontopolar cortex where patients showed relative hyperactivation. The DTI source localised reduced FA in patients in the splenium and posterior cingulum.This study promotes our understanding of structure-function relationships in SZ by characterising linked functional and white matter changes that contribute to working memory dysfunction in this disorder.
Project description:The purpose of this study was to develop an algorithm incorporating MRI metrics to classify patients with mild traumatic brain injury (mTBI) and controls.This was an institutional review board-approved, Health Insurance Portability and Accountability Act-compliant prospective study. We recruited patients with mTBI and healthy controls through the emergency department and general population. We acquired data on a 3.0T Siemens Trio magnet including conventional brain imaging, resting-state fMRI, diffusion-weighted imaging, and magnetic field correlation (MFC), and performed multifeature analysis using the following MRI metrics: mean kurtosis (MK) of thalamus, MFC of thalamus and frontal white matter, thalamocortical resting-state networks, and 5 regional gray matter and white matter volumes including the anterior cingulum and left frontal and temporal poles. Feature selection was performed using minimal-redundancy maximal-relevance. We used classifiers including support vector machine, naive Bayesian, Bayesian network, radial basis network, and multilayer perceptron to test maximal accuracy.We studied 24 patients with mTBI and 26 controls. Best single-feature classification uses thalamic MK yielding 74% accuracy. Multifeature analysis yields 80% accuracy using the full feature set, and up to 86% accuracy using minimal-redundancy maximal-relevance feature selection (MK thalamus, right anterior cingulate volume, thalamic thickness, thalamocortical resting-state network, thalamic microscopic MFC, and sex).Multifeature analysis using diffusion-weighted imaging, MFC, fMRI, and volumetrics may aid in the classification of patients with mTBI compared with controls based on optimal feature selection and classification methods.This study provides Class III evidence that classification algorithms using multiple MRI features accurately identifies patients with mTBI as defined by American Congress of Rehabilitation Medicine criteria compared with healthy controls.
Project description:Structural and functional MRI unveil many hidden properties of the human brain. We performed this multi-class classification study on selected subjects from the publically available attention deficit hyperactivity disorder ADHD-200 dataset of patients and healthy children. The dataset has three groups, namely, ADHD inattentive, ADHD combined, and typically developing. We calculated the global averaged functional connectivity maps across the whole cortex to extract anatomical atlas parcellation based features from the resting-state fMRI (rs-fMRI) data and cortical parcellation based features from the structural MRI (sMRI) data. In addition, the preprocessed image volumes from both of these modalities followed an ANOVA analysis separately using all the voxels. This study utilized the average measure from the most significant regions acquired from ANOVA as features for classification in addition to the multi-modal and multi-measure features of structural and functional MRI data. We extracted most discriminative features by hierarchical sparse feature elimination and selection algorithm. These features include cortical thickness, image intensity, volume, cortical thickness standard deviation, surface area, and ANOVA based features respectively. An extreme learning machine performed both the binary and multi-class classifications in comparison with support vector machines. This article reports prediction accuracy of both unimodal and multi-modal features from test data. We achieved 76.190% (p < 0.0001) classification accuracy in multi-class settings as well as 92.857% (p < 0.0001) classification accuracy in binary settings. In addition, we found ANOVA-based significant regions of the brain that also play a vital role in the classification of ADHD. Thus, from a clinical perspective, this multi-modal group analysis approach with multi-measure features may improve the accuracy of the ADHD differential diagnosis.
Project description:Identification of imaging biomarkers for schizophrenia is an important but still challenging problem. Even though considerable efforts have been made over the past decades, quantitative alterations between patients and healthy subjects have not yet provided a diagnostic measure with sufficient high sensitivity and specificity. One of the most important reasons is the lack of consistent findings, which is in part due to single-mode study, which only detects single dimensional information by each modality, and thus misses the most crucial differences between groups. Here, we hypothesize that multimodal integration of functional MRI (fMRI), structural MRI (sMRI), and diffusion tensor imaging (DTI) might yield more power for the diagnosis of schizophrenia. A novel multivariate data fusion method for combining these modalities is introduced without reducing the dimension or using the priors from 161 schizophrenia patients and 168 matched healthy controls. The multi-index feature for each ROI is constructed and summarized with Wilk's lambda by performing multivariate analysis of variance to calculate the significant difference between different groups. Our results show that, among these modalities, fMRI has the most significant featureby calculating the Jaccard similarity coefficient (0.7416) and Kappa index (0.4833). Furthermore, fusion of these modalities provides the most plentiful information and the highest predictive accuracy of 86.52%. This work indicates that multimodal integration can improve the ability of distinguishing differences between groups and might be assisting in further diagnosis of schizophrenia.
Project description:Multivariate decoding models are increasingly being applied to functional magnetic imaging (fMRI) data to interpret the distributed neural activity in the human brain. These models are typically formulated to optimize an objective function that maximizes decoding accuracy. For decoding models trained on full-brain data, this can result in multiple models that yield the same classification accuracy, though some may be more reproducible than others--i.e. small changes to the training set may result in very different voxels being selected. This issue of reproducibility can be partially controlled by regularizing the decoding model. Regularization, along with the cross-validation used to estimate decoding accuracy, typically requires retraining many (often on the order of thousands) of related decoding models. In this paper we describe an approach that uses a combination of bootstrapping and permutation testing to construct both a measure of cross-validated prediction accuracy and model reproducibility of the learned brain maps. This requires re-training our classification method on many re-sampled versions of the fMRI data. Given the size of fMRI datasets, this is normally a time-consuming process. Our approach leverages an algorithm called fast simultaneous training of generalized linear models (FaSTGLZ) to create a family of classifiers in the space of accuracy vs. reproducibility. The convex hull of this family of classifiers can be used to identify a subset of Pareto optimal classifiers, with a single-optimal classifier selectable based on the relative cost of accuracy vs. reproducibility. We demonstrate our approach using full-brain analysis of elastic-net classifiers trained to discriminate stimulus type in an auditory and visual oddball event-related fMRI design. Our approach and results argue for a computational approach to fMRI decoding models in which the value of the interpretation of the decoding model ultimately depends upon optimizing a joint space of accuracy and reproducibility.
Project description:Among dementia-like diseases, Alzheimer disease (AD) and vascular dementia (VD) are two of the most frequent. AD and VD may share multiple neurological symptoms that may lead to controversial diagnoses when using conventional clinical and MRI criteria. Therefore, other approaches are needed to overcome this issue. Machine learning (ML) combined with magnetic resonance imaging (MRI) has been shown to improve the diagnostic accuracy of several neurodegenerative diseases, including dementia. To this end, in this study, we investigated, first, whether different kinds of ML algorithms, combined with advanced MRI features, could be supportive in classifying VD from AD and, second, whether the developed approach might help in predicting the prevalent disease in subjects with an unclear profile of AD or VD. Three ML categories of algorithms were tested: artificial neural network (ANN), support vector machine (SVM), and adaptive neuro-fuzzy inference system (ANFIS). Multiple regional metrics from resting-state fMRI (rs-fMRI) and diffusion tensor imaging (DTI) of 60 subjects (33 AD, 27 VD) were used as input features to train the algorithms and find the best feature pattern to classify VD from AD. We then used the identified VD-AD discriminant feature pattern as input for the most performant ML algorithm to predict the disease prevalence in 15 dementia patients with a "mixed VD-AD dementia" (MXD) clinical profile using their baseline MRI data. ML predictions were compared with the diagnosis evidence from a 3-year clinical follow-up. ANFIS emerged as the most efficient algorithm in discriminating AD from VD, reaching a classification accuracy greater than 84% using a small feature pattern. Moreover, ANFIS showed improved classification accuracy when trained with a multimodal input feature data set (e.g., DTI + rs-fMRI metrics) rather than a unimodal feature data set. When applying the best discriminant pattern to the MXD group, ANFIS achieved a correct prediction rate of 77.33%. Overall, results showed that our approach has a high discriminant power to classify AD and VD profiles. Moreover, the same approach also showed potential in predicting earlier the prevalent underlying disease in dementia patients whose clinical profile is uncertain between AD and VD, therefore suggesting its usefulness in supporting physicians' diagnostic evaluations.
Project description:OBJECTIVES:Autism spectrum disorders (ASD) are diagnosed based on early-manifesting clinical symptoms, including markedly impaired social communication. We assessed the viability of resting-state functional MRI (rs-fMRI) connectivity measures as diagnostic biomarkers for ASD and investigated which connectivity features are predictive of a diagnosis. METHODS:Rs-fMRI scans from 59 high functioning males with ASD and 59 age- and IQ-matched typically developing (TD) males were used to build a series of machine learning classifiers. Classification features were obtained using 3 sets of brain regions. Another set of classifiers was built from participants' scores on behavioral metrics. An additional age and IQ-matched cohort of 178 individuals (89 ASD; 89 TD) from the Autism Brain Imaging Data Exchange (ABIDE) open-access dataset (http://fcon_1000.projects.nitrc.org/indi/abide/) were included for replication. RESULTS:High classification accuracy was achieved through several rs-fMRI methods (peak accuracy 76.67%). However, classification via behavioral measures consistently surpassed rs-fMRI classifiers (peak accuracy 95.19%). The class probability estimates, P(ASD|fMRI data), from brain-based classifiers significantly correlated with scores on a measure of social functioning, the Social Responsiveness Scale (SRS), as did the most informative features from 2 of the 3 sets of brain-based features. The most informative connections predominantly originated from regions strongly associated with social functioning. CONCLUSIONS:While individuals can be classified as having ASD with statistically significant accuracy from their rs-fMRI scans alone, this method falls short of biomarker standards. Classification methods provided further evidence that ASD functional connectivity is characterized by dysfunction of large-scale functional networks, particularly those involved in social information processing.