Predictive learning as a network mechanism for extracting low-dimensional latent space representations.
Ontology highlight
ABSTRACT:
SUBMITTER: Recanatesi S
PROVIDER: S-EPMC7930246 | biostudies-literature | 2021 Mar
REPOSITORIES: biostudies-literature
Publications
Predictive learning as a network mechanism for extracting low-dimensional latent space representations.
Recanatesi Stefano S Farrell Matthew M Lajoie Guillaume G Deneve Sophie S Rigotti Mattia M Shea-Brown Eric E
Nature communications 20210303 1
Artificial neural networks have recently achieved many successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task's low-dimensional latent structure in the network activity - i.e., in the learned neural representations. Here, we investigate the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure, possibly reflecting an underlying semantic organization, is through learning to ...[more]
Similar Datasets
Project description:In silico prediction of chemical ecotoxicity (HC50) represents an important complement to improve in vivo and in vitro toxicological assessment of manufactured chemicals. Recent application of machine learning models to predict chemical HC50 yields variable prediction performance that depends on effectively learning chemical representations from high-dimension data. To improve HC50 prediction performance, we developed an autoencoder model by learning latent space chemical embeddings. This novel approach achieved state-of-the-art prediction performance of HC50 with R2 of 0.668 ± 0.003 and mean absolute error (MAE) of 0.572 ± 0.001, and outperformed other dimension reduction methods including principal component analysis (PCA) (R2 = 0.601 ± 0.031 and MAE = 0.629 ± 0.005), kernel PCA (R2 = 0.631 ± 0.008 and MAE = 0.625 ± 0.006), and uniform manifold approximation and projection dimensionality reduction (R2 = 0.400 ± 0.008 and MAE = 0.801 ± 0.002). A simple linear layer with chemical embeddings learned from the autoencoder model performed better than random forest (R2 = 0.663 ± 0.007 and MAE = 0.591 ± 0.008), fully connected neural network (R2 = 0.614 ± 0.016 and MAE = 0.610 ± 0.008), least absolute shrinkage and selection operator (R2 = 0.617 ± 0.037 and MAE = 0.619 ± 0.007), and ridge regression (R2 = 0.638 ± 0.007 and MAE = 0.613 ± 0.005) using unlearned raw input features. Our results highlighted the usefulness of learning latent chemical representations, and our autoencoder model provides an alternative approach for robust HC50 prediction.
| S-EPMC9044254 | biostudies-literature
Project description:BackgroundScalable and accurate health outcome prediction using electronic health record (EHR) data has gained much attention in research recently. Previous machine learning models mostly ignore relations between different types of clinical data (ie, laboratory components, International Classification of Diseases codes, and medications).ObjectiveThis study aimed to model such relations and build predictive models using the EHR data from intensive care units. We developed innovative neural network models and compared them with the widely used logistic regression model and other state-of-the-art neural network models to predict the patient's mortality using their longitudinal EHR data.MethodsWe built a set of neural network models that we collectively called as long short-term memory (LSTM) outcome prediction using comprehensive feature relations or in short, CLOUT. Our CLOUT models use a correlational neural network model to identify a latent space representation between different types of discrete clinical features during a patient's encounter and integrate the latent representation into an LSTM-based predictive model framework. In addition, we designed an ablation experiment to identify risk factors from our CLOUT models. Using physicians' input as the gold standard, we compared the risk factors identified by both CLOUT and logistic regression models.ResultsExperiments on the Medical Information Mart for Intensive Care-III dataset (selected patient population: 7537) show that CLOUT (area under the receiver operating characteristic curve=0.89) has surpassed logistic regression (0.82) and other baseline NN models (<0.86). In addition, physicians' agreement with the CLOUT-derived risk factor rankings was statistically significantly higher than the agreement with the logistic regression model.ConclusionsOur results support the applicability of CLOUT for real-world clinical use in identifying patients at high risk of mortality.
| S-EPMC7136840 | biostudies-literature
Project description:The construction of appropriate representations remains essential for molecular predictions due to intricate molecular complexity. Additionally, it is often expensive and ethically constrained to generate labeled data for supervised learning in molecular sciences, leading to challenging small and diverse data sets. In this work, we develop a self-supervised learning approach to pretrain models from over 700 million unlabeled molecules in multiple databases. The intrinsic chemical logic learned from this approach enables the extraction of predictive representations from task-specific molecular sequences in a fine-tuned process. To understand the importance of self-supervised learning from unlabeled molecules, we assemble three models with different combinations of databases. Moreover, we propose a protocol based on data traits to automatically select the optimal model for a specific task. To validate the proposed method, we consider 10 benchmarks and 38 virtual screening data sets. Extensive validation indicates that the proposed method shows superb performance.
| S-EPMC9358546 | biostudies-literature
Project description:A perceptual space is a mental workspace of points in a sensory domain that supports similarity and difference judgments and enables further processing such as classification and naming. Perceptual spaces are present across sensory modalities; examples include colors, faces, auditory textures, and odors. Color is perhaps the best-studied perceptual space, but it is atypical in two respects. First, the dimensions of color space are directly linked to the three cone absorption spectra, but the dimensions of generic perceptual spaces are not as readily traceable to single-neuron properties. Second, generic perceptual spaces have more than three dimensions. This is important because representing each distinguishable point in a high-dimensional space by a separate neuron or population is unwieldy; combinatorial strategies may be needed to overcome this hurdle. To study the representation of a complex perceptual space, we focused on a well-characterized 10-dimensional domain of visual textures. Within this domain, we determine perceptual distances in a threshold task (segmentation) and a suprathreshold task (border salience comparison). In N=4 human observers, we find both quantitative and qualitative differences between these sets of measurements. Quantitatively, observers' segmentation thresholds were inconsistent with their uncertainty determined from border salience comparisons. Qualitatively, segmentation thresholds suggested that distances are determined by a coordinate representation with Euclidean geometry. Border salience comparisons, in contrast, indicated a global curvature of the space, and that distances are determined by activity patterns across broadly tuned elements. Thus, our results indicate two representations of this perceptual space, and suggest that they use differing combinatorial strategies.Significance statementTo move from sensory signals to decisions and actions, the brain carries out a sequence of transformations. An important stage in this process is the construction of a "perceptual space" - an internal workspace of sensory information that captures similarities and differences, and enables further processing, such as classification and naming. Perceptual spaces for color, faces, visual and haptic textures and shapes, sounds, and odors (among others) are known to exist. How such spaces are represented is at present unknown. Here, using visual textures as a model, we investigate this. Psychophysical measurements suggest roles for two combinatorial strategies: one based on projections onto coordinate-like axes, and one based on patterns of activity across broadly tuned elements scattered throughout the space.
| S-EPMC6002902 | biostudies-literature
Project description:Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view. We present a framework to visualize the missed regions per-frame during OC, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. The latter can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.
| S-EPMC8504422 | biostudies-literature
Project description:BackgroundAlzheimer's disease (AD) and frontotemporal dementia (FTD) show network dysfunctions linked with cognitive deficits. Within this framework, network abnormalities between AD and FTD show both convergent and divergent patterns. However, these functional patterns are far from being established and their relevance to cognitive processes remains to be elucidated.MethodsWe investigated the relationship between cognition and functional connectivity of major cognitive networks in these diseases. Twenty-three bvFTD (age: 71±10), 22 AD (age: 72±6), and 20 controls (age: 72±6) underwent cognitive evaluation and resting-state functional MRI. Principal component analysis was used to describe cognitive variance across participants. Brain network connectivity was estimated with connectome analysis. Connectivity matrices were created assessing correlations between parcels within each functional network. The following cognitive networks were considered: default mode (DMN), dorsal attention (DAN), ventral attention (VAN), and frontoparietal (FPN) networks. The relationship between cognition and connectivity was assessed using a bootstrapping correlation and interaction analyses.ResultsThree principal cognitive components explained more than 80% of the cognitive variance: the first component (cogPC1) loaded on memory, the second component (cogPC2) loaded on emotion and language, and the third component (cogPC3) loaded on the visuo-spatial and attentional domains. Compared to HC, AD and bvFTD showed impairment in all cogPCs (p<0.002), and bvFTD scored worse than AD in cogPC2 (p=0.031). At the network level, the DMN showed a significant association in the whole group with cogPC1 and cogPC2 and the VAN with cogPC2. By contrast, DAN and FPN showed a divergent pattern between diagnosis and connectivity for cogPC2. We confirmed these results by means of a multivariate analysis (canonical correlation).ConclusionsA low-dimensional representation can account for a large variance in cognitive scores in the continuum from normal to pathological aging. Moreover, cognitive components showed both convergent and divergent patterns with connectivity across AD and bvFTD. The convergent pattern was observed across the networks primarily involved in these diseases (i.e., the DMN and VAN), while a divergent FC-cognitive pattern was mainly observed between attention/executive networks and the language/emotion cognitive component, suggesting the co-existence of compensatory and detrimental mechanisms underlying these components.
| S-EPMC9798659 | biostudies-literature
Project description:Humans possess an exceptional aptitude to efficiently make decisions from high-dimensional sensory observations. However, it is unknown how the brain compactly represents the current state of the environment to guide this process. The deep Q-network (DQN) achieves this by capturing highly nonlinear mappings from multivariate inputs to the values of potential actions. We deployed DQN as a model of brain activity and behavior in participants playing three Atari video games during fMRI. Hidden layers of DQN exhibited a striking resemblance to voxel activity in a distributed sensorimotor network, extending throughout the dorsal visual pathway into posterior parietal cortex. Neural state-space representations emerged from nonlinear transformations of the pixel space bridging perception to action and reward. These transformations reshape axes to reflect relevant high-level features and strip away information about task-irrelevant sensory features. Our findings shed light on the neural encoding of task representations for decision-making in real-world situations.
| S-EPMC7897245 | biostudies-literature
Project description:Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Importantly, without supervision, human-understandable scene attributes, including the object's shape, material, and body color, spontaneously emerge in the model's layer-wise latent space in a scale-specific manner. By embedding an image into the learned latent space, we can manipulate specific layers' latent code to modify the appearance of the object in the image. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object's shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object's shape or body color. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts.
| S-EPMC9942964 | biostudies-literature
Project description:Social relationships consist of interactions along multiple dimensions. In social networks, this means that individuals form multiple types of relationships with the same person (e.g., an individual will not trust all of his/her acquaintances). Statistical models for these data require understanding two related types of dependence structure: (i) structure within each relationship type, or network view, and (ii) the association between views. In this paper, we propose a statistical framework that parsimoniously represents dependence between relationship types while also maintaining enough flexibility to allow individuals to serve different roles in different relationship types. Our approach builds on work on latent space models for networks [see, e.g., J. Amer. Statist. Assoc.97 (2002) 1090-1098]. These models represent the propensity for two individuals to form edges as conditionally independent given the distance between the individuals in an unobserved social space. Our work departs from previous work in this area by representing dependence structure between network views through a multivariate Bernoulli likelihood, providing a representation of between-view association. This approach infers correlations between views not explained by the latent space model. Using our method, we explore 6 multiview network structures across 75 villages in rural southern Karnataka, India [Banerjee et al. (2013)].
| S-EPMC5927604 | biostudies-literature
Project description:BackgroundIncreased data availability has prompted the creation of clinical decision support systems. These systems utilise clinical information to enhance health care provision, both to predict the likelihood of specific clinical outcomes or evaluate the risk of further complications. However, their adoption remains low due to concerns regarding the quality of recommendations, and a lack of clarity on how results are best obtained and presented.MethodsWe used autoencoders capable of reducing the dimensionality of complex datasets in order to produce a 2D representation denoted as latent space to support understanding of complex clinical data. In this output, meaningful representations of individual patient profiles are spatially mapped in an unsupervised manner according to their input clinical parameters. This technique was then applied to a large real-world clinical dataset of over 12,000 patients with an illness compatible with dengue infection in Ho Chi Minh City, Vietnam between 1999 and 2021. Dengue is a systemic viral disease which exerts significant health and economic burden worldwide, and up to 5% of hospitalised patients develop life-threatening complications.ResultsThe latent space produced by the selected autoencoder aligns with established clinical characteristics exhibited by patients with dengue infection, as well as features of disease progression. Similar clinical phenotypes are represented close to each other in the latent space and clustered according to outcomes broadly described by the World Health Organisation dengue guidelines. Balancing distance metrics and density metrics produced results covering most of the latent space, and improved visualisation whilst preserving utility, with similar patients grouped closer together. In this case, this balance is achieved by using the sigmoid activation function and one hidden layer with three neurons, in addition to the latent dimension layer, which produces the output (Pearson, 0.840; Spearman, 0.830; Procrustes, 0.301; GMM 0.321).ConclusionThis study demonstrates that when adequately configured, autoencoders can produce two-dimensional representations of a complex dataset that conserve the distance relationship between points. The output visualisation groups patients with clinically relevant features closely together and inherently supports user interpretability. Work is underway to incorporate these findings into an electronic clinical decision support system to guide individual patient management.
| S-EPMC9992802 | biostudies-literature