Explainable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons.
Ontology highlight
ABSTRACT:
SUBMITTER: Tasaki S
PROVIDER: S-EPMC10447434 | biostudies-literature | 2023 Aug
REPOSITORIES: biostudies-literature
Publications
Explainable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons.
Tasaki Shinya S Kim Namhee N Truty Tim T Zhang Ada A Buchman Aron S AS Lamar Melissa M Bennett David A DA
NPJ digital medicine 20230823 1
Hand drawing, which requires multiple neural systems for planning and controlling sequential movements, is a useful cognitive test for older adults. However, the conventional visual assessment of these drawings only captures limited attributes and overlooks subtle details that could help track cognitive states. Here, we utilized a deep-learning model, PentaMind, to examine cognition-related features from hand-drawn images of intersecting pentagons. PentaMind, trained on 13,777 images from 3111 p ...[more]
Similar Datasets
Project description:Inputting molecules into chemistry software, such as quantum chemistry packages, currently requires domain expertise, expensive software and/or cumbersome procedures. Leveraging recent breakthroughs in machine learning, we develop ChemPix: an offline, hand-drawn hydrocarbon structure recognition tool designed to remove these barriers. A neural image captioning approach consisting of a convolutional neural network (CNN) encoder and a long short-term memory (LSTM) decoder learned a mapping from photographs of hand-drawn hydrocarbon structures to machine-readable SMILES representations. We generated a large auxiliary training dataset, based on RDKit molecular images, by combining image augmentation, image degradation and background addition. Additionally, a small dataset of ∼600 hand-drawn hydrocarbon chemical structures was crowd-sourced using a phone web application. These datasets were used to train the image-to-SMILES neural network with the goal of maximizing the hand-drawn hydrocarbon recognition accuracy. By forming a committee of the trained neural networks where each network casts one vote for the predicted molecule, we achieved a nearly 10 percentage point improvement of the molecule recognition accuracy and were able to assign a confidence value for the prediction based on the number of agreeing votes. The ensemble model achieved an accuracy of 76% on hand-drawn hydrocarbons, increasing to 86% if the top 3 predictions were considered.
| S-EPMC8365825 | biostudies-literature
Project description:Hand-drawn mathematical geometric shapes are geometric figures, such as circles, triangles, squares, and polygons, sketched manually using pen and paper or digital tools. These shapes are fundamental in mathematics education and geometric problem-solving, serving as intuitive visual aids for understanding complex concepts and theories. Recognizing hand-drawn shapes accurately enables more efficient digitization of handwritten notes, enhances educational tools, and improves user interaction with mathematical software. This research proposes an innovative machine learning algorithm for the automatic classification of mathematical geometric shapes to identify and interpret these shapes from handwritten input, facilitating seamless integration with digital systems. We utilized a benchmark dataset of mathematical shapes based on a total of 20,000 images with eight classes circle, kite, parallelogram, square, rectangle, rhombus, trapezoid, and triangle. We introduced a novel machine-learning algorithm CnN-RFc that uses convolution neural networks (CNN) for spatial feature extraction and the random forest classifier for probabilistic feature extraction from image data. Experimental results illustrate that using the CnN-RFc method, the Light Gradient Boosting Machine (LGBM) algorithm surpasses state-of-the-art approaches with high accuracy scores of 98% for hand-drawn shape classification. Applications of the proposed mathematical geometric shape classification algorithm span various domains, including education, where it enhances interactive learning platforms and provides instant feedback to students.
| S-EPMC11798587 | biostudies-literature
Project description:New architectures of multilayer artificial neural networks and new methods for training them are rapidly revolutionizing the application of machine learning in diverse fields, including business, social science, physical sciences, and biology. Interpreting deep neural networks, however, currently remains elusive, and a critical challenge lies in understanding which meaningful features a network is actually learning. We present a general method for interpreting deep neural networks and extracting network-learned features from input data. We describe our algorithm in the context of biological sequence analysis. Our approach, based on ideas from statistical physics, samples from the maximum entropy distribution over possible sequences, anchored at an input sequence and subject to constraints implied by the empirical function learned by a network. Using our framework, we demonstrate that local transcription factor binding motifs can be identified from a network trained on ChIP-seq data and that nucleosome positioning signals are indeed learned by a network trained on chemical cleavage nucleosome maps. Imposing a further constraint on the maximum entropy distribution also allows us to probe whether a network is learning global sequence features, such as the high GC content in nucleosome-rich regions. This work thus provides valuable mathematical tools for interpreting and extracting learned features from feed-forward neural networks.
| S-EPMC5679649 | biostudies-literature
Project description:BackgroundWe aim to develop an explainable deep learning (DL) network for the prediction of all-cause mortality directly from positron emission tomography myocardial perfusion imaging flow and perfusion polar map data and evaluate it using prospective testing.MethodsA total of 4735 consecutive patients referred for stress and rest 82Rb positron emission tomography between 2010 and 2018 were followed up for all-cause mortality for 4.15 (2.24-6.3) years. DL network utilized polar maps of stress and rest perfusion, myocardial blood flow, myocardial flow reserve, and spill-over fraction combined with cardiac volumes, singular indices, and sex. Patients scanned from 2010 to 2016 were used for training and validation. The network was tested in a set of 1135 patients scanned from 2017 to 2018 to simulate prospective clinical implementation.ResultsIn prospective testing, the area under the receiver operating characteristic curve for all-cause mortality prediction by DL (0.82 [95% CI, 0.77-0.86]) was higher than ischemia (0.60 [95% CI, 0.54-0.66]; P <0.001), myocardial flow reserve (0.70 [95% CI, 0.64-0.76], P <0.001) or a comprehensive logistic regression model (0.75 [95% CI, 0.69-0.80], P <0.05). The highest quartile of patients by DL had an annual all-cause mortality rate of 11.87% and had a 16.8 ([95% CI, 6.12%-46.3%]; P <0.001)-fold increase in the risk of death compared with the lowest quartile patients. DL showed a 21.6% overall reclassification improvement as compared with established measures of ischemia.ConclusionsThe DL model trained directly on polar maps allows improved patient risk stratification in comparison with established methods for positron emission tomography flow or perfusion assessments.
| S-EPMC10035936 | biostudies-literature
Project description:ImportanceA retinopathy of prematurity (ROP) diagnosis currently relies on indirect ophthalmoscopy assessed by experienced ophthalmologists. A deep learning algorithm based on retinal images may facilitate early detection and timely treatment of ROP to improve visual outcomes.ObjectiveTo develop a retinal image-based, multidimensional, automated, deep learning platform for ROP screening and validate its performance accuracy.Design, setting, and participantsA total of 14 108 eyes of 8652 preterm infants who received ROP screening from 4 centers from November 4, 2010, to November 14, 2019, were included, and a total of 52 249 retinal images were randomly split into training, validation, and test sets. Four main dimensional independent classifiers were developed, including image quality, any stage of ROP, intraocular hemorrhage, and preplus/plus disease. Referral-warranted ROP was automatically generated by integrating the results of 4 classifiers at the image, eye, and patient levels. DeepSHAP, a method based on DeepLIFT and Shapley values (solution concepts in cooperative game theory), was adopted as the heat map technology to explain the predictions. The performance of the platform was further validated as compared with that of the experienced ROP experts. Data were analyzed from February 12, 2020, to June 24, 2020.ExposureA deep learning algorithm.Main outcomes and measuresThe performance of each classifier included true negative, false positive, false negative, true positive, F1 score, sensitivity, specificity, receiver operating characteristic, area under curve (AUC), and Cohen unweighted κ.ResultsA total of 14 108 eyes of 8652 preterm infants (mean [SD] gestational age, 32.9 [3.1] weeks; 4818 boys [60.4%] of 7973 with known sex) received ROP screening. The performance of all classifiers achieved an F1 score of 0.718 to 0.981, a sensitivity of 0.918 to 0.982, a specificity of 0.949 to 0.992, and an AUC of 0.983 to 0.998, whereas that of the referral system achieved an F1 score of 0.898 to 0.956, a sensitivity of 0.981 to 0.986, a specificity of 0.939 to 0.974, and an AUC of 0.9901 to 0.9956. Fine-grained and class-discriminative heat maps were generated by DeepSHAP in real time. The platform achieved a Cohen unweighted κ of 0.86 to 0.98 compared with a Cohen κ of 0.93 to 0.98 by the ROP experts.Conclusions and relevanceIn this diagnostic study, an automated ROP screening platform was able to identify and classify multidimensional pathologic lesions in the retinal images. This platform may be able to assist routine ROP screening in general and children hospitals.
| S-EPMC8100867 | biostudies-literature
Project description:Bombus terrestris, commonly known as the buff-tailed bumblebee, is native to Europe, parts of Africa and Asia. It is commercially bred for use as a pollinator of greenhouse crops. Larvae pupate within a silken cocoon that they construct from proteins produced in modified salivary glands. The amino acid composition and protein structure of hand drawn B. terrestris, silk fibres was investigated through the use of micro-Raman spectroscopy. Spectra were obtained from single fibres drawn from the larvae salivary gland at a rate of 0.14 cm/s. Raman spectroscopy enabled the identification of poly(alanine), poly(alanine-glycine), phenylalanine, tryptophan, and methionine, which is consistent with the results of amino acid analysis. The dominant protein conformation was found to be coiled coil (73%) while the β-sheet content of 10% is, as expected, lower than those reported for hornets and ants. Polarized Raman spectra revealed that the coiled coils were highly aligned along the fibre axis while the β-sheet and random coil components had their peptide carbonyl groups roughly perpendicular to the fibre axis. The protein orientation distribution is compared to those of other natural and recombinant silks. A structural model for the B. terrestris silk fibre is proposed based on these results.
| S-EPMC4964541 | biostudies-literature
Project description:In radiomics, utilizing features extracted from pretrained deep networks could result in models with a higher predictive performance than those relying on hand-crafted features. This study compared the predictive performance of models trained with either deep features, hand-crafted features, or a combination of these features in terms of the area under the receiver-operating characteristic curve (AUC) and other metrics. We trained models on ten radiological datasets using five feature selection methods and three classifiers. Our results indicate that models based on deep features did not show an improved AUC compared to those utilizing hand-crafted features (deep: AUC 0.775, hand-crafted: AUC 0.789; p = 0.28). Including morphological features alongside deep features led to overall improvements in prediction performance for all models (+0.02 gain in AUC; p < 0.001); however, the best model did not benefit from this (+0.003 gain in AUC; p = 0.57). Using all hand-crafted features in addition to the deep features resulted in a further overall improvement (+0.034 in AUC; p < 0.001), but only a minor improvement could be observed for the best model (deep: AUC 0.798, hand-crafted: AUC 0.789; p = 0.92). Furthermore, our results show that models based on deep features extracted from networks pretrained on medical data have no advantage in predictive performance over models relying on features extracted from networks pretrained on ImageNet data. Our study contributes a benchmarking analysis of models trained on hand-crafted and deep features from pretrained networks across multiple datasets. It also provides a comprehensive understanding of their applicability and limitations in radiomics. Our study shows, in conclusion, that models based on features extracted from pretrained deep networks do not outperform models trained on hand-crafted ones.
| S-EPMC10606594 | biostudies-literature
Project description:Convolutional neural networks (CNNs) are revolutionizing digital pathology by enabling machine learning-based classification of a variety of phenotypes from hematoxylin and eosin (H&E) whole slide images (WSIs), but the interpretation of CNNs remains difficult. Most studies have considered interpretability in a post hoc fashion, e.g. by presenting example regions with strongly predicted class labels. However, such an approach does not explain the biological features that contribute to correct predictions. To address this problem, here we investigate the interpretability of H&E-derived CNN features (the feature weights in the final layer of a transfer-learning-based architecture). While many studies have incorporated CNN features into predictive models, there has been little empirical study of their properties. We show such features can be construed as abstract morphological genes ("mones") with strong independent associations to biological phenotypes. Many mones are specific to individual cancer types, while others are found in multiple cancers especially from related tissue types. We also observe that mone-mone correlations are strong and robustly preserved across related cancers. Importantly, linear mone-based classifiers can very accurately separate 38 distinct classes (19 tumor types and their adjacent normals, AUC = [Formula: see text] for each class prediction), and linear classifiers are also highly effective for universal tumor detection (AUC = [Formula: see text]). This linearity provides evidence that individual mones or correlated mone clusters may be associated with interpretable histopathological features or other patient characteristics. In particular, the statistical similarity of mones to gene expression values allows integrative mone analysis via expression-based bioinformatics approaches. We observe strong correlations between individual mones and individual gene expression values, notably mones associated with collagen gene expression in ovarian cancer. Mone-expression comparisons also indicate that immunoglobulin expression can be identified using mones in colon adenocarcinoma and that immune activity can be identified across multiple cancer types, and we verify these findings by expert histopathological review. Our work demonstrates that mones provide a morphological H&E decomposition that can be effectively associated with diverse phenotypes, analogous to the interpretability of transcription via gene expression values. Our work also demonstrates mones can be interpreted without using a classifier as a proxy.
| S-EPMC9177767 | biostudies-literature
Project description:BACKGROUND:There are many types of hand tumors, and it is often difficult for imaging diagnosticians to make a correct diagnosis, which can easily lead to misdiagnosis and delay in treatment. Thus in this paper, we propose a deep neural network for diagnose on MR Images of tumors of the hand in order to better define preoperative diagnosis and standardize surgical treatment. METHODS:We collected MRI figures of 221 patients with hand tumors from one medical center from 2016 to 2019, invited medical experts to annotate the images to form the annotation data set. Then the original image is preprocessed to get the image data set. The data set is randomly divided into ten parts, nine for training and one for test. Next, the data set is input into the neural network system for testing. Finally, average the results of ten experiments as an estimate of the accuracy of the algorithm. RESULTS:This research uses 221 images as dataset and the system shows an average confidence level of 71.6% in segmentation of hand tumors. The segmented tumor regions are validated through ground truth analysis and manual analysis by a radiologist. CONCLUSIONS:With the recent advances in convolutional neural networks, vast improvements have been made for image segmentation, mainly based on the skip-connection-linked encoder decoder deep architectures. Therefore, in this paper, we propose an automatic segmentation method based on DeepLab v3+ and achieved a good diagnostic accuracy rate.
| S-EPMC7428075 | biostudies-literature
Project description:COVID-19 leads to radiological evidence of lower respiratory tract lesions, which support analysis to screen this disease using chest X-ray. In this scenario, deep learning techniques are applied to detect COVID-19 pneumonia in X-ray images, aiding a fast and precise diagnosis. Here, we investigate seven deep learning architectures associated with data augmentation and transfer learning techniques to detect different pneumonia types. We also propose an image resizing method with the maximum window function that preserves anatomical structures of the chest. The results are promising, reaching an accuracy of 99.8% considering COVID-19, normal, and viral and bacterial pneumonia classes. The differentiation between viral pneumonia and COVID-19 achieved an accuracy of 99.8%, and 99.9% of accuracy between COVID-19 and bacterial pneumonia. We also evaluated the impact of the proposed image resizing method on classification performance comparing with the bilinear interpolation; this pre-processing increased the classification rate regardless of the deep learning architectures used. We c ompared our results with ten related works in the state-of-the-art using eight sets of experiments, which showed that the proposed method outperformed them in most cases. Therefore, we demonstrate that deep learning models trained with pre-processed X-ray images could precisely assist the specialist in COVID-19 detection.
| S-EPMC8193974 | biostudies-literature