Incorporating the image formation process into deep learning improves network performance.
Ontology highlight
ABSTRACT:
SUBMITTER: Li Y
PROVIDER: S-EPMC9636023 | biostudies-literature | 2022 Nov
REPOSITORIES: biostudies-literature
Publications
Incorporating the image formation process into deep learning improves network performance.
Li Yue Y Su Yijun Y Guo Min M Han Xiaofei X Liu Jiamin J Vishwasrao Harshad D HD Li Xuesong X Christensen Ryan R Sengupta Titas T Moyle Mark W MW Rey-Suarez Ivan I Chen Jiji J Upadhyaya Arpita A Usdin Ted B TB Colón-Ramos Daniel Alfonso DA Liu Huafeng H Wu Yicong Y Shroff Hari H
Nature methods 20221031 11
We present Richardson-Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson-Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation process and thereby improving network performance. Containing only roughly 16,000 parameters, RLN enables four- to 50-fold faster processing than purely data-driven networks with many more parameters. By ...[more]
Similar Datasets
Project description:In recent years, deep learning (DL) networks have been widely used in super-resolution (SR) and exhibit improved performance. In this paper, an image quality assessment (IQA)-guided single image super-resolution (SISR) method is proposed in DL architecture, in order to achieve a nice tradeoff between perceptual quality and distortion measure of the SR result. Unlike existing DL-based SR algorithms, an IQA net is introduced to extract perception features from SR results, calculate corresponding loss fused with original absolute pixel loss, and guide the adjustment of SR net parameters. To solve the problem of heterogeneous datasets used by IQA and SR networks, an interactive training model is established via cascaded network. We also propose a pairwise ranking hinge loss method to overcome the shortcomings of insufficient samples during training process. The performance comparison between our proposed method with recent SISR methods shows that the former achieves a better tradeoff between perceptual quality and distortion measure than the latter. Extensive benchmark experiments and analyses also prove that our method provides a promising and opening architecture for SISR, which is not confined to a specific network model.
| S-EPMC7595386 | biostudies-literature
Project description:Engineering simulation accelerates the development of reliable and repeatable design processes in various domains. However, the computing resource consumption is dramatically raised in the whole development processes. Making the most of these simulation data becomes more and more important in modern industrial product design. In the present study, we proposed a workflow comprised of a series of machine learning algorithms (mainly deep neuron networks) to be an alternative to the numerical simulation. We have applied the workflow to the field of dental implant design process. The process is based on a complex, time-dependent, multi-physical biomechanical theory, known as mechano-regulatory method. It has been used to evaluate the performance of dental implants and to assess the tissue recovery after the oral surgery procedures. We provided a deep learning network (DLN) with calibrated simulation data that came from different simulation conditions with experimental verification. The DLN achieves nearly exact result of simulated bone healing history around implants. The correlation of the predicted essential physical properties of surrounding bones (e.g. strain and fluid velocity) and performance indexes of implants (e.g. bone area and bone-implant contact) were greater than 0.980 and 0.947, respectively. The testing AUC values for the classification of each tissue phenotype were ranging from 0.90 to 0.99. The DLN reduced hours of simulation time to seconds. Moreover, our DLN is explainable via Deep Taylor decomposition, suggesting that the transverse fluid velocity, upper and lower parts of dental implants are the keys that influence bone healing and the distribution of tissue phenotypes the most. Many examples of commercial dental implants with designs which follow these design strategies can be found. This work demonstrates that DLN with proper network design is capable to replace complex, time-dependent, multi-physical models/theories, as well as to reveal the underlying features without prior professional knowledge.
| S-EPMC8602721 | biostudies-literature
Project description:Our purpose was to assess the performance of full-dose (FD) PET image synthesis in both image and sinogram space from low-dose (LD) PET images and sinograms without sacrificing diagnostic quality using deep learning techniques. Methods: Clinical brain PET/CT studies of 140 patients were retrospectively used for LD-to-FD PET conversion. Five percent of the events were randomly selected from the FD list-mode PET data to simulate a realistic LD acquisition. A modified 3-dimensional U-Net model was implemented to predict FD sinograms in the projection space (PSS) and FD images in image space (PIS) from their corresponding LD sinograms and images, respectively. The quality of the predicted PET images was assessed by 2 nuclear medicine specialists using a 5-point grading scheme. Quantitative analysis using established metrics including the peak signal-to-noise ratio (PSNR), structural similarity index metric (SSIM), regionwise SUV bias, and first-, second- and high-order texture radiomic features in 83 brain regions for the test and evaluation datasets was also performed. Results: All PSS images were scored 4 or higher (good to excellent) by the nuclear medicine specialists. PSNR and SSIM values of 0.96 ± 0.03 and 0.97 ± 0.02, respectively, were obtained for PIS, and values of 31.70 ± 0.75 and 37.30 ± 0.71, respectively, were obtained for PSS. The average SUV bias calculated over all brain regions was 0.24% ± 0.96% and 1.05% ± 1.44% for PSS and PIS, respectively. The Bland-Altman plots reported the lowest SUV bias (0.02) and variance (95% confidence interval, -0.92 to +0.84) for PSS, compared with the reference FD images. The relative error of the homogeneity radiomic feature belonging to the gray-level cooccurrence matrix category was -1.07 ± 1.77 and 0.28 ± 1.4 for PIS and PSS, respectively. Conclusion: The qualitative assessment and quantitative analysis demonstrated that the FD PET PSS led to superior performance, resulting in higher image quality and lower SUV bias and variance than for FD PET PIS.
| S-EPMC7456177 | biostudies-literature
Project description:BackgroundSingle photon emission computed tomography (SPECT) is an important functional tool for clinical diagnosis and scientific research of brain disorders, but suffers from limited spatial resolution and high noise due to hardware design and imaging physics. The present study is to develop a deep learning technique for SPECT image reconstruction that directly converts raw projection data to image with high resolution and low noise, while an efficient training method specifically applicable to medical image reconstruction is presented.MethodsA custom software was developed to generate 20,000 2-D brain phantoms, of which 16,000 were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To reduce development difficulty, a two-step training strategy for network design was adopted. We first compressed full-size activity image (128×128 pixels) to a one-D vector consisting of 256×1 pixels, accomplished by an autoencoder (AE) consisting of an encoder and a decoder. The vector is a good representation of the full-size image in a lower-dimensional space and was used as a compact label to develop the second network that maps between the projection-data domain and the vector domain. Since the label had 256 pixels only, the second network was compact and easy to converge. The second network, when successfully developed, was connected to the decoder (a portion of AE) to decompress the vector to a regular 128×128 image. Therefore, a complex network was essentially divided into two compact neural networks trained separately in sequence but eventually connectable.ResultsA total of 2,000 test examples, a synthetic brain phantom, and de-identified patient data were used to validate SPECTnet. Results obtained from SPECTnet were compared with those obtained from our clinic OS-EM method. Images with lower noise and more accurate information in the uptake areas were obtained by SPECTnet.ConclusionsThe challenge of developing a complex deep neural network is reduced by training two separate compact connectable networks. The combination of the two networks forms the full version of SPECTnet. Results show that the developed neural network can produce more accurate SPECT images.
| S-EPMC8246183 | biostudies-literature
Project description:Deep learning for ultrasound image formation is rapidly garnering research support and attention, quickly rising as the latest frontier in ultrasound image formation, with much promise to balance both image quality and display speed. Despite this promise, one challenge with identifying optimal solutions is the absence of unified evaluation methods and datasets that are not specific to a single research group. This article introduces the largest known international database of ultrasound channel data and describes the associated evaluation methods that were initially developed for the challenge on ultrasound beamforming with deep learning (CUBDL), which was offered as a component of the 2020 IEEE International Ultrasonics Symposium. We summarize the challenge results and present qualitative and quantitative assessments using both the initially closed CUBDL evaluation test dataset (which was crowd-sourced from multiple groups around the world) and additional in vivo breast ultrasound data contributed after the challenge was completed. As an example quantitative assessment, single plane wave images from the CUBDL Task 1 dataset produced a mean generalized contrast-to-noise ratio (gCNR) of 0.67 and a mean lateral resolution of 0.42 mm when formed with delay-and-sum beamforming, compared with a mean gCNR as high as 0.81 and a mean lateral resolution as low as 0.32 mm when formed with networks submitted by the challenge winners. We also describe contributed CUBDL data that may be used for training of future networks. The compiled database includes a total of 576 image acquisition sequences. We additionally introduce a neural-network-based global sound speed estimator implementation that was necessary to fairly evaluate the results obtained with this international database. The integration of CUBDL evaluation methods, evaluation code, network weights from the challenge winners, and all datasets described herein are publicly available (visit https://cubdl.jhu.edu for details).
| S-EPMC8818124 | biostudies-literature
Project description:Numerous studies have proven the potential of deep learning models for classifying wildlife. Such models can reduce the workload of experts by automating species classification to monitor wild populations and global trade. Although deep learning models typically perform better with more input data, the available wildlife data are ordinarily limited, specifically for rare or endangered species. Recently, citizen science programs have helped accumulate valuable wildlife data, but such data is still not enough to achieve the best performance of deep learning models compared to benchmark datasets. Recent studies have applied the hierarchical classification of a given wildlife dataset to improve model performance and classification accuracy. This study applied hierarchical classification by transfer learning for classifying Amazon parrot species. Specifically, a hierarchy was built based on diagnostic morphological features. Upon evaluating model performance, the hierarchical model outperformed the non-hierarchical model in detecting and classifying Amazon parrots. Notably, the hierarchical model achieved the mean Average Precision (mAP) of 0.944, surpassing the mAP of 0.908 achieved by the non-hierarchical model. Moreover, the hierarchical model improved classification accuracy between morphologically similar species. The outcomes of this study may facilitate the monitoring of wild populations and the global trade of Amazon parrots for conservation purposes.
| S-EPMC11782500 | biostudies-literature
Project description:This study is aimed to explore the performance of texture-based machine learning and image-based deep-learning for enhancing detection of Transitional-zone prostate cancer (TZPCa) in the background of benign prostatic hyperplasia (BPH), using a one-to-one correlation between prostatectomy-based pathologically proven lesion and MRI. Seventy patients confirmed as TZPCa and twenty-nine patients confirmed as BPH without TZPCa by radical prostatectomy. For texture analysis, a radiologist drew the region of interest (ROI) for the pathologically correlated TZPCa and the surrounding BPH on T2WI. Significant features were selected using Least Absolute Shrinkage and Selection Operator (LASSO), trained by 3 types of machine learning algorithms (logistic regression [LR], support vector machine [SVM], and random forest [RF]) and validated by the leave-one-out method. For image-based machine learning, both TZPCa and BPH without TZPCa images were trained using convolutional neural network (CNN) and underwent 10-fold cross validation. Sensitivity, specificity, positive and negative predictive values were presented for each method. The diagnostic performances presented and compared using an ROC curve and AUC value. All the 3 Texture-based machine learning algorithms showed similar AUC (0.854-0.861)among them with generally high specificity (0.710-0.775). The Image-based deep learning showed high sensitivity (0.946) with good AUC (0.802) and moderate specificity (0.643). Texture -based machine learning can be expected to serve as a support tool for diagnosis of human-suspected TZ lesions with high AUC values. Image-based deep learning could serve as a screening tool for detecting suspicious TZ lesions in the context of clinically suspected TZPCa, on the basis of the high sensitivity.
| S-EPMC10545268 | biostudies-literature
Project description:In medical image classification tasks, it is common to find that the number of normal samples far exceeds the number of abnormal samples. In such class-imbalanced situations, reliable training of deep neural networks continues to be a major challenge, therefore biasing the predicted class probabilities toward the majority class. Calibration has been proposed to alleviate some of these effects. However, there is insufficient analysis explaining whether and when calibrating a model would be beneficial. In this study, we perform a systematic analysis of the effect of model calibration on its performance on two medical image modalities, namely, chest X-rays and fundus images, using various deep learning classifier backbones. For this, we study the following variations: (i) the degree of imbalances in the dataset used for training; (ii) calibration methods; and (iii) two classification thresholds, namely, default threshold of 0.5, and optimal threshold from precision-recall (PR) curves. Our results indicate that at the default classification threshold of 0.5, the performance achieved through calibration is significantly superior (p < 0.05) to using uncalibrated probabilities. However, at the PR-guided threshold, these gains are not significantly different (p > 0.05). This observation holds for both image modalities and at varying degrees of imbalance. The code is available at https://github.com/sivaramakrishnan-rajaraman/Model_calibration.
| S-EPMC8794113 | biostudies-literature
Project description:In plant phenotyping, it has become important to be able to measure many features on large image sets in order to aid genetic discovery. The size of the datasets, now often captured robotically, often precludes manual inspection, hence the motivation for finding a fully automated approach. Deep learning is an emerging field that promises unparalleled results on many data analysis problems. Building on artificial neural networks, deep approaches have many more hidden layers in the network, and hence have greater discriminative and predictive power. We demonstrate the use of such approaches as part of a plant phenotyping pipeline. We show the success offered by such techniques when applied to the challenging problem of image-based plant phenotyping and demonstrate state-of-the-art results (>97% accuracy) for root and shoot feature identification and localization. We use fully automated trait identification using deep learning to identify quantitative trait loci in root architecture datasets. The majority (12 out of 14) of manually identified quantitative trait loci were also discovered using our automated approach based on deep learning detection to locate plant features. We have shown deep learning-based phenotyping to have very good detection and localization accuracy in validation and testing image sets. We have shown that such features can be used to derive meaningful biological traits, which in turn can be used in quantitative trait loci discovery pipelines. This process can be completely automated. We predict a paradigm shift in image-based phenotyping bought about by such deep learning approaches, given sufficient training sets.
| S-EPMC5632296 | biostudies-literature
Project description:Purpose: Explainability and fairness are two key factors for the effective and ethical clinical implementation of deep learning-based machine learning models in healthcare settings. However, there has been limited work on investigating how unfair performance manifests in explainable artificial intelligence (XAI) methods, and how XAI can be used to investigate potential reasons for unfairness. Thus, the aim of this work was to analyze the effects of previously established sociodemographic-related confounders on classifier performance and explainability methods. Approach: A convolutional neural network (CNN) was trained to predict biological sex from T1-weighted brain MRI datasets of 4547 9- to 10-year-old adolescents from the Adolescent Brain Cognitive Development study. Performance disparities of the trained CNN between White and Black subjects were analyzed and saliency maps were generated for each subgroup at the intersection of sex and race. Results: The classification model demonstrated a significant difference in the percentage of correctly classified White male ( 90.3%±1.7% ) and Black male ( 81.1%±4.5% ) children. Conversely, slightly higher performance was found for Black female ( 89.3%±4.8% ) compared with White female ( 86.5%±2.0% ) children. Saliency maps showed subgroup-specific differences, corresponding to brain regions previously associated with pubertal development. In line with this finding, average pubertal development scores of subjects used in this study were significantly different between Black and White females ( p<0.001 ) and males ( p<0.001 ). Conclusions: We demonstrate that a CNN with significantly different sex classification performance between Black and White adolescents can identify different important brain regions when comparing subgroup saliency maps. Importance scores vary substantially between subgroups within brain structures associated with pubertal development, a race-associated confounder for predicting sex. We illustrate that unfair models can produce different XAI results between subgroups and that these results may explain potential reasons for biased performance.
| S-EPMC9412191 | biostudies-literature