Inversion-free image recovery from strong aberration using a minimally sampled transmission matrix.
ABSTRACT: A transmission matrix (TM), a characteristic response for an input-output relation of an optical system, has been used for achieving diffraction-limited and aberration-free images through highly-aberrant imaging systems. However, its requirement of acquiring a huge-size TM along with its heavy computational load limit its widespread applications. Here we propose a method for TM-based image reconstruction, which is more efficient in terms of data manipulation and computational time. Only 10% of the TM elements for a fish-eye (FE) lens with strong aberration were sampled compared to that required for the image reconstruction by the conventional inversion method. The missing information was filled in by an iterative interpolation algorithm working in k-space. In addition, as a replacement of the time-consuming matrix inversion process, a phase pattern was created from the minimally sampled TM in order to compensate for the angle-dependent phase retardation caused by the FE lens. The focal distortion could be corrected by applying the phase correction pattern to the angular spectrums of the measured object images. The remaining spatial distortion could also be determined through the geometrical transformation also determined by the minimally sampled TM elements. Through the use of these procedures, the object image can be reconstructed 55 times faster than through the use of the usual inversion method using the full-sized TM, without compromising the reconstruction performances.
Project description:The fish-eye lens camera offers the advantage of efficient acquisition of image data through a wide field of view. However, unlike the popular perspective projection camera, a strong distortion effect appears as the periphery of the image is compressed. Such characteristics must be precisely analyzed through camera self-calibration. In this study, we carried out a fish-eye lens camera self-calibration while considering different types of test objects and projection models. Self-calibration was performed using the V-, A-, Plane-, and Room-type test objects. In the fish-eye lens camera, the V-type test object was the most advantageous for ensuring the accuracy of the principal point coordinates and focal length, because the correlations between parameters were relatively low. On the other hand, the other test objects were advantageous for ensuring the accuracy of distortion parameters because of the well-distributed image points. Based on the above analysis, we proposed, an accurate fish-eye lens camera self-calibration method that applies the V-type test object. The RMS-residuals of the proposed method were less than 1 pixel.
Project description:In this paper, excitation light wavefront modulation is performed considering the curved sample surface shape to demonstrate high-quality deep observation using two-photon excitation microscopy (TPM) with a dry objective lens. A large spherical aberration typically occurs when the refractive index (RI) interface between air and the sample is a plane perpendicular to the optical axis. Moreover, the curved sample surface shape and the RI mismatch cause various aberrations, including spherical ones. Consequently, the fluorescence intensity and resolution of the obtained image are degraded in the deep regions. To improve them, we designed a pre-distortion wavefront for correcting the aberration caused by the curved sample surface shape by using a novel, simple optical path length difference calculation method. The excitation light wavefront is modulated to the pre-distortion wavefront by a spatial light modulator incorporated in the TPM system before passing through the interface, where the RI mismatch occurs. Thus, the excitation light is condensed without aberrations. Blood vessels were thereby observed up to an optical depth of 2,000 ?m in a cleared mouse brain by using a dry objective lens.
Project description:Vision measurement on the basis of structured light plays a significant role in the optical inspection research. The 2D target fixed with a line laser projector is designed to realize the transformations among the world coordinate system, the camera coordinate system and the image coordinate system. The laser projective point and five non-collinear points that are randomly selected from the target are adopted to construct a projection invariant. The closed form solutions of the 3D laser points are solved by the homogeneous linear equations generated from the projection invariants. The optimization function is created by the parameterized re-projection errors of the laser points and the target points in the image coordinate system. Furthermore, the nonlinear optimization solutions of the world coordinates of the projection points, the camera parameters and the lens distortion coefficients are contributed by minimizing the optimization function. The accuracy of the 3D reconstruction is evaluated by comparing the displacements of the reconstructed laser points with the actual displacements. The effects of the image quantity, the lens distortion and the noises are investigated in the experiments, which demonstrate that the reconstruction approach is effective to contribute the accurate test in the measurement system.
Project description:This paper investigates a postprocessing approach to correct spatial distortion in two-photon fluorescence microscopy images for vascular network reconstruction. It is aimed at in vivo imaging of large field-of-view, deep-tissue studies of vascular structures. Based on simple geometric modelling of the object-of-interest, a distortion function is directly estimated from the image volume by deconvolution analysis. Such distortion function is then applied to subvolumes of the image stack to adaptively adjust for spatially varying distortion and reduce the image blurring through blind deconvolution. The proposed technique was first evaluated in phantom imaging of fluorescent microspheres that are comparable in size to the underlying capillary vascular structures. The effectiveness of restoring three-dimensional (3D) spherical geometry of the microspheres using the estimated distortion function was compared with empirically measured point-spread function. Next, the proposed approach was applied to in vivo vascular imaging of mouse skeletal muscle to reduce the image distortion of the capillary structures. We show that the proposed method effectively improve the image quality and reduce spatially varying distortion that occurs in large field-of-view deep-tissue vascular dataset. The proposed method will help in qualitative interpretation and quantitative analysis of vascular structures from fluorescence microscopy images.
Project description:The dynamic personalities and structural heterogeneity of proteins are essential for proper functioning. Structural determination of dynamic/heterogeneous proteins is limited by conventional approaches of X-ray and electron microscopy (EM) of single-particle reconstruction that require an average from thousands to millions different molecules. Cryo-electron tomography (cryoET) is an approach to determine three-dimensional (3D) reconstruction of a single and unique biological object such as bacteria and cells, by imaging the object from a series of tilting angles. However, cconventional reconstruction methods use large-size whole-micrographs that are limited by reconstruction resolution (lower than 20 Å), especially for small and low-symmetric molecule (<400 kDa). In this study, we demonstrated the adverse effects from image distortion and the measuring tilt-errors (including tilt-axis and tilt-angle errors) both play a major role in limiting the reconstruction resolution. Therefore, we developed a "focused electron tomography reconstruction" (FETR) algorithm to improve the resolution by decreasing the reconstructing image size so that it contains only a single-instance protein. FETR can tolerate certain levels of image-distortion and measuring tilt-errors, and can also precisely determine the translational parameters via an iterative refinement process that contains a series of automatically generated dynamic filters and masks. To describe this method, a set of simulated cryoET images was employed; to validate this approach, the real experimental images from negative-staining and cryoET were used. Since this approach can obtain the structure of a single-instance molecule/particle, we named it individual-particle electron tomography (IPET) as a new robust strategy/approach that does not require a pre-given initial model, class averaging of multiple molecules or an extended ordered lattice, but can tolerate small tilt-errors for high-resolution single "snapshot" molecule structure determination. Thus, FETR/IPET provides a completely new opportunity for a single-molecule structure determination, and could be used to study the dynamic character and equilibrium fluctuation of macromolecules.
Project description:Aberrations in optical microscopy reduce image resolution and contrast, and can limit imaging depth when focusing into biological samples. Static correction of aberrations may be achieved through appropriate lens design, but this approach does not offer the flexibility of simultaneously correcting aberrations for all imaging depths, nor the adaptability to correct for sample-specific aberrations for high-quality tomographic optical imaging. Incorporation of adaptive optics (AO) methods have demonstrated considerable improvement in optical image contrast and resolution in noninterferometric microscopy techniques, as well as in optical coherence tomography. Here we present a method to correct aberrations in a tomogram rather than the beam of a broadband optical interferometry system. Based on Fourier optics principles, we correct aberrations of a virtual pupil using Zernike polynomials. When used in conjunction with the computed imaging method interferometric synthetic aperture microscopy, this computational AO enables object reconstruction (within the single scattering limit) with ideal focal-plane resolution at all depths. Tomographic reconstructions of tissue phantoms containing subresolution titanium-dioxide particles and of ex vivo rat lung tissue demonstrate aberration correction in datasets acquired with a highly astigmatic illumination beam. These results also demonstrate that imaging with an aberrated astigmatic beam provides the advantage of a more uniform depth-dependent signal compared to imaging with a standard gaussian beam. With further work, computational AO could enable the replacement of complicated and expensive optical hardware components with algorithms implemented on a standard desktop computer, making high-resolution 3D interferometric tomography accessible to a wider group of users and nonspecialists.
Project description:Camera images and video recordings are simple and non-invasive tools to investigate animals in their natural habitat. Quantitative evaluations, however, often require an exact reconstruction of object positions, sizes, and distances in the image. Here, we provide an open source software package to perform such calculations. Our approach allows the user to correct for perspective distortion, transform images to "bird's-eye" view projections, or transform image-coordinates to real-world coordinates and vice versa. The extrinsic camera parameters that are necessary to perform such image corrections and transformations (elevation, tilt/roll angle, and heading of the camera) are obtained from the image using contextual information such as a visible horizon, GPS coordinates of landmarks, known object sizes, or images of the same object obtained from different viewing angles. All mathematical operations are implemented in the Python package CameraTransform. The performance of the implementation is evaluated using computer-generated synthetic images with known camera parameters. Moreover, we test our algorithm on images of emperor penguin colonies, and demonstrate that the camera tilt and roll angles can be estimated with an error of less than one degree, and the camera elevation with an error of less than 5%. The CameraTransform software package simplifies camera matrix-based image transformations and the extraction of quantitative image information. An extensive documentation and usage examples in an ecological context are provided at http://cameratransform.readthedocs.io.
Project description:Purpose:Investigate the effects of the absence of 17 amino acids at the C-terminal end of Aquaporin 0 (AQP0) on lens transparency, focusing property, and homeostasis. Methods:A knockin (KI) mouse model (AQP0ΔC/ΔC) was developed to express AQP0 only as the end-cleaved form in the lens. For this, AQP0 was genetically engineered as C-terminally end-cleaved with amino acids 1 to 246, instead of the full length 1 to 263 of the wild type (WT). After verifying the KI integration into the genome and its expression, the mouse model was bred for several generations. AQP0 KI homozygous (AQP0ΔC/ΔC) and heterozygous (AQP0+/ΔC) lenses were imaged and analyzed at different developmental stages for transparency. Correspondingly, aberrations in the lens were characterized using the standard metal grid focusing method. Data were compared with age-matched WT, AQP0 knockout (AQP0-/-), and AQP0 heterozygous (AQP0+/-) lenses. Results:AQP0ΔC/ΔC lenses were transparent throughout the embryonic development and until postnatal day 15 (P15) in contrast to age-matched AQP0-/- lenses, which developed cataract at embryonic stage itself. However, there was distortion aberration in AQP0ΔC/ΔC lens at P5; after P15, cataract began to develop and progressed faster surpassing that of age-matched AQP0-/- lenses. AQP0+/ΔC lenses were transparent even at the age of 1 year in contrast to AQP0+/- lenses; however, there was distortion aberration starting at P15. Conclusions:A specific distribution profile of intact and end-cleaved AQP0 from the outer cortex to the inner nucleus is required in the lens for establishing refractive index gradient to enable proper focusing without aberrations and for maintaining transparency.
Project description:Conventional imaging and recognition systems require an extensive amount of data storage, pre-processing, and chip-to-chip communications as well as aberration-proof light focusing with multiple lenses for recognizing an object from massive optical inputs. This is because separate chips (i.e., flat image sensor array, memory device, and CPU) in conjunction with complicated optics should capture, store, and process massive image information independently. In contrast, human vision employs a highly efficient imaging and recognition process. Here, inspired by the human visual recognition system, we present a novel imaging device for efficient image acquisition and data pre-processing by conferring the neuromorphic data processing function on a curved image sensor array. The curved neuromorphic image sensor array is based on a heterostructure of MoS2 and poly(1,3,5-trimethyl-1,3,5-trivinyl cyclotrisiloxane). The curved neuromorphic image sensor array features photon-triggered synaptic plasticity owing to its quasi-linear time-dependent photocurrent generation and prolonged photocurrent decay, originated from charge trapping in the MoS2-organic vertical stack. The curved neuromorphic image sensor array integrated with a plano-convex lens derives a pre-processed image from a set of noisy optical inputs without redundant data storage, processing, and communications as well as without complex optics. The proposed imaging device can substantially improve efficiency of the image acquisition and recognition process, a step forward to the next generation machine vision.
Project description:Intact optical information of an object delivered through an imaging system is deteriorated by imperfect optical elements and unwanted defects. Image deconvolution has been widely exploited as a recovery technique due to its practical feasibility, and operates by assuming linear shift-invariant property of the imaging system. However, shift invariance does not rigorously hold in all imaging situations and is not a necessary condition for solving an inverse problem of light propagation. Several improved deconvolution techniques exploiting spatially variant point spread functions have been proposed in previous studies. However, the full characterization of an optical imaging system for compensating aberrations has not been considered. Here, we present a generalized method to solve the linear inverse problem of coherent light propagations without any regularization method or constraint on shift invariance by fully measuring the transmission matrix of the imaging system. Our results show that severe aberrations produced by a tilted lens or an inserted disordered layer can be corrected properly only by the proposed generalized image deconvolution. This work generalizes the theory of image deconvolution, and enables distortion-free imaging under general imaging condition.