Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Eardrum-inspired soft viscoelastic diaphragms for CNN-based speech recognition with audio visualization images.

ABSTRACT: In this study, we present initial efforts for a new speech recognition approach aimed at producing different input images for convolutional neural network (CNN)-based speech recognition. We explored the potential of the tympanic membrane (eardrum)-inspired viscoelastic membrane-type diaphragms to deliver audio visualization images using a cross-recurrence plot (CRP). These images were formed by the two phase-shifted vibration responses of viscoelastic diaphragms. We expect this technique to replace the fast Fourier transform (FFT) spectrum currently used for speech recognition. Herein, we report that the new creation method of color images enabled by combining two phase-shifted vibration responses of viscoelastic diaphragms with CRP shows a lower computation burden and a promising potential alternative way to STFT (conventional spectrogram) when the image resolution (pixel size) is below critical resolution.

SUBMITTER: Park SJ

PROVIDER: S-EPMC10115895 | biostudies-literature | 2023 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Eardrum-inspired soft viscoelastic diaphragms for CNN-based speech recognition with audio visualization images.

Park Seok-Jin SJ Lee Hee-Beom HB Kim Gi-Woo GW

Scientific reports 20230419 1

In this study, we present initial efforts for a new speech recognition approach aimed at producing different input images for convolutional neural network (CNN)-based speech recognition. We explored the potential of the tympanic membrane (eardrum)-inspired viscoelastic membrane-type diaphragms to deliver audio visualization images using a cross-recurrence plot (CRP). These images were formed by the two phase-shifted vibration responses of viscoelastic diaphragms. We expect this technique to repl ...[more]

PMID: 37076548

Similar Datasets

Direct visualization of large and protein-free hemifusion diaphragms.

Project description:Fusion of cellular membranes is a ubiquitous biological process requiring remodeling of two phospholipid bilayers. We believe it is very likely that merging of membranes proceeds via similar sequential intermediates. Contacting membranes form a stalk between the proximal leaflets that expands radially into an hemifusion diaphragm (HD) and subsequently open to a fusion pore. Although considered to be a key intermediate in fusion, direct experimental verification of this structure is difficult due to its transient nature. Using confocal fluorescence microscopy we have investigated the fusion of giant unilamellar vesicles (GUVs) containing phosphatidylserine and fluorescent virus derived transmembrane peptides or membrane proteins in the presence of divalent cations. Time-resolved imaging revealed that fusion was preceded by displacement of peptides and fluorescent lipid analogs from the GUV-GUV adhesion region. A detailed analysis of this area being several mum in size revealed that peptides were completely sequestered as expected for an HD. Lateral distribution of lipid analogs was consistent with formation of an HD but not with the presence of two adherent bilayers. Formation and size of the HD were dependent on lipid composition and peptide concentration.

| S-EPMC2849057 | biostudies-literature

Audio-visual speech cue combination.

Project description:BackgroundDifferent sources of sensory information can interact, often shaping what we think we have seen or heard. This can enhance the precision of perceptual decisions relative to those made on the basis of a single source of information. From a computational perspective, there are multiple reasons why this might happen, and each predicts a different degree of enhanced precision. Relatively slight improvements can arise when perceptual decisions are made on the basis of multiple independent sensory estimates, as opposed to just one. These improvements can arise as a consequence of probability summation. Greater improvements can occur if two initially independent estimates are summated to form a single integrated code, especially if the summation is weighted in accordance with the variance associated with each independent estimate. This form of combination is often described as a Bayesian maximum likelihood estimate. Still greater improvements are possible if the two sources of information are encoded via a common physiological process.Principal findingsHere we show that the provision of simultaneous audio and visual speech cues can result in substantial sensitivity improvements, relative to single sensory modality based decisions. The magnitude of the improvements is greater than can be predicted on the basis of either a Bayesian maximum likelihood estimate or a probability summation.ConclusionOur data suggest that primary estimates of speech content are determined by a physiological process that takes input from both visual and auditory processing, resulting in greater sensitivity than would be possible if initially independent audio and visual estimates were formed and then subsequently combined.

| S-EPMC2855706 | biostudies-literature

Oscillations of a soft viscoelastic drop.

Project description:A soft viscoelastic drop has dynamics governed by the balance between surface tension, viscosity, and elasticity, with the material rheology often being frequency dependent, which are utilized in bioprinting technologies for tissue engineering and drop-deposition processes for splash suppression. We study the free and forced oscillations of a soft viscoelastic drop deriving (1) the dispersion relationship for free oscillations, and (2) the frequency response for forced oscillations, of a soft material with arbitrary rheology. We then restrict our analysis to the classical cases of a Kelvin-Voigt and Maxwell model, which are relevant to soft gels and polymer fluids, respectively. We compute the complex frequencies, which are characterized by an oscillation frequency and decay rate, as they depend upon the dimensionless elastocapillary and Deborah numbers and map the boundary between regions of underdamped and overdamped motions. We conclude by illustrating how our theoretical predictions for the frequency-response diagram could be used in conjunction with drop-oscillation experiments as a "drop vibration rheometer", suggesting future experiments using either ultrasonic levitation or a microgravity environment.

| S-EPMC8563899 | biostudies-literature

Contributions of local speech encoding and functional connectivity to audio-visual speech perception.

Project description:Seeing a speaker's face enhances speech intelligibility in adverse environments. We investigated the underlying network mechanisms by quantifying local speech representations and directed connectivity in MEG data obtained while human participants listened to speech of varying acoustic SNR and visual context. During high acoustic SNR speech encoding by temporally entrained brain activity was strong in temporal and inferior frontal cortex, while during low SNR strong entrainment emerged in premotor and superior frontal cortex. These changes in local encoding were accompanied by changes in directed connectivity along the ventral stream and the auditory-premotor axis. Importantly, the behavioral benefit arising from seeing the speaker's face was not predicted by changes in local encoding but rather by enhanced functional connectivity between temporal and inferior frontal cortex. Our results demonstrate a role of auditory-frontal interactions in visual speech representations and suggest that functional connectivity along the ventral pathway facilitates speech comprehension in multisensory environments.

| S-EPMC5462535 | biostudies-literature

SUST Bangla Emotional Speech Corpus (SUBESCO): An audio-only emotional speech corpus for Bangla.

Project description:SUBESCO is an audio-only emotional speech corpus for Bangla language. The total duration of the corpus is in excess of 7 hours containing 7000 utterances, and it is the largest emotional speech corpus available for this language. Twenty native speakers participated in the gender-balanced set, each recording of 10 sentences simulating seven targeted emotions. Fifty university students participated in the evaluation of this corpus. Each audio clip of this corpus, except those of Disgust emotion, was validated four times by male and female raters. Raw hit rates and unbiased rates were calculated producing scores above chance level of responses. Overall recognition rate was reported to be above 70% for human perception tests. Kappa statistics and intra-class correlation coefficient scores indicated high-level of inter-rater reliability and consistency of this corpus evaluation. SUBESCO is an Open Access database, licensed under Creative Common Attribution 4.0 International, and can be downloaded free of charge from the web link: https://doi.org/10.5281/zenodo.4526477.

| S-EPMC8087046 | biostudies-literature

Hemispherical Cell-Inspired Soft Actuator.

Project description:As soft robots have been popular, interest in soft actuators is also increasing. In particular, new types of actuators have been proposed through biomimetics. An actuator that we proposed in this study was inspired by a motor cell that enables plants to move. This actuator is an electrostatic actuator utilizing electrostatic attraction and elastic force, and can be used repeatedly. In addition, this actuator, which can produce large and diverse movements by collecting individual movements like a cell, has a wide application field. As one of them, this actuator is stacked to construct a layer structure and propose an application example. In addition, a piezo sensor was built inside the actuator and real-time motion monitoring was attempted. As a result, the point laser sensor value and the piezo sensor value coincided with each other, which means that it is possible to detect motion in real-time with the built-in sensor.

| S-EPMC7042176 | biostudies-literature

Tough, aorta-inspired soft composites.

Project description:Spatial variations in fiber alignment (and, therefore, in mechanical anisotropy) play a central role in the excellent toughness and fatigue characteristics of many biological materials. In this work, we examine the effect of fiber alignment in soft composites, including both "in-plane" and "out-of-plane" fiber arrangements. We take inspiration from the spatial variations of fiber alignment found in the aorta to three-dimensionally (3D) print soft, tough silicone composites with an excellent combination of stiffness, toughness, and fatigue threshold, regardless of the direction of loading. These aorta-inspired composites exhibit mechanical properties comparable to skin, with excellent combinations of stiffness and toughness not previously observed in synthetic soft materials.

| S-EPMC9282241 | biostudies-literature

Bone-Inspired Bending Soft Robot.

Project description:Bending soft robots must be structured and predictable to be used in applications such as a grasping hand. We developed soft robot fingers with embedded bones to improve the performance of a puppetry robot with haptic feedback. The manufacturing process for bone-inspired soft robots is described, and two mathematical models are reported: one to predict the stiffness and natural frequency of the robot finger and the other for trajectory planning. Experiments using different prototypes were used to set model parameters. The first model, which had a fourth-order lumped mass-spring-damper configuration, was able to predict the natural frequency of the soft robot with a maximum error of 18%. The model and the experimental data demonstrated that bone-inspired soft robots have higher natural frequency, lower phase shift, better controllability, and higher stiffness compared with traditional fiber-reinforced bending soft robots. We also showed that the dynamic performance of a bending soft robot is independent of whether water or air is used for the media and independent of the media pressure. Results from the second model showed that the path of a bone-inspired soft robot is a function of the relative lengths of the bone segments, which means that the model can be used to direct the design of the robot to achieve the desired trajectory. This model was able to correctly predict the trajectory path of the robot.

| S-EPMC8403216 | biostudies-literature

Surface textures suppress viscoelastic braking on soft substrates.

Project description:A gravity-driven droplet will rapidly flow down an inclined substrate, resisted only by stresses inside the liquid. If the substrate is compliant, with an elastic modulus G < 100 kPa, the droplet will markedly slow as a consequence of viscoelastic braking. This phenomenon arises due to deformations of the solid at the moving contact line, enhancing dissipation in the solid phase. Here, we pattern compliant surfaces with textures and probe their interaction with droplets. We show that the superhydrophobic Cassie state, where a droplet is supported atop air-immersed textures, is preserved on soft textured substrates. Confocal microscopy reveals that every texture in contact with the liquid is deformed by capillary stresses. This deformation is coupled to liquid pinning induced by the orientation of contact lines atop soft textures. Thus, compared to flat substrates, greater forcing is required for the onset of drop motion when the soft solid is textured. Surprisingly, droplet velocities down inclined soft or hard textured substrates are indistinguishable; the textures thus suppress viscoelastic braking despite substantial fluid-solid contact. High-speed microscopy shows that contact line velocities atop the pillars vastly exceed those associated with viscoelastic braking. This velocity regime involves less deformation, thus less dissipation, in the solid phase. Such rapid motions are only possible because the textures introduce a new scale and contact-line geometry. The contact-line orientation atop soft pillars induces significant deflections of the pillars on the receding edge of the droplet; calculations confirm that this does not slow down the droplet.

| S-EPMC7768693 | biostudies-literature

Audio-visual speech timing sensitivity is enhanced in cluttered conditions.

Project description:Events encoded in separate sensory modalities, such as audition and vision, can seem to be synchronous across a relatively broad range of physical timing differences. This may suggest that the precision of audio-visual timing judgments is inherently poor. Here we show that this is not necessarily true. We contrast timing sensitivity for isolated streams of audio and visual speech, and for streams of audio and visual speech accompanied by additional, temporally offset, visual speech streams. We find that the precision with which synchronous streams of audio and visual speech are identified is enhanced by the presence of additional streams of asynchronous visual speech. Our data suggest that timing perception is shaped by selective grouping processes, which can result in enhanced precision in temporally cluttered environments. The imprecision suggested by previous studies might therefore be a consequence of examining isolated pairs of audio and visual events. We argue that when an isolated pair of cross-modal events is presented, they tend to group perceptually and to seem synchronous as a consequence. We have revealed greater precision by providing multiple visual signals, possibly allowing a single auditory speech stream to group selectively with the most synchronous visual candidate. The grouping processes we have identified might be important in daily life, such as when we attempt to follow a conversation in a crowded room.

| S-EPMC3071827 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data