On the use of symmetrized dot patterns for the visual characterization of speech waveforms and other sampled data.
Ontology highlight
ABSTRACT:
SUBMITTER: Pickover CA
PROVIDER: S-EPMC10477927 | biostudies-literature | 1986 Sep
REPOSITORIES: biostudies-literature
Publications
On the use of symmetrized dot patterns for the visual characterization of speech waveforms and other sampled data.
The Journal of the Acoustical Society of America 19860901 3
While the spectrogram (and related graphic analyses) have been invaluable in showing the general frequency content of an input signal, sometimes it is difficult for trained and untrained users to see on the spectrogram differences which are perceptible to the ear. In this paper, several demonstrations of a novel representation are presented which, in some cases, can make subtle differences in input signals obvious to the human analyst. The representation, a "symmetrized dot pattern" (SDP), provi ...[more]
Similar Datasets
Project description:Extracellular recordings were made from 642 units in the primary visual cortex (V1) of a highly visual marsupial, the Tammar wallaby. The receptive field (RF) characteristics of the cells were objectively estimated using the non-linear input model (NIM), and these were correlated with spike shapes. We found that wallaby cortical units had 68% regular spiking (RS), 12% fast spiking (FS), 4% triphasic spiking (TS), 5% compound spiking (CS) and 11% positive spiking (PS). RS waveforms are most often associated with recordings from pyramidal or spiny stellate cell bodies, suggesting that recordings from these cell types dominate in the wallaby cortex. In wallaby, 70-80% of FS and RS cells had orientation selective RFs and had evenly distributed linear and nonlinear RFs. We found that 47% of wallaby PS units were non-orientation selective and they were dominated by linear RFs. Previous studies suggest that the PS units represent recordings from the axon terminals of non-orientation selective cells originating in the lateral geniculate nucleus (LGN). If this is also true in wallaby, as strongly suggested by their low response latencies and bursty spiking properties, the results suggest that significantly more neurons in wallaby LGN are already orientation selective. In wallaby, less than 10% of recorded spikes had triphasic (TS) or sluggish compound spiking (CS) waveforms. These units had a mixture of orientation selective and non-oriented properties, and their cellular origins remain difficult to classify.
| S-EPMC10517629 | biostudies-literature
Project description:We investigated whether visual speech fills in non-intact auditory speech (excised consonant onsets) in typically developing children from 4 to 14 years of age. Stimuli with the excised auditory onsets were presented in the audiovisual (AV) and auditory-only (AO) modes. A visual speech fill-in effect occurs when listeners experience hearing the same non-intact auditory stimulus (e.g., /-b/ag) as different depending on the presence/absence of visual speech such as hearing /bag/ in the AV mode but hearing /ag/ in the AO mode. We quantified the visual speech fill-in effect by the difference in the number of correct consonant onset responses between the modes. We found that easy visual speech cues /b/ provided greater filling in than difficult cues /g/. Only older children benefited from difficult visual speech cues, whereas all children benefited from easy visual speech cues, although 4- and 5-year-olds did not benefit as much as older children. To explore task demands, we compared results on our new task with those on the McGurk task. The influence of visual speech was uniquely associated with age and vocabulary abilities for the visual speech fill--in effect but was uniquely associated with speechreading skills for the McGurk effect. This dissociation implies that visual speech--as processed by children-is a complicated and multifaceted phenomenon underpinned by heterogeneous abilities. These results emphasize that children perceive a speaker's utterance rather than the auditory stimulus per se. In children, as in adults, there is more to speech perception than meets the ear.
| S-EPMC4106987 | biostudies-literature
Project description:When acting in a complex visual environment, it is essential to be able to flexibly allocate attention to parts of the visual scene that may contain goal-relevant information. The paper by Jia et al. provides novel evidence that our brains sequentially sample different objects in a visual scene. The results were obtained using "temporal response functions," in which unique electroencephalographic (EEG) signals corresponding to the processing of 2 continuously presented objects were isolated in an object-specific way. These response functions were dominated by 10-Hz alpha-band activity. Crucially, the different objects were sequentially sampled at a rate of about 2 Hz. These findings provide important neurophysiological insights into how our visual system operates in complex environments.
| S-EPMC5542713 | biostudies-other
Project description:Numerous studies have demonstrated that the vision of lip movements can alter the perception of auditory speech syllables (McGurk effect). While there is ample evidence for integration of text and auditory speech, there are only a few studies on the orthographic equivalent of the McGurk effect. Here, we examined whether written text, like visual speech, can induce an illusory change in the perception of speech sounds on both the behavioural and neural levels. In a sound categorization task, we found that both text and visual speech changed the identity of speech sounds from an /aba/-/ada/ continuum, but the size of this audiovisual effect was considerably smaller for text than visual speech. To examine at which level in the information processing hierarchy these multisensory interactions occur, we recorded electroencephalography in an audiovisual mismatch negativity (MMN, a component of the event-related potential reflecting preattentive auditory change detection) paradigm in which deviant text or visual speech was used to induce an illusory change in a sequence of ambiguous sounds halfway between /aba/ and /ada/. We found that only deviant visual speech induced an MMN, but not deviant text, which induced a late P3-like positive potential. These results demonstrate that text has much weaker effects on sound processing than visual speech does, possibly because text has different biological roots than visual speech.
| S-EPMC5969231 | biostudies-literature
Project description:This study examined whether visual speech provides speech-rhythm information that perceivers can use in speech perception. This was tested by using speech that naturally varied in the familiarity of its rhythm. Thirty Australian English L1 listeners performed a speech perception in noise task with English sentences produced by three speakers: an English L1 speaker (familiar rhythm); an experienced English L2 speaker who had a weak foreign accent (familiar rhythm), and an inexperienced English L2 speaker who had a strong foreign accent (unfamiliar speech rhythm). The spoken sentences were presented in three conditions: Audio-Only (AO), Audio-Visual with mouth covered (AVm), and Audio-Visual (AV). Speech was best recognized in the AV condition regardless of the degree of foreign accent. However, speech recognition in AVm was better than AO for the speech with no foreign accent and with a weak accent, but not for the speech with a strong accent. A follow-up experiment was conducted that only used the speech with a strong foreign accent, under more audible conditions. The results also showed no difference between the AVm and AO conditions, indicating the null effect was not due to a floor effect. We propose that speech rhythm is conveyed by the motion of the jaw opening and closing, and perceivers use this information to better perceive speech in noise.
| S-EPMC10296142 | biostudies-literature
Project description:BackgroundDifferent sources of sensory information can interact, often shaping what we think we have seen or heard. This can enhance the precision of perceptual decisions relative to those made on the basis of a single source of information. From a computational perspective, there are multiple reasons why this might happen, and each predicts a different degree of enhanced precision. Relatively slight improvements can arise when perceptual decisions are made on the basis of multiple independent sensory estimates, as opposed to just one. These improvements can arise as a consequence of probability summation. Greater improvements can occur if two initially independent estimates are summated to form a single integrated code, especially if the summation is weighted in accordance with the variance associated with each independent estimate. This form of combination is often described as a Bayesian maximum likelihood estimate. Still greater improvements are possible if the two sources of information are encoded via a common physiological process.Principal findingsHere we show that the provision of simultaneous audio and visual speech cues can result in substantial sensitivity improvements, relative to single sensory modality based decisions. The magnitude of the improvements is greater than can be predicted on the basis of either a Bayesian maximum likelihood estimate or a probability summation.ConclusionOur data suggest that primary estimates of speech content are determined by a physiological process that takes input from both visual and auditory processing, resulting in greater sensitivity than would be possible if initially independent audio and visual estimates were formed and then subsequently combined.
| S-EPMC2855706 | biostudies-literature
Project description:Many animals have polka dot patterns on their body surface, some of which are known to have signalling functions; however, their evolutionary origins remain unclear. Dot patterns can trigger a fear response (trypophobia) in humans and are known to function as aposematic signals in non-human animals, suggesting that dots may deserve attention for biological reasons. Interestingly in many birds, plumage dot patterns serve for social/sexual signalling. To understand their evolution, we have focused on the sensory bias hypothesis, which predicts the role of pre-existing sensory preference driven by natural selection in shaping signal design. Our previous phylogenetic comparative study supported the hypothesis and showed that diet-driven visual preference promoted the evolution of plumage patterns, as there was an evolutionary correlation between termite-eating (white roundish gregarious prey) and the presence of plumage dot patterns in species of the family Estrildidae. This suggests that these species possess an intrinsic preference for dots. To test this, we compared the responses of an Estrildid species with dot plumage pattern (star finch Neochmia ruficauda) towards simultaneously presented monochrome-printed white dot vs white stripe patterns under both food-deprived and -supplied conditions. Overall, star finches preferred dots to stripes. They showed foraging-like behaviours almost only toward dots when hungry and gazed at dots frequently even when food was available, suggesting both hunger-related and hunger-neutral dot preferences. These results are rather surprising, given how strongly the subjects were attracted to abstract dot patterns without organic structure, but provided good support for the sensory bias hypothesis.
| S-EPMC9617841 | biostudies-literature
Project description:Speech information inherent in face movements is important for understanding what is said in face-to-face communication. Individuals with autism spectrum disorders (ASD) have difficulties in extracting speech information from face movements, a process called visual-speech recognition. Currently, it is unknown what dysfunctional brain regions or networks underlie the visual-speech recognition deficit in ASD. We conducted a functional magnetic resonance imaging (fMRI) study with concurrent eye tracking to investigate visual-speech recognition in adults diagnosed with high-functioning autism and pairwise matched typically developed controls. Compared to the control group (n = 17), the ASD group (n = 17) showed decreased Blood Oxygenation Level Dependent (BOLD) response during visual-speech recognition in the right visual area 5 (V5/MT) and left temporal visual speech area (TVSA) - brain regions implicated in visual-movement perception. The right V5/MT showed positive correlation with visual-speech task performance in the ASD group, but not in the control group. Psychophysiological interaction analysis (PPI) revealed that functional connectivity between the left TVSA and the bilateral V5/MT and between the right V5/MT and the left IFG was lower in the ASD than in the control group. In contrast, responses in other speech-motor regions and their connectivity were on the neurotypical level. Reduced responses and network connectivity of the visual-movement regions in conjunction with intact speech-related mechanisms indicate that perceptual mechanisms might be at the core of the visual-speech recognition deficit in ASD. Communication deficits in ASD might at least partly stem from atypical sensory processing and not higher-order cognitive processing of socially relevant information.
| S-EPMC6202694 | biostudies-literature
Project description:The visual mismatch negativity (vMMN), deriving from the brain's response to stimulus deviance, is thought to be generated by the cortex that represents the stimulus. The vMMN response to visual speech stimuli was used in a study of the lateralization of visual speech processing. Previous research suggested that the right posterior temporal cortex has specialization for processing simple non-speech face gestures, and the left posterior temporal cortex has specialization for processing visual speech gestures. Here, visual speech consonant-vowel (CV) stimuli with controlled perceptual dissimilarities were presented in an electroencephalography (EEG) vMMN paradigm. The vMMNs were obtained using the comparison of event-related potentials (ERPs) for separate CVs in their roles as deviant vs. their roles as standard. Four separate vMMN contrasts were tested, two with the perceptually far deviants (i.e., "zha" or "fa") and two with the near deviants (i.e., "zha" or "ta"). Only far deviants evoked the vMMN response over the left posterior temporal cortex. All four deviants evoked vMMNs over the right posterior temporal cortex. The results are interpreted as evidence that the left posterior temporal cortex represents speech contrasts that are perceived as different consonants, and the right posterior temporal cortex represents face gestures that may not be perceived as different CVs.
| S-EPMC3712324 | biostudies-literature
Project description:Network models are widely used to represent relational information among interacting units and the structural implications of these relations. Recently, social network studies have focused a great deal of attention on random graph models of networks whose nodes represent individual social actors and whose edges represent a specified relationship between the actors. Most inference for social network models assumes that the presence or absence of all possible links is observed, that the information is completely reliable, and that there are no measurement (e.g., recording) errors. This is clearly not true in practice, as much network data is collected though sample surveys. In addition even if a census of a population is attempted, individuals and links between individuals are missed (i.e., do not appear in the recorded data). In this paper we develop the conceptual and computational theory for inference based on sampled network information. We first review forms of network sampling designs used in practice. We consider inference from the likelihood framework, and develop a typology of network data that reflects their treatment within this frame. We then develop inference for social network models based on information from adaptive network designs. We motivate and illustrate these ideas by analyzing the effect of link-tracing sampling designs on a collaboration network.
| S-EPMC4637981 | biostudies-literature