Cross-modal associations between paintings and sounds: Effects of embodiment.
Ontology highlight
ABSTRACT:
SUBMITTER: Iosifyan M
PROVIDER: S-EPMC9720465 | biostudies-literature | 2022 Dec
REPOSITORIES: biostudies-literature
Publications
Cross-modal associations between paintings and sounds: Effects of embodiment.
Iosifyan Marina M Sidoroff-Dorso Anton A Wolfe Judith J
Perception 20221011 12
The present study investigated cross-modal associations between a series of paintings and sounds. We studied the effects of sound congruency (congruent vs. non-congruent sounds) and embodiment (embodied vs. synthetic sounds) on the evaluation of abstract and figurative paintings. Participants evaluated figurative and abstract paintings paired with congruent and non-congruent embodied and synthetic sounds. They also evaluated the perceived meaningfulness of the paintings, aesthetic value and imme ...[more]
Similar Datasets
Project description:Despite Saussure's famous observation that sound-meaning relationships are in principle arbitrary, we now have a substantial body of evidence that sounds themselves can have meanings, patterns often referred to as "sound symbolism". Previous studies have found that particular sounds can be associated with particular meanings, and also with particular static visual shapes. Less well studied is the association between sounds and dynamic movements. Using a free elicitation method, the current experiment shows that several sound symbolic associations between sounds and dynamic movements exist: (1) front vowels are more likely to be associated with small movements than with large movements; (2) front vowels are more likely to be associated with angular movements than with round movements; (3) obstruents are more likely to be associated with angular movements than with round movements; (4) voiced obstruents are more likely to be associated with large movements than with small movements. All of these results are compatible with the results of the previous studies of sound symbolism using static images or meanings. Overall, the current study supports the hypothesis that particular dynamic motions can be associated with particular sounds. Building on the current results, we discuss a possible practical application of these sound symbolic associations in sports instructions.
| S-EPMC5040269 | biostudies-literature
Project description:The study analyses the existence of cross-modal associations in the general population between a series of paintings and a series of clips of classical (guitar) music. Because of the complexity of the stimuli, the study differs from previous analyses conducted on the association between visual and auditory stimuli, which predominantly analyzed single tones and colors by means of psychophysical methods and forced choice responses. More recently, the relation between music and shape has been analyzed in terms of music visualization, or relatively to the role played by emotion in the association, and free response paradigms have also been accepted. In our study, in order to investigate what attributes may be responsible for the phenomenon of the association between visual and auditory stimuli, the clip/painting association was tested in two experiments: the first used the semantic differential on a unidimensional rating scale of adjectives; the second employed a specific methodology based on subjective perceptual judgments in first person account. Because of the complexity of the stimuli, it was decided to have the maximum possible uniformity of style, composition and musical color. The results show that multisensory features expressed by adjectives such as "quick," "agitated," and "strong," and their antonyms "slow," "calm," and "weak" characterized both the visual and auditory stimuli, and that they may have had a role in the associations. The results also suggest that the main perceptual features responsible for the clip/painting associations were hue, lightness, timbre, and musical tempo. Contrary to what was expected, the musical mode usually related to feelings of happiness (major mode), or to feelings of sadness (minor mode), and spatial orientation (vertical and horizontal) did not play a significant role in the association. The consistency of the associations was shown when evaluated on the whole sample, and after considering the different backgrounds and expertise of the subjects. No substantial difference was found between expert and non-expert subjects. The methods used in the experiment (semantic differential and subjective judgements in first person account) corroborated the interpretation of the results as associations due to patterns of qualitative similarity present in stimuli of different sensory modalities and experienced as such by the subjects. The main result of the study consists in showing the existence of cross-modal associations between highly complex stimuli; furthermore, the second experiment employed a specific methodology based on subjective perceptual judgments.
| S-EPMC4404723 | biostudies-literature
Project description:Cross-modal interactions can lead to enhancement of visual perception, even for visual events below awareness. However, the underlying mechanism is still unclear. Can purely bottom-up cross-modal integration break through the threshold of awareness? We used a binocular rivalry paradigm to measure perceptual switches after brief flashes or sounds which, sometimes, co-occurred. When flashes at the suppressed eye coincided with sounds, perceptual switches occurred the earliest. Yet, contrary to the hypothesis of cross-modal integration, this facilitation never surpassed the assumption of probability summation of independent sensory signals. A follow-up experiment replicated the same pattern of results using silent gaps embedded in continuous noise, instead of sounds. This manipulation should weaken putative sound-flash integration, although keep them salient as bottom-up attention cues. Additional results showed that spatial congruency between flashes and sounds did not determine the effectiveness of cross-modal facilitation, which was again not better than probability summation. Thus, the present findings fail to fully support the hypothesis of bottom-up cross-modal integration, above and beyond the independent contribution of two transient signals, as an account for cross-modal enhancement of visual events below level of awareness.
| S-EPMC5282564 | biostudies-literature
Project description:Pain sensitivity increases when a noxious stimulus is preceded by cues predicting higher intensity. However, it is unclear whether the modulation of nociception by expectancy is sensory-specific ("modality based") or reflects the aversive-affective consequence of the upcoming event ("unpleasantness"), potentially common with other negative events. Here we compared expectancy effects for pain and disgust by using different, but equally unpleasant, nociceptive (thermal) and olfactory stimulations. Indeed both pain and disgust are aversive, associated with threat to the organism, and processed in partly overlapping brain networks. Participants saw cues predicting the unpleasantness (high/low) and the modality (pain/disgust) of upcoming thermal or olfactory stimulations, and rated the associated unpleasantness after stimuli delivery. Results showed that identical thermal stimuli were perceived as more unpleasant when preceded by cues threatening about high (as opposed to low) pain. A similar expectancy effect was found for olfactory disgust. Critically, cross-modal expectancy effects were observed on inconsistent trials when thermal stimuli were preceded by high-disgust cues or olfactory stimuli preceded by high-pain cues. However, these effects were stronger in consistent than inconsistent conditions. Taken together, our results suggest that expectation of an unpleasant event elicits representations of both its modality-specific properties and its aversive consequences.
| S-EPMC4668356 | biostudies-literature
Project description:Despite extensive evidence of the possible interactions between multisensory signals, it remains unclear at what level of sensory processing these interactions take place. When two identical auditory beeps (inducers) are presented in quick succession accompanied by a single visual flash, observers often report seeing two visual flashes, rather than the physical one - the double flash illusion. This compelling illusion has often been considered to reflect direct interactions between neural activations in different primary sensory cortices. Against this simple account, here we show that by simply changing the inducer signals between featurally distinct signals (e.g. high- and low-pitch beeps) the illusory double flash is abolished. This result suggests that a critical component underlying the illusion is perceptual grouping of the inducer signals, consistent with the notion that multisensory combination is preceded by determination of whether the relevant signals share a common source of origin.
| S-EPMC3853685 | biostudies-literature
Project description:Reliable duration perception of external events is necessary to coordinate perception with action, precisely discriminate speech, and for other daily functions. Visual duration perception can be heavily influenced by concurrent auditory signals; however, age-related effects on this process have received minimal attention. In the present study, we examined the effect of aging on duration perception by quantifying (1) duration discrimination thresholds, (2) auditory temporal dominance, and (3) visual duration expansion/compression percepts induced by an accompanying auditory stimulus of longer/shorter duration. Duration discrimination thresholds were significantly greater for visual than auditory tasks in both age groups, however there was no effect of age. While the auditory modality retained dominance in duration perception with age, older adults still performed worse than young adults when comparing durations of two target stimuli (e.g., visual) in the presence of distractors from the other modality (e.g., auditory). Finally, both age groups perceived similar visual duration compression, whereas older adults exhibited visual duration expansion over a wider range of auditory durations compared to their younger counterparts. Results are discussed in terms of multisensory integration and possible decision strategies that change with age.
| S-EPMC7195395 | biostudies-literature
Project description:Temperature is an important characteristic of food and drink. In addition to food-intrinsic temperature (i.e., serving temperature), consumers often experience food-extrinsic temperature (e.g., physical warmth). Emerging research on cross-modal correspondence has revealed that people reliably associate temperature with other sensory features. Building on the literature on cross-modal correspondence and sensation transference theory, the present study aimed to reveal mental representations of temperature-taste correspondence and cross-modal mental representations influencing corresponding sensory/hedonic perceptions of beverages, with a focus on manipulating food-extrinsic warmth. To reveal mental representations of temperature-taste correspondence, Experiment 1 investigated whether temperature words (warm, cool) are associated with sensory/hedonic attributes (e.g., sweet, sour, salty, bitter). The results of Experiment 1 demonstrated that warm (vs. cool) was matched more with saltiness, tastiness, healthfulness, and preference (intention to buy), whereas cool (vs. warm) was matched more with sourness and freshness. Experiment 2 assessed whether cross-modal mental representations influence corresponding sensory/hedonic perceptions of beverages. The participants wore hot and cold pads and rated sensory/hedonic attributes of Japanese tea (Experiment 2a) or black coffee (Experiment 2b) before and after tasting it. The results of Experiment 2a demonstrated that physical warmth (vs. coldness) increased healthfulness and the intention to buy Japanese tea. The results of Experiment 2b did not reveal any effects of physical warmth on sensory/hedonic ratings. These findings provide evidence of taste-temperature correspondence and provide preliminary support for the influence of food-extrinsic warmth on taste attributes related to positivity.
| S-EPMC7546214 | biostudies-literature
Project description:In natural environments objects comprise multiple features from the same or different sensory modalities but it is not known how perception of an object is affected by the value associations of its constituent parts. The present study compares intra- and cross-modal value-driven effects on behavioral and electrophysiological correlates of perception. Human participants first learned the reward associations of visual and auditory cues. Subsequently, they performed a visual discrimination task in the presence of previously rewarded, task-irrelevant visual or auditory cues (intra- and cross-modal cues, respectively). During the conditioning phase, when reward associations were learned and reward cues were the target of the task, high value stimuli of both modalities enhanced the electrophysiological correlates of sensory processing in posterior electrodes. During the post-conditioning phase, when reward delivery was halted and previously rewarded stimuli were task-irrelevant, cross-modal value significantly enhanced the behavioral measures of visual sensitivity, whereas intra-modal value produced only an insignificant decrement. Analysis of the simultaneously recorded event-related potentials (ERPs) of posterior electrodes revealed similar findings. We found an early (90-120 ms) suppression of ERPs evoked by high-value, intra-modal stimuli. Cross-modal stimuli led to a later value-driven modulation, with an enhancement of response positivity for high- compared to low-value stimuli starting at the N1 window (180-250 ms) and extending to the P3 (300-600 ms) responses. These results indicate that sensory processing of a compound stimulus comprising a visual target and task-irrelevant visual or auditory cues is modulated by the reward value of both sensory modalities, but such modulations rely on distinct underlying mechanisms.
| S-EPMC10313067 | biostudies-literature
Project description:An individual may need to take different actions to the same stimulus in different situations to achieve a given goal. The selection of the appropriate action hinges on the previously learned associations between stimuli, actions, and outcomes in the situations. Here, using a go/no-go paradigm and a symmetrical reward, we show that early auditory cortex of nonhuman primates represents such associations, in both the spiking activity and the local field potentials. Sound-evoked neuronal responses changed with sensorimotor associations shortly after sound onset, and the neuronal responses were largest when the sound signaled that a no-go response was required in a trial to obtain a reward. Our findings suggest that association processes take place in the auditory system and do not necessarily rely on association cortex. Thus, auditory cortex may contribute to a rapid selection of the appropriate motor responses to sounds during goal-directed behavior.
| S-EPMC6467566 | biostudies-literature
Project description:In the present study, we examined whether the cross-modal effect can be obtained between odors and colors, which has been confirmed under olfactory recognizable conditions and also occurs under unrecognizable conditions. We used two flavors of red fruits such as strawberries and tomatoes for this purpose. We also aimed to compare whether similar cross-modal effects could be achieved by setting the flavors at recognizable (liminal) and unrecognizable (subliminal) concentrations in the experiment. One flavor at a normal concentration (0.1%, Liminal condition) and one at a concentration below the subliminal threshold (0.015%, Subliminal condition), were presented, and the color that resembled the smell most closely from among the 10 colors, was selected by participants. Except for the subliminal tomato condition, each odor was significantly associated with at least one color (p < 0.01). Participants selected pink and red for liminal strawberry (0.1%) (p < 0.05), pink for subliminal strawberry (0.015%) (p < 0.05), and orange for liminal tomato (0.1%) (p < 0.05), but there was no color selected for subliminal tomato (0.015%) (p < 0.05). The results of this study suggest that the flavor of tomato produced a cross-modal effect in liminal conditions, but not in subliminal conditions. On the other hand, the results of the present study suggest that the flavor of strawberries produces a cross-modal effect even under subliminal conditions. This study showed that cross-modal effects might exist, even at unrecognizable levels of flavor.
| S-EPMC9940644 | biostudies-literature