Dataset Information

Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network.

ABSTRACT: The features of an image can be represented at multiple levels-from its low-level visual properties to high-level meaning. What drives some images to be memorable while others are forgettable? We address this question across two behavioral experiments. In the first, different layers of a convolutional neural network (CNN), which represent progressively higher levels of features, were used to select the images that would be shown to 100 participants through a form of prospective assignment. Here, the discriminability/similarity of an image with others, according to different CNN layers dictated the images presented to different groups, who made a simple indoor versus outdoor judgment for each scene. We found that participants remember more scene images that were selected based on their low-level discriminability or high-level similarity. A second experiment replicated these results in an independent sample of 50 participants, with a different order of postencoding tasks. Together, these experiments provide evidence that both discriminability and similarity, at different visual levels, predict image memorability.

SUBMITTER: Koch GE

PROVIDER: S-EPMC7670863 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network.

Koch Griffin E GE Akpan Essang E Coutanche Marc N MN

Learning & memory (Cold Spring Harbor, N.Y.) 20201116 12

The features of an image can be represented at multiple levels-from its low-level visual properties to high-level meaning. What drives some images to be memorable while others are forgettable? We address this question across two behavioral experiments. In the first, different layers of a convolutional neural network (CNN), which represent progressively higher levels of features, were used to select the images that would be shown to 100 participants through a form of prospective assignment. Here, ...[more]

PMID: 33199475

Similar Datasets

Project description:Images have been shown to consistently differ in terms of their memorability in healthy adults: some images stick in one's mind while others are forgotten quickly. Studies have suggested that memorability is an intrinsic, continuous property of a visual stimulus that can be both measured and manipulated. Memory literature suggests that important developmental changes occur throughout adolescence that have an impact on recognition memory, yet the effect that these changes have on image memorability has not yet been investigated. In the current study, we recruited adolescents ages 11-18 (n = 273, mean = 16) to an online visual memory experiment to explore the effects of developmental changes throughout adolescence on image memorability, and determine if memorability findings in adults can be generalized to the adolescent age group. We used the online experiment to calculate adolescent memorability scores for 1,000 natural images, and compared the results to the MemCat dataset-a memorability dataset that is annotated with adult memorability scores (ages 19-27). Our study finds that memorability scores in adolescents and adults are strongly and significantly correlated (Spearman's rank correlation, r = 0.76, p < 0.001). This correlation persists even when comparing adults with developmentally different sub-groups of adolescents (ages 11-14: r = 0.67, p < 0.001; ages 15-18: r = 0.60, p < 0.001). Moreover, the rankings of image categories by mean memorability scores were identical in both adolescents and adults (including the adolescent sub-groups), indicating that broadly, certain image categories are more memorable for both adolescents and adults. Interestingly, however, adolescents experienced significantly higher false alarm rates than adults, supporting studies that show increased impulsivity and reward-seeking behaviour in adolescents. Our results reveal that the memorability of images remains consistent across individuals at different stages of development. This consistency aligns with and strengthens prior research, indicating that memorability is an intrinsic property of images. Our findings open new pathways for applying memorability studies in adolescent populations, with profound implications in fields such as education, marketing, and psychology. Our work paves the way for innovative approaches in these domains, leveraging the consistent nature of image memorability across age groups.

Project description:BackgroundMedical image segmentation is crucial for improving healthcare outcomes. Convolutional neural networks (CNNs) have been widely applied in medical image analysis; however, their inherent inductive biases limit their ability to capture global contextual information. Vision transformer (ViT) architectures address this limitation by leveraging attention mechanisms to model global relationships; however, they typically require large-scale datasets for effective training, which is challenging in the field of medical imaging due to limited data availability. This study aimed to integrate the advantages of CNN and ViT architectures to improve segmentation performance on small-scale medical image datasets.MethodsIn this study, we established a U-shaped network architecture based on a Transformer-assisted convolutional neural network (TAC-UNet). The TAC-UNet is primarily composed of a hybrid structure integrating CNN and Transformer components. Specifically, the hybrid architecture follows a dual-path design in which the Transformer branch continuously conveys global contextual information to the CNN backbone. This allows the CNN backbone to enhance its global perception while building on the local features it extracts, thereby improving its ability to comprehend complex image structures. A channel cross-attention (CCA) module is also incorporated as a bridge between the encoder and decoder to better reconcile the semantic discrepancies between them.ResultsDetailed experiments on three public datasets were conducted. Specifically, our model was trained on 30 images from the Multi-organ Nucleus Segmentation (MoNuSeg) training dataset, 85 images from the Gland Segmentation (GlaS) training dataset, and 551 images from the Computer Vision Center Colorectal Cancer-Clinic Database (CVC-ClinicDB) dataset. We evaluated the performance of our model on the corresponding test sets. Our TAC-UNet achieved the best Dice scores (80.36%, 90.70%, and 91.81% on the MoNuSeg, GlaS, and CVC-ClinicDB datasets, respectively) of all the models. Compared to other CNN-based, Transformer-based, and hybrid methods, the TAC-UNet demonstrated significantly superior segmentation performance.ConclusionsOur TAC-UNet model showed advanced segmentation performance on small-scale medical image datasets. The detailed experimental results showed the effectiveness of the method. Our model's code is available at: https://github.com/hejlhello/TAC-UNet.

Dataset Information

Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network.

Publications

Image memorability is predicted by discriminability and similarity in different stages of a convolutional neural network.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets