Incorporating simulated spatial context information improves the effectiveness of contrastive learning models.

ABSTRACT: Visual learning often occurs in a specific context, where an agent acquires skills through exploration and tracking of its location in a consistent environment. The historical spatial context of the agent provides a similarity signal for self-supervised contrastive learning. We present a unique approach, termed environmental spatial similarity (ESS), that complements existing contrastive learning methods. Using images from simulated, photorealistic environments as an experimental setting, we demonstrate that ESS outperforms traditional instance discrimination approaches. Moreover, sampling additional data from the same environment substantially improves accuracy and provides new augmentations. ESS allows remarkable proficiency in room classification and spatial prediction tasks, especially

SUBMITTER: Zhu L 

PROVIDER: S-EPMC11117056 | biostudies-literature | 2024 May

REPOSITORIES: biostudies-literature