In-context learning enables multimodal large language models to classify cancer pathology images.

ABSTRACT: Medical image classification requires labeled, task-specific datasets which are used to train deep learning networks de novo, or to fine-tune foundation models. However, this process is computationally and technically demanding. In language processing, in-context learning provides an alternative, where models learn from within prompts, bypassing the need for parameter updates. Yet, in-context learning remains underexplored in medical image analysis. Here, we systematically evaluate the model Generative Pretrained Transformer 4 with Vision capabilities (GPT-4V) on cancer image processing with in-context learning on three cancer histopathology tasks of high importance: Classification of tissue subtypes in colorectal cancer, colon polyp subtyping and breast tumor detection in lymph node secti

SUBMITTER: Ferber D 

PROVIDER: S-EPMC11582649 | biostudies-literature | 2024 Nov

REPOSITORIES: biostudies-literature