Probing the limitations of multimodal language models for chemistry and materials research.

ABSTRACT: Recent advancements in artificial intelligence have sparked interest in scientific assistants that could support researchers across the full spectrum of scientific workflows, from literature review to experimental design and data analysis. A key capability for such systems is the ability to process and reason about scientific information in both visual and textual forms-from interpreting spectroscopic data to understanding laboratory set-ups. Here we introduce MaCBench, a comprehensive benchmark for evaluating how vision language models handle real-world chemistry and materials science tasks across three core aspects: data extraction, experimental execution and results interpretation. Through a systematic evaluation of leading models, we find that although these systems show promising capa

SUBMITTER: Alampara N 

PROVIDER: S-EPMC12513823 | biostudies-literature | 2025 Oct

REPOSITORIES: biostudies-literature