Assessing Generative AI Chatbots for Alcohol Misuse Support: A Longitudinal Simulation Study.

ABSTRACT: Large language model (LLM)-based chatbots are increasingly used for behavioral health support. Few studies have rigorously evaluated their advice on alcohol misuse. We evaluated seven publicly available chatbots-including general-purpose and behavioral health-focused tools-in responding to alcohol misuse-related questions. Using a fictional case, we simulated longitudinal chatbot interactions over seven days, using 25 prompts derived from real-world Reddit posts. Using an evaluation framework specific to chatbots, four clinicians independently rated each chatbot's transcript along five domains: empathy, quality of information, usefulness, responsiveness, and scope awareness. Clinicians also assessed secondary dimensions, including stigmatizing language and challenging the user (vs. only va

SUBMITTER: Uscher-Pines L 

PROVIDER: S-EPMC12829918 | biostudies-literature | 2026 Feb

REPOSITORIES: biostudies-literature