Lightweight Language Models are Prone to Reasoning Errors for Complex Computational Phenotyping Tasks.

ABSTRACT:

Objective

Although computational phenotyping is a central informatics activity with resulting cohorts supporting a wide variety of applications, it is time-intensive because of manual data review. We previously assessed the ability of LLMs to perform computational phenotyping tasks using computable phenotypes for ARF respiratory support therapies. They successfully performed concept classification and classification of single-therapy phenotypes, but underperformed on multiple-therapy phenotypes. To understand issues with these complex tasks, we expanded PHEONA, a generalizable framework for evaluation of LLMs, to include methods specifically for evaluating faulty reasoning.

Materials and methods

We assessed the responses of three lightweight LLMs (DeepSeek-r1 32 billion, Mis

SUBMITTER: Yadav S 

PROVIDER: S-EPMC12324558 | biostudies-literature | 2025 Jul

REPOSITORIES: biostudies-literature