Unknown

Dataset Information

How suitable are clinical vignettes for the evaluation of symptom checker apps? A test theoretical perspective.


ABSTRACT:

Objective

To evaluate the ability of case vignettes to assess the performance of symptom checker applications and to suggest refinements to the methodology used in case vignette-based audit studies.

Methods

We re-analyzed the publicly available data of two prominent case vignette-based symptom checker audit studies by calculating common metrics of test theory. Furthermore, we developed a new metric, the Capability Comparison Score (CCS), which compares symptom checker capability while controlling for the difficulty of the set of cases each symptom checker evaluated. We then scrutinized whether applying test theory and the CCS altered the performance ranking of the investigated symptom checkers.

Results

In both studies, most symptom checkers changed their rank order wh

SUBMITTER: Kopka M 

PROVIDER: S-EPMC10444026 | biostudies-literature | 2023 Jan-Dec

REPOSITORIES: biostudies-literature