Evaluating Binary Outcome Classifiers Estimated from Survey Data.

ABSTRACT: Surveys are commonly used to facilitate research in epidemiology, health, and the social and behavioral sciences. Often, these surveys are not simple random samples, and respondents are given weights reflecting their probability of selection into the survey. We show that using survey weights can be beneficial for evaluating the quality of predictive models when splitting data into training and test sets. In particular, we characterize model assessment statistics, such as sensitivity and specificity, as finite population quantities and compute survey-weighted estimates of these quantities with test data comprising a random subset of the original data. Using simulations with data from the National Survey on Drug Use and Health and the National Comorbidity Survey, we show that unweighted metr

SUBMITTER: Wadekar AS 

PROVIDER: S-EPMC12268360 | biostudies-literature | 2024 Nov

REPOSITORIES: biostudies-literature