Utility Metrics for Evaluating Synthetic Health Data Generation Methods: Validation Study.

ABSTRACT:

Background

A regular task by developers and users of synthetic data generation (SDG) methods is to evaluate and compare the utility of these methods. Multiple utility metrics have been proposed and used to evaluate synthetic data. However, they have not been validated in general or for comparing SDG methods.

Objective

This study evaluates the ability of common utility metrics to rank SDG methods according to performance on a specific analytic workload. The workload of interest is the use of synthetic data for logistic regression prediction models, which is a very frequent workload in health research.

Methods

We evaluated 6 utility metrics on 30 different health data sets and 3 different SDG methods (a Bayesian network, a Generative Adversarial Network, and sequential

SUBMITTER: El Emam K 

PROVIDER: S-EPMC9030990 | biostudies-literature | 2022 Apr

REPOSITORIES: biostudies-literature