biostudies-literatureNichols ENIA NIH HHSNHLBI NIH HHS81https://www.ebi.ac.uk/biostudies/studies/S-EPMC8961895biostudies-literatureUnknown22(1)<h4>Background</h4>Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.<h4>Methods</h4>We used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.<h4>Results</h4>When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.<h4>Conclusions</h4>Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.BMC medical research methodologyThe effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods.PMC8961895K01 AG052640K01 AG054693HHSN268201700003IHHSN268201700004IHHSN268201700005IU01 HL096917P30 AG066507HHSN268201700001IHHSN268201700002IU01 HL096902R21 AG060243U01 HL096814U01 HL096899P30 AG066587U01 HL096812R01 HL070825K01 AG050699Deal JACarlson MCReed NSArmstrong NMGross ALGriswold MSwenor BKLin FRBandeen-Roche KMosley THAbraham AGRamulu PYNichols ESharrett ARfalseThe effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods.<h4>Background</h4>Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.<h4>Methods</h4>We used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.<h4>Results</h4>When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.<h4>Conclusions</h4>Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.2022-01-01T00:00:00Z2022 Mar2025-04-04T13:22:32.468Z2025-02-19T01:23:03.571ZS-EPMC89618953534605610.1186/s12874-022-01572-2