Statistical inference for natural language processing algorithms with a demonstration using type 2 diabetes prediction from electronic health record notes.

ABSTRACT: The pointwise mutual information statistic (PMI), which measures how often two words occur together in a document corpus, is a cornerstone of recently proposed popular natural language processing algorithms such as word2vec. PMI and word2vec reveal semantic relationships between words and can be helpful in a range of applications such as document indexing, topic analysis, or document categorization. We use probability theory to demonstrate the relationship between PMI and word2vec. We use the theoretical results to demonstrate how the PMI can be modeled and estimated in a simple and straight forward manner. We further describe how one can obtain standard error estimates that account for within-patient clustering that arises from patterns of repeated words within a patient's health record d

SUBMITTER: Egleston BL 

PROVIDER: S-EPMC7854976 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature