Corpus creation and language identification for code-mixed Indonesian-Javanese-English Tweets.

ABSTRACT: With the massive use of social media today, mixing between languages in social media text is prevalent. In linguistics, the phenomenon of mixing languages is known as code-mixing. The prevalence of code-mixing exposes various concerns and challenges in natural language processing (NLP), including language identification (LID) tasks. This study presents a word-level language identification model for code-mixed Indonesian, Javanese, and English tweets. First, we introduce a code-mixed corpus for Indonesian-Javanese-English language identification (IJELID). To ensure reliable dataset annotation, we provide full details of the data collection and annotation standards construction procedures. Some challenges encountered during corpus creation are also discussed in this paper. Then, we investiga

SUBMITTER: Hidayatullah AF 

PROVIDER: S-EPMC10319257 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature