The influence of preprocessing on text classification using a bag-of-words representation.

ABSTRACT: Text classification (TC) is the task of automatically assigning documents to a fixed number of categories. TC is an important component in many text applications. Many of these applications perform preprocessing. There are different types of text preprocessing, e.g., conversion of uppercase letters into lowercase letters, HTML tag removal, stopword removal, punctuation mark removal, lemmatization, correction of common misspelled words, and reduction of replicated characters. We hypothesize that the application of different combinations of preprocessing methods can improve TC results. Therefore, we performed an extensive and systematic set of TC experiments (and this is our main research contribution) to explore the impact of all possible combinations of five/six basic preprocessing methods

SUBMITTER: HaCohen-Kerner Y 

PROVIDER: S-EPMC7194364 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature