SensiMix: Sensitivity-Aware 8-bit index & 1-bit value mixed precision quantization for BERT compression.

ABSTRACT: Given a pre-trained BERT, how can we compress it to a fast and lightweight one while maintaining its accuracy? Pre-training language model, such as BERT, is effective for improving the performance of natural language processing (NLP) tasks. However, heavy models like BERT have problems of large memory cost and long inference time. In this paper, we propose SensiMix (Sensitivity-Aware Mixed Precision Quantization), a novel quantization-based BERT compression method that considers the sensitivity of different modules of BERT. SensiMix effectively applies 8-bit index quantization and 1-bit value quantization to the sensitive and insensitive parts of BERT, maximizing the compression rate while minimizing the accuracy drop. We also propose three novel 1-bit training methods to minimize the accu

SUBMITTER: Piao T 

PROVIDER: S-EPMC9015158 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature