Hugging Face's logo

Intel
/
bert-base-uncased-mrpc-int8-qat-inc

Text Classification
Transformers
PyTorch
English
bert
text-classfication
int8
Intel® Neural Compressor
QuantizationAwareTraining
Inference Endpoints
Model card Files Community
2
bert-base-uncased-mrpc-int8-qat-inc / README.md
xinhe's picture
xinhe
Update README.md
8a1d1fd
preview code
|
raw
history blame
1.58 kB
metadata
language: en
license: apache-2.0
tags:
  - text-classfication
  - int8
  - QuantizationAwareTraining
datasets:
  - mrpc
metrics:
  - f1

INT8 BERT base uncased finetuned MRPC

QuantizationAwareTraining

This is an INT8 PyTorch model quantized with Intel® Neural Compressor.

The original fp32 model comes from the fine-tuned model Intel/bert-base-uncased-mrpc.

Test result

INT8 FP32
Throughput (samples/sec) 24.263 11.202
Accuracy (eval-f1) 0.9153 0.9042
Model size (MB) 174 418

Load with Intel® Neural Compressor (build from source): 

from neural_compressor.utils.load_huggingface import OptimizedModel
int8_model = OptimizedModel.from_pretrained(
    'Intel/bert-base-uncased-mrpc-int8-qat',
)

Notes:

  • The INT8 model has better performance than the FP32 model when the CPU is fully occupied. Otherwise, there will be the illusion that INT8 is inferior to FP32.

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3.0
  • train_batch_size: 8
  • eval_batch_size: 8
  • eval_steps: 100
  • load_best_model_at_end: True
  • metric_for_best_model: f1
  • early_stopping_patience = 6
  • early_stopping_threshold = 0.001