README.md

metadata

license: cc-by-nc-4.0
language:
  - hu
  - en
metrics:
  - accuracy
  - f1
model-index:
  - name: Hun_Eng_RoBERTa_base_Plain
    results:
      - task:
          type: text-classification
        metrics:
          - type: accuracy
            value: 0.75 (hu) / 0.65 (en)
          - type: f1
            value: 0.74 (hu) / 0.64 (en)
widget:
  - text: >-
      A tanúsítvány meghatározott adatainak a 2008/118/EK irányelv IV. fejezete
      szerinti szállításához szükséges adminisztratív okmányban...
    example_title: Incomprehensible
  - text: >-
      Az AEO-engedély birtokosainak listáján – keresésre – megjelenő
      információk: az engedélyes neve, az engedélyt kibocsátó ország...
    example_title: Comprehensible

Model description

Cased fine-tuned XLM-RoBERTa-base model for Hungarian and English, trained on datasets provided by the National Tax and Customs Administration - Hungary (NAV) and translated versions of the same dataset using Google Translate API.

Intended uses & limitations

The model is designed to classify sentences as either "comprehensible" or "not comprehensible" (according to Plain Language guidelines):

• Label_0 - "comprehensible" - The sentence is in Plain Language.
• Label_1 - "not comprehensible" - The sentence is not in Plain Language.

Training

Fine-tuned version of the original xlm-roberta-base model, trained on a dataset of Hungarian legal and administrative texts. The model was also trained on the translated version of this dataset (via Google Translate API) for English classification.

Eval results

Hungarian Results:

Class	Precision	Recall	F1-Score
Comprehensible / Label_0	0.82	0.62	0.70
Not comprehensible / Label_1	0.71	0.88	0.78
accuracy			0.75
macro avg	0.77	0.75	0.74
weighted avg	0.76	0.75	0.74

English Results:

Class	Precision	Recall	F1-Score
Comprehensible / Label_0	0.70	0.50	0.58
Not comprehensible / Label_1	0.63	0.80	0.70
accuracy			0.65
macro avg	0.66	0.65	0.64
weighted avg	0.66	0.65	0.64

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("uvegesistvan/Hun_Eng_RoBERTa_base_Plain")
model = AutoModelForSequenceClassification.from_pretrained("uvegesistvan/Hun_Eng_RoBERTa_base_Plain")