metadata
license: apache-2.0
language:
- tr
tags:
- deprem-clf-v1
metrics:
- accuracy
- recall
- f1
library_name: transformers
pipeline_tag: text-classification
model-index:
- name: deprem_v12
results:
- task:
type: text-classification
dataset:
type: deprem_private_dataset_v1_2
name: deprem_private_dataset_v1_2
metrics:
- type: recall
value: 0.75
verified: false
- type: f1
value: 0.75
verified: false
widget:
- text: >-
HATAY DEFNE İLÇESİNE yardımlar gitmiyor Özellikle çadıra battaniyeye
yiyeceğe ihtiyaç var. Antakyanın dışında olduğu için tüm yardimlar
İSKENDERUNA ANTAKYAYA gidiyor. Bu bölgeye gitmiyor..DEFNE İLÇESİNE GİDECEK
ERZAK Çadır yardımlarını
example_title: Örnek
Train-Test Set: "intent-multilabel-v1-2.zip"
Model: "dbmdz/bert-base-turkish-cased"
Tokenizer Params
max_length=128
padding="max_length"
truncation=True
Training Params
evaluation_strategy = "epoch"
save_strategy = "epoch"
per_device_train_batch_size = 16
per_device_eval_batch_size = 16
num_train_epochs = 4
load_best_model_at_end = True
Train-Val Splitting Configuration
train_test_split(df_train,
test_size=0.1,
random_state=1111)
Class Loss Weights
- Alakasiz: 1.0
- Barinma: 1.5167249178108022
- Elektronik: 1.7547338578655642
- Giysi: 1.9610520059358458
- Kurtarma: 1.269341370129623
- Lojistik: 1.8684086209021484
- Saglik: 1.8019018017117145
- Su: 2.110648663094536
- Yagma: 3.081208739200435
- Yemek: 1.7994815143101963
Training Log (Class-Scaled)
Epoch Training Loss Validation Loss
1 No log 0.216295
2 0.260000 0.171498
3 0.142700 0.175608
4 0.142700 0.169851
Threshold Optimization
- Best Threshold: 0.15
- F1 @ Threshold: 0.7503
Eval Results
precision recall f1-score support
Alakasiz 0.91 0.87 0.89 734
Barinma 0.85 0.81 0.83 207
Elektronik 0.72 0.78 0.75 130
Giysi 0.73 0.67 0.70 94
Kurtarma 0.86 0.81 0.83 362
Lojistik 0.68 0.56 0.62 112
Saglik 0.72 0.81 0.76 108
Su 0.61 0.69 0.65 78
Yagma 0.67 0.65 0.66 31
Yemek 0.79 0.85 0.82 117
micro avg 0.82 0.81 0.81 1973
macro avg 0.75 0.75 0.75 1973
weighted avg 0.83 0.81 0.81 1973
samples avg 0.84 0.84 0.83 1973