|
--- |
|
license: apache-2.0 |
|
language: |
|
- tr |
|
tags: |
|
- deprem-clf-v1 |
|
metrics: |
|
- accuracy |
|
- recall |
|
- f1 |
|
library_name: transformers |
|
pipeline_tag: text-classification |
|
model-index: |
|
- name: deprem_v12 |
|
results: |
|
- task: |
|
type: text-classification |
|
dataset: |
|
type: deprem_private_dataset_v1_2 |
|
name: deprem_private_dataset_v1_2 |
|
metrics: |
|
- type: recall |
|
value: 0.75 |
|
verified: false |
|
- type: f1 |
|
value: 0.75 |
|
verified: false |
|
widget: |
|
- text: >- |
|
HATAY DEFNE İLÇESİNE yardımlar gitmiyor Özellikle çadıra battaniyeye yiyeceğe ihtiyaç var. Antakyanın dışında olduğu için tüm yardimlar |
|
İSKENDERUNA ANTAKYAYA gidiyor. Bu bölgeye gitmiyor..DEFNE İLÇESİNE GİDECEK ERZAK Çadır yardımlarını |
|
example_title: Örnek |
|
--- |
|
**Train-Test Set:** "intent-multilabel-v1-2.zip" |
|
|
|
**Model:** "dbmdz/bert-base-turkish-cased" |
|
|
|
## Tokenizer Params |
|
``` |
|
max_length=128 |
|
padding="max_length" |
|
truncation=True |
|
``` |
|
|
|
## Training Params |
|
``` |
|
evaluation_strategy = "epoch" |
|
save_strategy = "epoch" |
|
per_device_train_batch_size = 16 |
|
per_device_eval_batch_size = 16 |
|
num_train_epochs = 4 |
|
load_best_model_at_end = True |
|
``` |
|
|
|
## Train-Val Splitting Configuration |
|
``` |
|
train_test_split(df_train, |
|
test_size=0.1, |
|
random_state=1111) |
|
``` |
|
|
|
## Class Loss Weights |
|
- **Alakasiz:** 1.0 |
|
- **Barinma:** 1.5167249178108022 |
|
- **Elektronik:** 1.7547338578655642 |
|
- **Giysi:** 1.9610520059358458 |
|
- **Kurtarma:** 1.269341370129623 |
|
- **Lojistik:** 1.8684086209021484 |
|
- **Saglik:** 1.8019018017117145 |
|
- **Su:** 2.110648663094536 |
|
- **Yagma:** 3.081208739200435 |
|
- **Yemek:** 1.7994815143101963 |
|
|
|
## Training Log (Class-Scaled) |
|
``` |
|
Epoch Training Loss Validation Loss |
|
1 No log 0.216295 |
|
2 0.260000 0.171498 |
|
3 0.142700 0.175608 |
|
4 0.142700 0.169851 |
|
``` |
|
|
|
## Threshold Optimization |
|
- **Best Threshold:** 0.15 |
|
- **F1 @ Threshold:** 0.7503 |
|
|
|
## Eval Results |
|
``` |
|
precision recall f1-score support |
|
|
|
Alakasiz 0.91 0.87 0.89 734 |
|
Barinma 0.85 0.81 0.83 207 |
|
Elektronik 0.72 0.78 0.75 130 |
|
Giysi 0.73 0.67 0.70 94 |
|
Kurtarma 0.86 0.81 0.83 362 |
|
Lojistik 0.68 0.56 0.62 112 |
|
Saglik 0.72 0.81 0.76 108 |
|
Su 0.61 0.69 0.65 78 |
|
Yagma 0.67 0.65 0.66 31 |
|
Yemek 0.79 0.85 0.82 117 |
|
|
|
micro avg 0.82 0.81 0.81 1973 |
|
macro avg 0.75 0.75 0.75 1973 |
|
weighted avg 0.83 0.81 0.81 1973 |
|
samples avg 0.84 0.84 0.83 1973 |
|
``` |
