Shotaro30678
commited on
Commit
•
2910e15
1
Parent(s):
4adb4e3
Update README.md
Browse files
README.md
CHANGED
@@ -27,33 +27,32 @@ Use dpo trainer to do the RLHF so that the model can be more precise and consist
|
|
27 |
|
28 |
## Model performance
|
29 |
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
- incomplete output: 25
|
57 |
|
58 |
on [hermeschen1116/daily_dialog_for_RG](https://huggingface.co/datasets/hermeschen1116/daily_dialog_for_RG) test split.
|
59 |
|
|
|
27 |
|
28 |
## Model performance
|
29 |
|
30 |
+
### Model Comparison
|
31 |
+
|
32 |
+
**Sentiment Score:**
|
33 |
+
**[Shotaro30678/emotion_text_classifier_on_dd_v1](https://huggingface.co/Shotaro30678/emotion_text_classifier_on_dd_v1)**
|
34 |
+
|
35 |
+
| **Metric** | **DPO Trained Model** | **SFT Model (Reference)** |
|
36 |
+
|--------------|-----------------------|---------------------------|
|
37 |
+
| **Accuracy** | 0.851 | 0.788 |
|
38 |
+
| **F1-score** | 0.8564 | 0.7975 |
|
39 |
+
|
40 |
+
**Gibberish Distribution:**
|
41 |
+
**[madhurjindal/autonlp-Gibberish-Detector-492513457](https://huggingface.co/madhurjindal/autonlp-Gibberish-Detector-492513457)**
|
42 |
+
|
43 |
+
| **Category** | **DPO Trained Model** | **SFT Model (Reference)** |
|
44 |
+
|---------------------|-----------------------|---------------------------|
|
45 |
+
| **Clean** | 882 | 898 |
|
46 |
+
| **Mild Gibberish** | 94 | 58 |
|
47 |
+
| **Word Salad** | 21 | 33 |
|
48 |
+
| **Noise** | 3 | 11 |
|
49 |
+
|
50 |
+
**Cut-Off Output:**
|
51 |
+
|
52 |
+
| **Output Type** | **DPO Trained Model** | **SFT Model (Reference)** |
|
53 |
+
|---------------------|-----------------------|---------------------------|
|
54 |
+
| **Complete Output** | 985 | 975 |
|
55 |
+
| **Incomplete Output** | 15 | 25 |
|
|
|
56 |
|
57 |
on [hermeschen1116/daily_dialog_for_RG](https://huggingface.co/datasets/hermeschen1116/daily_dialog_for_RG) test split.
|
58 |
|