Shotaro30678 commited on
Commit
2910e15
1 Parent(s): 4adb4e3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -27
README.md CHANGED
@@ -27,33 +27,32 @@ Use dpo trainer to do the RLHF so that the model can be more precise and consist
27
 
28
  ## Model performance
29
 
30
- #### DPO Trained
31
- - **[sentiment score](https://huggingface.co/Shotaro30678/emotion_text_classifier_on_dd_v1)**
32
- - Accuracy: 0.851
33
- - F1-score: 0.85637
34
-
35
- - **[gibberish detection](https://huggingface.co/madhurjindal/autonlp-Gibberish-Detector-492513457)**
36
- | **Clean** | **Mild Gibberish** | **Word Salad** | **Noise** |
37
- |-----------|--------------------|----------------|-----------|
38
- | 882 | 94 | 21 | 3 |
39
-
40
- - **cut-off output**
41
- - complete output: 985
42
- - incomplete output: 15
43
-
44
- #### SFT model (ref)
45
- - **[sentiment score](https://huggingface.co/Shotaro30678/emotion_text_classifier_on_dd_v1)**
46
- - Accuracy: 0.788
47
- - F1-score: 0.79749
48
-
49
- - **[gibberish detection](https://huggingface.co/madhurjindal/autonlp-Gibberish-Detector-492513457)**
50
- | **Clean** | **Mild Gibberish** | **Word Salad** | **Noise** |
51
- |-----------|--------------------|----------------|-----------|
52
- | 898 | 58 | 33 | 11 |
53
-
54
- - **cut-off output**
55
- - complete output: 975
56
- - incomplete output: 25
57
 
58
  on [hermeschen1116/daily_dialog_for_RG](https://huggingface.co/datasets/hermeschen1116/daily_dialog_for_RG) test split.
59
 
 
27
 
28
  ## Model performance
29
 
30
+ ### Model Comparison
31
+
32
+ **Sentiment Score:**
33
+ **[Shotaro30678/emotion_text_classifier_on_dd_v1](https://huggingface.co/Shotaro30678/emotion_text_classifier_on_dd_v1)**
34
+
35
+ | **Metric** | **DPO Trained Model** | **SFT Model (Reference)** |
36
+ |--------------|-----------------------|---------------------------|
37
+ | **Accuracy** | 0.851 | 0.788 |
38
+ | **F1-score** | 0.8564 | 0.7975 |
39
+
40
+ **Gibberish Distribution:**
41
+ **[madhurjindal/autonlp-Gibberish-Detector-492513457](https://huggingface.co/madhurjindal/autonlp-Gibberish-Detector-492513457)**
42
+
43
+ | **Category** | **DPO Trained Model** | **SFT Model (Reference)** |
44
+ |---------------------|-----------------------|---------------------------|
45
+ | **Clean** | 882 | 898 |
46
+ | **Mild Gibberish** | 94 | 58 |
47
+ | **Word Salad** | 21 | 33 |
48
+ | **Noise** | 3 | 11 |
49
+
50
+ **Cut-Off Output:**
51
+
52
+ | **Output Type** | **DPO Trained Model** | **SFT Model (Reference)** |
53
+ |---------------------|-----------------------|---------------------------|
54
+ | **Complete Output** | 985 | 975 |
55
+ | **Incomplete Output** | 15 | 25 |
 
56
 
57
  on [hermeschen1116/daily_dialog_for_RG](https://huggingface.co/datasets/hermeschen1116/daily_dialog_for_RG) test split.
58