jgrosjean commited on
Commit
ab82a4b
1 Parent(s): c6948ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -142,7 +142,7 @@ The fine-tuning script can be accessed [here](https://github.com/jgrosjean-mathe
142
 
143
  The two evaluation tasks make use of the [20 Minuten dataset](https://www.zora.uzh.ch/id/eprint/234387/) compiled by Kew et al. (2023), which contains Swiss news articles with topic tags and summaries. Parts of the dataset were automatically translated to French, Italian using a Google Cloud API and to Romash via a [Textshuttle](https://textshuttle.com/en) API.
144
 
145
- #### Evaluation via Semantic Textual Similarity
146
 
147
  <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
148
 
@@ -166,10 +166,10 @@ Sentence SwissBERT achieves comparable or better results as the best-performing
166
  | Evaluation task |Swissbert | |Sentence Swissbert| |Sentence-BERT| |
167
  |------------------------|----------|-----------|------------------|-----------|-------------|-----------|
168
  | |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
169
- | Semantic Similarity DE | 87.20 % | -- | **93.40 %** | -- | 91.80 % | -- |
170
- | Semantic Similarity FR | 84.97 % | -- | **93.99 %** | -- | 93.19 % | -- |
171
- | Semantic Similarity IT | 84.17 % | -- | **92.18 %** | -- | 91.58 % | -- |
172
- | Semantic Similarity RM | 83.17 % | -- | **91.58 %** | -- | 73.35 % | -- |
173
  | Text Classification DE | -- | 77.93 % | -- |**78.49 %**| -- | 77.23 % |
174
  | Text Classification FR | -- | 69.62 % | -- |**77.18 %**| -- | 76.83 % |
175
  | Text Classification IT | -- | 67.09 % | -- | 76.65 % | -- |**76.90 %**|
 
142
 
143
  The two evaluation tasks make use of the [20 Minuten dataset](https://www.zora.uzh.ch/id/eprint/234387/) compiled by Kew et al. (2023), which contains Swiss news articles with topic tags and summaries. Parts of the dataset were automatically translated to French, Italian using a Google Cloud API and to Romash via a [Textshuttle](https://textshuttle.com/en) API.
144
 
145
+ #### Evaluation via Document Retrieval
146
 
147
  <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
148
 
 
166
  | Evaluation task |Swissbert | |Sentence Swissbert| |Sentence-BERT| |
167
  |------------------------|----------|-----------|------------------|-----------|-------------|-----------|
168
  | |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
169
+ | Document Retrieval DE | 87.20 % | -- | **93.40 %** | -- | 91.80 % | -- |
170
+ | Document Retrieval FR | 84.97 % | -- | **93.99 %** | -- | 93.19 % | -- |
171
+ | Document Retrieval IT | 84.17 % | -- | **92.18 %** | -- | 91.58 % | -- |
172
+ | Document Retrieval RM | 83.17 % | -- | **91.58 %** | -- | 73.35 % | -- |
173
  | Text Classification DE | -- | 77.93 % | -- |**78.49 %**| -- | 77.23 % |
174
  | Text Classification FR | -- | 69.62 % | -- |**77.18 %**| -- | 76.83 % |
175
  | Text Classification IT | -- | 67.09 % | -- | 76.65 % | -- |**76.90 %**|