patrickvonplaten commited on
Commit
b5c7a9c
·
1 Parent(s): e5d0e83

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -101,9 +101,9 @@ For more details, please refer to the checkpoints linked with the scores. On ove
101
  | RTE | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 63 |04:51 | 03:24 |
102
  | WNLI | Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |03:23 | 02:37 |
103
 
104
- | Task | Training time | Metric | | Result | | |
105
- | ----- | ---------------------- | ------------- | -------- | -------------------------------------------------------------- |----------------- | ------------------------------------------------------------------------- |
106
- | | | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | FNet (Flax) - Official |
107
  | MNLI | 09:52:33 | 06:40:55 |Accuracy or Match/Mismatch | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli) (Accuracy) | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli) (Accuracy) | 72/73 (Match/Mismatch) |
108
  | QQP | 09:25:01 | 06:21:16 |mean(Accuracy,F1) | [89.26](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp) | [86.5](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp) | 83 |
109
  | QNLI | 02:40:22 | 01:48:22 |Accuracy | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 80 |
@@ -115,6 +115,7 @@ For more details, please refer to the checkpoints linked with the scores. On ove
115
  | WNLI | 03:23 | 02:37 |Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |
116
 
117
 
 
118
  We can see that FNet-base achieves around 93% of BERT-base's performance while it requires *ca.* 30% less time to fine-tune on the downstream tasks.
119
 
120
  ### How to use
 
101
  | RTE | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 63 |04:51 | 03:24 |
102
  | WNLI | Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |03:23 | 02:37 |
103
 
104
+ | Task | Training time | | Metric | Result | | |
105
+ | ----- | ---------------------- | ------------- | -------- | -------------------------------------------------------------- |----------------- | ------------------------------------------------------------------------- |
106
+ | | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | FNet (Flax) - Official |
107
  | MNLI | 09:52:33 | 06:40:55 |Accuracy or Match/Mismatch | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli) (Accuracy) | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli) (Accuracy) | 72/73 (Match/Mismatch) |
108
  | QQP | 09:25:01 | 06:21:16 |mean(Accuracy,F1) | [89.26](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp) | [86.5](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp) | 83 |
109
  | QNLI | 02:40:22 | 01:48:22 |Accuracy | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 80 |
 
115
  | WNLI | 03:23 | 02:37 |Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |
116
 
117
 
118
+
119
  We can see that FNet-base achieves around 93% of BERT-base's performance while it requires *ca.* 30% less time to fine-tune on the downstream tasks.
120
 
121
  ### How to use