Commit
·
e5d0e83
1
Parent(s):
9a4739c
Update README.md
Browse files
README.md
CHANGED
@@ -101,6 +101,20 @@ For more details, please refer to the checkpoints linked with the scores. On ove
|
|
101 |
| RTE | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 63 |04:51 | 03:24 |
|
102 |
| WNLI | Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |03:23 | 02:37 |
|
103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
104 |
We can see that FNet-base achieves around 93% of BERT-base's performance while it requires *ca.* 30% less time to fine-tune on the downstream tasks.
|
105 |
|
106 |
### How to use
|
|
|
101 |
| RTE | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 63 |04:51 | 03:24 |
|
102 |
| WNLI | Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |03:23 | 02:37 |
|
103 |
|
104 |
+
| Task | Training time | Metric | | Result | | |
|
105 |
+
| ----- | ---------------------- | ------------- | -------- | -------------------------------------------------------------- |----------------- | ------------------------------------------------------------------------- |
|
106 |
+
| | | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | Bert (PyTorch) - Reproduced | FNet (PyTorch) - Reproduced | FNet (Flax) - Official |
|
107 |
+
| MNLI | 09:52:33 | 06:40:55 |Accuracy or Match/Mismatch | [84.10](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mnli) (Accuracy) | [76.75](https://huggingface.co/gchhablani/fnet-base-finetuned-mnli) (Accuracy) | 72/73 (Match/Mismatch) |
|
108 |
+
| QQP | 09:25:01 | 06:21:16 |mean(Accuracy,F1) | [89.26](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qqp) | [86.5](https://huggingface.co/gchhablani/fnet-base-finetuned-qqp) | 83 |
|
109 |
+
| QNLI | 02:40:22 | 01:48:22 |Accuracy | [90.99](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [84.39](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 80 |
|
110 |
+
| SST-2 | 01:42:17 | 01:09:27 | Accuracy | [92.32](https://huggingface.co/gchhablani/bert-base-cased-finetuned-sst2) | [89.45](https://huggingface.co/gchhablani/fnet-base-finetuned-sst2) | 95 |
|
111 |
+
| CoLA | 14:20 | 09:47 | Matthews corr or Accuracy | [59.57](https://huggingface.co/gchhablani/bert-base-cased-finetuned-cola) (Matthews corr) | [35.94](https://huggingface.co/gchhablani/fnet-base-finetuned-cola) (Matthews Corr) | 69 (Accuracy) |
|
112 |
+
| STS-B | 10:24 | 07:09 |Spearman corr. | [88.98](https://huggingface.co/gchhablani/bert-base-cased-finetuned-stsb) | [82.19](https://huggingface.co/gchhablani/fnet-base-finetuned-stsb) | 79 |
|
113 |
+
| MRPC | 11:12 | 07:48 | mean(F1/Accuracy) | [88.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-mrpc) | [81.15](https://huggingface.co/gchhablani/fnet-base-finetuned-mrpc) | 76 |
|
114 |
+
| RTE | 04:51 | 03:24 | Accuracy | [67.15](https://huggingface.co/gchhablani/bert-base-cased-finetuned-qnli) | [62.82](https://huggingface.co/gchhablani/fnet-base-finetuned-qnli) | 63 |
|
115 |
+
| WNLI | 03:23 | 02:37 |Accuracy | [46.48](https://huggingface.co/gchhablani/bert-base-cased-finetuned-wnli) | [54.93](https://huggingface.co/gchhablani/fnet-base-finetuned-wnli) | - |
|
116 |
+
|
117 |
+
|
118 |
We can see that FNet-base achieves around 93% of BERT-base's performance while it requires *ca.* 30% less time to fine-tune on the downstream tasks.
|
119 |
|
120 |
### How to use
|