euclaise
/

ReMask-3B

Text Generation

Inference Endpoints

Model card Files Files and versions Community

euclaise commited on Apr 2, 2024

Commit

c7cf442

·

verified ·

1 Parent(s): 896b291

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -100,7 +100,7 @@ Here are some benchmark results, computed using the the LM Evaluation Harness wi
 | Model          | GSM8K (strict, 5-shot) | ARC-c (acc_norm, 25-shot) |
 |:--------------:|-----------------------:|--------------------------:|
 | SFT            | 24.34%                 | 42.92%                    |
-| Masked Thought | 24.18%                 | **43.60%**                |
 | **ReMask**     | **27.90%**             | 43.26%                    |
 As I expected, it improves GSM8K doesn't do much to ARC.

 | Model          | GSM8K (strict, 5-shot) | ARC-c (acc_norm, 25-shot) |
 |:--------------:|-----------------------:|--------------------------:|
 | SFT            | 24.34%                 | 42.92%                    |
+| Masked Thought | 24.18%                 | *43.60%*                |
 | **ReMask**     | **27.90%**             | 43.26%                    |
 As I expected, it improves GSM8K doesn't do much to ARC.