Hugging Face's logo

Geralt-Targaryen
/
QwQ-1.5B-Persona

PyTorch
qwen2
Model card Files Community
Geralt-Targaryen commited on
Commit
776bb55
·
1 Parent(s): ff5a6e1

update readme

Browse files
Files changed (1) hide show
  1. README.md +8 -10
README.md CHANGED
@@ -15,20 +15,18 @@ Currently QwQ-1.5B-Persona is meant to serve as a draft model for losslessly acc
15
  We provide the speedup of QwQ 32B on MATH (200 samples spanning level 1-5), GPQA (diamond), and AIME (2023,2024). All experiments are run on 2 A100 with 40G memory each.
16
 
17
  **Qwen2.5-1.5B-Instruct** as draft model:
18
-
19
- | Draft Length Policy | MATH (l1) | MATH (l2) | MATH (l3) | MATH (l4) | MATH (l5) | GPQA | AIME | Avg |
20
  | ------------------- | --------- | --------- | --------- | --------- | --------- | ---- | ---- | ---- |
21
- | **Constant** | 1.18 | 1.16 | 1.22 | 1.29 | 1.30 | 1.28 | 1.13 | 1.22 |
22
- | **Heuristics** | 1.12 | 1.15 | 1.17 | 1.19 | 1.22 | 1.25 | 1.13 | 1.18 |
23
- | **SVIP** | 1.45 | 1.47 | 1.51 | 1.58 | 1.61 | 1.57 | 1.45 | 1.52 |
24
 
25
  **QwQ-1.5B-Persona** as draft model:
26
-
27
- | Draft Length Policy | MATH (l1) | MATH (l2) | MATH (l3) | MATH (l4) | MATH (l5) | GPQA | AIME | Avg |
28
  | ------------------- | --------- | --------- | --------- | --------- | --------- | ---- | ---- | ---- |
29
- | **Constant** | 1.45 | 1.50 | 1.52 | 1.56 | 1.56 | 1.58 | 1.25 | 1.49 |
30
- | **Heuristics** | 1.29 | 1.26 | 1.27 | 1.30 | 1.33 | 1.34 | 1.18 | 1.28 |
31
- | **SVIP** | 1.65 | 1.68 | 1.75 | 1.78 | 1.82 | 1.77 | 1.52 | 1.71 |
32
 
33
  The three draft length policies are:
34
 
 
15
  We provide the speedup of QwQ 32B on MATH (200 samples spanning level 1-5), GPQA (diamond), and AIME (2023,2024). All experiments are run on 2 A100 with 40G memory each.
16
 
17
  **Qwen2.5-1.5B-Instruct** as draft model:
18
+ | Draft Length Policy | MATH (l1) | MATH (l2) | MATH (l3) | MATH (l4) | MATH (l5) | GPQA | AIME | Avg |
 
19
  | ------------------- | --------- | --------- | --------- | --------- | --------- | ---- | ---- | ---- |
20
+ | **Constant** | 1.18 | 1.16 | 1.22 | 1.29 | 1.30 | 1.28 | 1.13 | 1.22 |
21
+ | **Heuristics** | 1.12 | 1.15 | 1.17 | 1.19 | 1.22 | 1.25 | 1.13 | 1.18 |
22
+ | **SVIP** | 1.45 | 1.47 | 1.51 | 1.58 | 1.61 | 1.57 | 1.45 | 1.52 |
23
 
24
  **QwQ-1.5B-Persona** as draft model:
25
+ | Draft Length Policy | MATH (l1) | MATH (l2) | MATH (l3) | MATH (l4) | MATH (l5) | GPQA | AIME | Avg |
 
26
  | ------------------- | --------- | --------- | --------- | --------- | --------- | ---- | ---- | ---- |
27
+ | **Constant** | 1.45 | 1.50 | 1.52 | 1.56 | 1.56 | 1.58 | 1.25 | 1.49 |
28
+ | **Heuristics** | 1.29 | 1.26 | 1.27 | 1.30 | 1.33 | 1.34 | 1.18 | 1.28 |
29
+ | **SVIP** | 1.65 | 1.68 | 1.75 | 1.78 | 1.82 | 1.77 | 1.52 | 1.71 |
30
 
31
  The three draft length policies are:
32