Hugging Face's logo

facebook
/
MobileLLM-350M-layer-share

Text Generation
Transformers
PyTorch
mobilellm
custom_code
Model card Files Community
2
zechunliu commited on
Commit
1170189
·
verified ·
1 Parent(s): fb8254d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -85,7 +85,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
85
 
86
  ## MobileLLM-125M
87
 
88
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
89
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
90
  | OPT-125M | 41.3 | 25.2 | 57.5 | 62.0 | 41.9 | 31.1 | 31.2 | 50.8 | 42.6 |
91
  | GPT-neo-125M | 40.7 | 24.8 | 61.3 | 62.5 | 41.9 | 29.7 | 31.6 | 50.7 | 42.9 |
@@ -95,7 +95,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
95
 
96
  ## MobileLLM-350M
97
 
98
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
99
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
100
  | OPT-350M | 41.9 | 25.7 | 54.0 | 64.8 | 42.6 | 36.2 | 33.3 | 52.4 | 43.9 |
101
  | Pythia-410M | 47.1 | 30.3 | 55.3 | 67.2 | 43.1 | 40.1 | 36.2 | 53.4 | 46.6 |
@@ -104,7 +104,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
104
 
105
  ## MobileLLM-600M
106
 
107
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
108
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
109
  | Qwen1.5-500M | 54.7 | 32.1 | 46.9 | 68.9 | 46.0 | 48.8 | 37.7 | 55.0 | 48.8 |
110
  | BLOOM-560M | 43.7 | 27.5 | 53.7 | 65.1 | 42.5 | 36.5 | 32.6 | 52.2 | 44.2 |
@@ -113,7 +113,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
113
 
114
  ## MobileLLM-1B
115
 
116
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
117
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
118
  | Pythia-1B | 49.9 | 30.4 | 58.7 | 69.2 | 43.3 | 47.4 | 38.6 | 52.2 | 48.7 |
119
  | MobiLlama-1B | 59.7 | 38.4 | 59.2 | 74.5 | 44.9 | 62.0 | 43.7 | 59.0 | 55.2 |
@@ -124,7 +124,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
124
 
125
  ## MobileLLM-1.5B
126
 
127
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
128
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
129
  | GPT-neo-1.3B | 51.3 | 33.0 | 61.8 | 70.9 | 43.7 | 48.6 | 41.2 | 54.5 | 50.6 |
130
  | OPT-1.3B | 54.4 | 31.7 | 58.4 | 71.5 | 44.7 | 53.7 | 44.6 | 59.1 | 52.3 |
 
85
 
86
  ## MobileLLM-125M
87
 
88
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
89
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
90
  | OPT-125M | 41.3 | 25.2 | 57.5 | 62.0 | 41.9 | 31.1 | 31.2 | 50.8 | 42.6 |
91
  | GPT-neo-125M | 40.7 | 24.8 | 61.3 | 62.5 | 41.9 | 29.7 | 31.6 | 50.7 | 42.9 |
 
95
 
96
  ## MobileLLM-350M
97
 
98
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
99
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
100
  | OPT-350M | 41.9 | 25.7 | 54.0 | 64.8 | 42.6 | 36.2 | 33.3 | 52.4 | 43.9 |
101
  | Pythia-410M | 47.1 | 30.3 | 55.3 | 67.2 | 43.1 | 40.1 | 36.2 | 53.4 | 46.6 |
 
104
 
105
  ## MobileLLM-600M
106
 
107
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
108
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
109
  | Qwen1.5-500M | 54.7 | 32.1 | 46.9 | 68.9 | 46.0 | 48.8 | 37.7 | 55.0 | 48.8 |
110
  | BLOOM-560M | 43.7 | 27.5 | 53.7 | 65.1 | 42.5 | 36.5 | 32.6 | 52.2 | 44.2 |
 
113
 
114
  ## MobileLLM-1B
115
 
116
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
117
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
118
  | Pythia-1B | 49.9 | 30.4 | 58.7 | 69.2 | 43.3 | 47.4 | 38.6 | 52.2 | 48.7 |
119
  | MobiLlama-1B | 59.7 | 38.4 | 59.2 | 74.5 | 44.9 | 62.0 | 43.7 | 59.0 | 55.2 |
 
124
 
125
  ## MobileLLM-1.5B
126
 
127
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
128
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
129
  | GPT-neo-1.3B | 51.3 | 33.0 | 61.8 | 70.9 | 43.7 | 48.6 | 41.2 | 54.5 | 50.6 |
130
  | OPT-1.3B | 54.4 | 31.7 | 58.4 | 71.5 | 44.7 | 53.7 | 44.6 | 59.1 | 52.3 |