Hugging Face's logo

mgoin
/
Nemotron-4-340B-Base-hf-FP8

Text Generation
Transformers
Safetensors
nemotron
fp8
vllm
Model card Files Community
mgoin commited on
Commit
387bc8f
·
verified ·
1 Parent(s): 121bb1d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -8
README.md CHANGED
@@ -5,14 +5,6 @@ base_model: mgoin/Nemotron-4-340B-Base-hf
5
  ---
6
 
7
  ```
8
- lm_eval --model vllm --model_args pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend="ray",max_model_len=4096 --tasks gsm8k --num_fewshot 5 --batch_size auto
9
- vllm (pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend=ray,max_model_len=4096), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: auto
10
- |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
11
- |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
12
- |gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.2949|± |0.0126|
13
- | | |strict-match | 5|exact_match|↑ |0.1600|± |0.0101|
14
-
15
-
16
  lm_eval --model vllm --model_args pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend="ray",max_model_len=4096,gpu_memory_utilization=0.6 --tasks truthfulqa_mc2 --num_fewshot 0 --batch_size 16
17
  vllm (pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend=ray,max_model_len=4096,gpu_memory_utilization=0.6), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16
18
  | Tasks |Version|Filter|n-shot|Metric| |Value | |Stderr|
 
5
  ---
6
 
7
  ```
 
 
 
 
 
 
 
 
8
  lm_eval --model vllm --model_args pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend="ray",max_model_len=4096,gpu_memory_utilization=0.6 --tasks truthfulqa_mc2 --num_fewshot 0 --batch_size 16
9
  vllm (pretrained=/home/mgoin/code/Nemotron-4-340B-Base-hf-FP8,tensor_parallel_size=8,distributed_executor_backend=ray,max_model_len=4096,gpu_memory_utilization=0.6), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16
10
  | Tasks |Version|Filter|n-shot|Metric| |Value | |Stderr|