vietphuon
/

Llama-3.2-1B-Instruct-bnb-4bit-alpaca-then-quizgen-241016-1

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

vietphuon commited on 15 days ago

Commit

ececa2d

•

1 Parent(s): cca0f00

Update README.md

Files changed (1) hide show

README.md +10 -10

README.md CHANGED Viewed

@@ -13,16 +13,16 @@ tags:
 FINAL BENCHMARKING
 ------------------------------
-Time to First Token (TTFT): 0.001s
-Time Per Output Token (TPOT): 41.83ms/token
-Throughput (token/s): 24.35token/s
-Average Token Latency (ms/token): 41.92ms/token
-Total Generation Time: 18.427s
-Input Tokenization Time: 0.009s
-Input Tokens: 1909
-Output Tokens: 443
-Total Tokens: 2352
-Memory Usage (GPU): 3.38GB
 # Uploaded  model

 FINAL BENCHMARKING
 ------------------------------
+- **Time to First Token (TTFT)**: 0.001s
+- **Time Per Output Token (TPOT):** 41.83ms/token
+- **Throughput (token/s):** 24.35token/s
+- **Average Token Latency (ms/token):** 41.92ms/token
+- **Total Generation Time:** 18.427s
+- **Input Tokenization Time:** 0.009s
+- **Input Tokens:** 1909
+- **Output Tokens:** 443
+- **Total Tokens:** 2352
+- **Memory Usage (GPU):** 3.38GB
 # Uploaded  model