pankajroark
/

llama-fp16-engine

Model card Files Files and versions

llama-fp16-engine / 7b-sq-int8kv-tp1

7.01 GB

1 contributor

History: 2 commits

pankajroark's picture

inflight batching engine for 7b-sq-int8kv-tp1

2319002 almost 2 years ago

config.json

1.31 kB

inflight batching engine for 7b-sq-int8kv-tp1 almost 2 years ago
llama_float16_tp1_rank0.engine

7.01 GB
xet

inflight batching engine for 7b-sq-int8kv-tp1 almost 2 years ago
model.cache

96.1 kB

inflight batching engine for 7b-sq-int8kv-tp1 almost 2 years ago