File size: 1,232 Bytes
efa14b2 16158bb efa14b2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
license: bsd
---
Converted INT8/INT4 files for [fastllm](https://github.com/ztxz16/fastllm) with [baichuan-13b-chat](https://huggingface.co/baichuan-inc/Baichuan-13B-Chat)
Directly download from Baidu Netdisk:
Link：https://pan.baidu.com/s/1zADu6rd749zkkNAfl-aqtg
Code：jqkl
Updated time: 2023/07/27
```
baichuan-13b-chat-int4.flm:
+-----------------------------------------------------------------------------+
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| N/A 53C P0 56W / 250W | 7083MiB / 23040MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
baichuan-13b-chat-int8.flm:
+-----------------------------------------------------------------------------+
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| N/A 51C P0 162W / 250W | 13151MiB / 23040MiB | 95% Default |
+-------------------------------+----------------------+----------------------+
```
```python
from fastllm_pytools import llm
model = llm.model("baichuan-13b-chat-int4.flm")
for response in model.stream_response("介绍一下南京"):
print(response, flush = True, end = "")
```
|