Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,84 @@
|
|
| 1 |
---
|
| 2 |
license: openrail
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: openrail
|
| 3 |
+
|
| 4 |
+
datasets:
|
| 5 |
+
- LinkSoul/LLaSA-Audio-Instructions
|
| 6 |
+
language:
|
| 7 |
+
- zh
|
| 8 |
+
- en
|
| 9 |
---
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
# LLaSA: Large Language and Speech Assistant
|
| 13 |
+
|
| 14 |
+
开源,可商用的**中英文双语语音-语言助手 LLaSA 以及中英文语音 SFT 数据集 LLaSA-Audio-Instructions**,第一个支持中英文语音-文本多模态对话的开源可商用对话模型。
|
| 15 |
+
|
| 16 |
+
<!--
|
| 17 |
+
<div align="center">
|
| 18 |
+
<img src="https://huggingface.co/LinkSoul/LLaSA-Cllama2/blob/main/meta/preview.jpg" width="40%">
|
| 19 |
+
</div>
|
| 20 |
+
|
| 21 |
+

|
| 22 |
+
-->
|
| 23 |
+
|
| 24 |
+
## 基础演示
|
| 25 |
+
|
| 26 |
+

|
| 27 |
+
|
| 28 |
+
## 在线试玩
|
| 29 |
+
|
| 30 |
+
> Talk is cheap, Show you the Demo.
|
| 31 |
+
- [Demo 地址 / HuggingFace Spaces](https://huggingface.co/spaces/LinkSoul/LLaSA)
|
| 32 |
+
|
| 33 |
+
## 资源下载
|
| 34 |
+
|
| 35 |
+
- 模型:
|
| 36 |
+
- [LLaSA-Chinese-Llama-2-7B](https://huggingface.co/LinkSoul/LLaSA-Cllama2)
|
| 37 |
+
- [LLaSA-Baichuan-7B](https://huggingface.co/LinkSoul/LLaSA-Baichuan)
|
| 38 |
+
|
| 39 |
+
- 数据集:[LLaSA-Audio-Instructions](https://huggingface.co/datasets/LinkSoul/LLaSA-Audio-Instructions)
|
| 40 |
+
|
| 41 |
+
## 环境安装
|
| 42 |
+
```shell
|
| 43 |
+
# clone the repository
|
| 44 |
+
git clone https://github.com/LinkSoul-AI/LLaSA
|
| 45 |
+
cd LLaSA
|
| 46 |
+
|
| 47 |
+
# install package
|
| 48 |
+
conda create -n llasa python=3.10 -y
|
| 49 |
+
conda activate llasa
|
| 50 |
+
pip install --upgrade pip
|
| 51 |
+
pip install -e .
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
## 快速测试
|
| 55 |
+
|
| 56 |
+
```shell
|
| 57 |
+
export LLASA_DEVICE="cuda:0"
|
| 58 |
+
python infer.py \
|
| 59 |
+
--input_audio_file PATH/TO/YOUR/AUDIO \
|
| 60 |
+
--llasa_model PATH/TO/LLaSA/MODEL \
|
| 61 |
+
--llasa_audio_tower PATH/TO/WHISPER/MODEL \
|
| 62 |
+
--llm_type "Chinese_llama2" or "baichuan" \
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
## TODO
|
| 66 |
+
- 如何训练
|
| 67 |
+
- int4 量化
|
| 68 |
+
- docker 部署
|
| 69 |
+
|
| 70 |
+
## 相关项目
|
| 71 |
+
- [Chinese-Llama-2-7B](https://huggingface.co/LinkSoul/Chinese-Llama-2-7b)
|
| 72 |
+
- [Whisper](https://ai.meta.com/llama/)
|
| 73 |
+
- [baichuan-inc/Baichuan-7B](https://huggingface.co/baichuan-inc/Baichuan-7B)
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
## 项目协议
|
| 77 |
+
|
| 78 |
+
[Apache-2.0 license](https://github.com/LinkSoul-AI/LLaSA/blob/main/LICENSE)
|
| 79 |
+
|
| 80 |
+
## 微信交流群
|
| 81 |
+
<!--
|
| 82 |
+
<img src="meta/QRcode.jpg" alt="微信交流群" width="300"/>
|
| 83 |
+
-->
|
| 84 |
+
欢迎加入[微信群](meta/QRcode.jpg)
|