metadata
license: apache-2.0
datasets:
- wenbopan/Fusang-v1
- wenbopan/OpenOrca-zh-20k
language:
- zh
- en
Fi-9B
Fi-9B is an improved Yi-9B-200K with extensive instruction tuning on Fusang-V1.
Compare to Yi-9B-200K, Fi-9B gains greater capability at various downstream tasks and long-context modeling thanks to large-scale synthestic data in Fusang-V1.
Performance
Fact-based Evaluation
Fi is competitive amongst all models at ~9B size range:
| Metric | winogrande | hellaswag | truthfulqa | ai2_arc |
|---|---|---|---|---|
| Yi-9B-200K | 0.7167 | 0.5672 | 0.3380 | 0.6925 |
| Fi-9B-200K | 0.7111 | 0.5728 | 0.4086 | 0.7258 |
Long-context Modeling
Fi make even further progresses than Yi-9B-200K, which is already impressive in terms of handling long-range tasks:
Results on LongBench:
| Name | Average_zh | Average_en | Code Completion |
|---|---|---|---|
| Yi-9B-200K | 30.288 | 36.7071 | 72.2 |
| Fi-9B-200K | 41.092 | 40.9536 | 46.0 |
Detailed score decomposition on LongBench
| Name | Few-shot Learning_en | Synthetic Tasks_en | Single-Doc QA_en | Multi-Doc QA_en | Summarization_en | Few-shot Learning_zh | Synthetic Tasks_zh | Single-Doc QA_zh | Multi-Doc QA_zh | Summarization_zh |
|---|---|---|---|---|---|---|---|---|---|---|
| Yi-9B-200K | 60.6 | 22.8 | 30.9 | 38.9 | 25.8 | 46.5 | 28.0 | 49.6 | 17.7 | 9.7 |
| Fi-9B-200K | 63.8 | 40.2 | 36.2 | 38.0 | 26.3 | 30.0 | 75.1 | 55.6 | 30.7 | 14.1 |
Bilingual Ability
Coming soon...
How to use Fi
Coming soon...
Current Limitations
This version of Fi-9B may not be able to stop generation in some scenarios. I will fix that soon.
Compare to the original Yi-9B-200K, Fi-9B has degraded ability for code completion. This may due to lack of raw code data during instruction tuning.