base_model:
- Alibaba-NLP/gte-Qwen2-7B-instruct
language:
- en
- zh
tags:
- sentence-similarity
- transformers
INF-Retriever-Qwen2-7B-Preview
INF-Retriever-Qwen2-7B-Preview is a temporary version of the INF-Retriever-Qwen2-7B model, which has been fine-tuned for dense retrieval tasks in both Chinese and English. This preview version is being made available for testing and evaluation purposes.
Please note: The INF-Retriever-Qwen2-7B model, which includes further refinements and optimizations, will be released as the official version soon. We encourage you to stay tuned for the upcoming full release, which will offer enhanced performance and additional features.
Model Overview
INF-Retriever-Qwen2-7B is an LLM-based dense retrieval model developed by INF TECH. It is built upon the gte-Qwen2-7B-instruct model and specifically fine-tuned to excel in retrieval tasks, particularly for Chinese and English data.
As of November 13, 2024, INF-Retriever-Qwen2-7B-Preview ranks No.1 on the Automated Heterogeneous Information Retrieval Benchmark (AIR-Bench), showcasing its cutting-edge performance in heterogeneous information retrieval tasks.
Key Features
- Optimized for Chinese and English retrieval: The model has been specifically fine-tuned with retrieval-focused datasets in both languages, significantly improving its accuracy and efficiency for a variety of retrieval scenarios.
- Top-tier performance: INF-Retriever-Qwen2-7B has achieved outstanding results on the AIR-Bench leaderboard, making it a top choice for heterogeneous information retrieval tasks across various domains.
AIR-Bench Evaluation Results
INF-Retriever-Qwen2-7B-Preview has demonstrated superior retrieval capabilities across multiple domains and languages. The results from the Automated Heterogeneous Information Retrieval Benchmark (AIR-Bench) as of November 13 are as follows:
| Model Name | Average | wiki_en | wiki_zh | web_en | web_zh | healthcare_en | healthcare_zh | law_en | arxiv_en | news_en | news_zh | finance_en | finance_zh | msmarco_en |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BGE-M3 | 46.65 | 60.49 | 62.36 | 47.35 | 50.38 | 49.1 | 42.38 | 26.68 | 40.76 | 48.04 | 40.75 | 51.52 | 32.18 | 54.4 |
| BGE-Multilingual-Gemma2 | 46.83 | 63.71 | 67.3 | 50.38 | 53.24 | 47.24 | 42.13 | 22.58 | 23.28 | 50.91 | 44.02 | 49.3 | 31.6 | 63.14 |
| GTE-Qwen2-7B-instruct | 48.38 | 63.46 | 66.44 | 51.2 | 51.98 | 54.2 | 38.82 | 22.31 | 40.27 | 54.07 | 43.03 | 58.2 | 26.63 | 58.39 |
| INF-Retriever-Qwen2-7B-Preview | 52.22 | 64.96 | 67.87 | 52.84 | 55.54 | 58.82 | 37.71 | 34.89 | 52.35 | 53.6 | 47.93 | 58.5 | 33.92 | 59.96 |
Final Release Coming Soon
The official version of INF-Retriever-Qwen2-7B will provide even better performance, stability, and additional features. We are working hard to finalize the model, and we look forward to sharing the full release with the community in the near future.