metadata

language:
  - en
  - ko
tags:
  - qwen
  - lora
  - rag
  - instruction-tuning
  - email
  - qwen-2.5
  - peft
  - question-answering
library_name: peft
pipeline_tag: text-generation
license: mit

Qwen-RAG-LoRA

This repository contains LoRA weights for Qwen/Qwen2.5-7B-Instruct, fine-tuned for email-based question answering tasks. The model has been trained to handle both English and Korean queries about email content.

Model Description

Base Model: Qwen/Qwen2.5-7B-Instruct
Training Type: LoRA (Low-Rank Adaptation)
Checkpoint: checkpoint-600
Languages: English and Korean
Task: Email-based Question Answering
Domain: Email Content

Training Details

LoRA Configuration

lora_config = LoraConfig(
    r=8,
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM"
)

Usage with vLLM

from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest

# Initialize LLM with LoRA support
llm = LLM(
    model="Qwen/Qwen2.5-7B-Instruct",
    tensor_parallel_size=2,
    enable_lora=True
)

sampling_params = SamplingParams(temperature=0.0, max_tokens=50)

# Create LoRA request
lora_request = LoRARequest(
    "rag_adapter",
    1,
    "doubleyyh/qwen-rag-lora"  # HuggingFace repo name
)

# Example prompt
prompt = """Using the context provided below, answer the question concisely. Respond in Korean if the question is in Korean, and in English if the question is in English.

Context: subject: Meeting Schedule Update
from: [['John Smith', '[email protected]']]
to: [['Team', '[email protected]']]
text_body: The project review meeting is rescheduled to 3 PM tomorrow.

Question: When is the meeting rescheduled to?

Answer: """

# Generate with LoRA
outputs = llm.generate([prompt], sampling_params, lora_request=lora_request)
print(outputs[0].outputs[0].text)

Example Input/Output

# English Query
Q: When is the project review scheduled?
A: The project review meeting is rescheduled to 3 PM tomorrow.

# Korean Query
Q: 프로젝트 미팅이 언제로 변경되었나요?
A: 내일 오후 3시로 변경되었습니다.

Limitations

The model is specifically trained for email-related queries
Performance might vary between English and Korean
Optimal results when used with email content in standard format
Limited to the capabilities of the base Qwen model

Citation

@misc{qwen-rag-lora,
  author = {doubleyyh},
  title = {Qwen-RAG-LoRA: Fine-tuned LoRA Weights for Email QA},
  year = {2024},
  publisher = {Hugging Face}
}

License

This model follows the same license as Qwen2.5-7B-Instruct.