large-traversaal/Alif-Llama-3.1-8B-Instruct

Model Card for Alif Llama 3.1 8B Instruct

Alif Llama 3.1 8B Instruct is an open-weight model with highly advanced multilingual reasoning capabilities. It utilizes human refined multilingual synthetic data paired with reasoning to enhance cultural nuance and reasoning capabilities in english and urdu languages.

• Developed by: large-traversaal
• License: apache-2.0
• Finetuned from model : unsloth/Meta-Llama-3.1-8B
• Model: Alif Llama 3.1 8B Instruct
• Model Size: 8 billion parameters

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

How to Use Alif Llama

Install the transformers library and load Alif Llama 3.1 8B Instruct as follows:

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch
from transformers import BitsAndBytesConfig

model_id = "large-traversaal/Alif-Llama-3.1-8B-Instruct"  # Replace with your model

# 4-bit quantization configuration
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

# Load tokenizer and model in 4-bit
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quantization_config,
    device_map="auto"
)

# Create text generation pipeline
chatbot = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map="auto")

# Function to chat
def chat(message):
    response = chatbot(message, max_new_tokens=100, do_sample=True, temperature=0.3)
    return response[0]["generated_text"]

# Example chat
user_input = "شہر کراچی کی کیا اہمیت ہے؟"
bot_response = chat(user_input)

print(bot_response)

Model Details

Input: Models input text only.

Output: Models generate text only.

Model Architecture: Alif Llama 8B is an auto-regressive language model that uses an optimized transformer architecture. Post-training includes continued pretraining and supervised finetuning.

For more details about how the model was trained, check out our blogpost.

Evaluation

Model Card Contact

For errors or additional questions about details in this model card, contact.