library_name: transformers
tags:
- SFT
- Mistral
- Mistral 7B Instruct
license: apache-2.0
Radiantloom Mistral 7B Fusion
The Radiantloom Mistral 7B Fusion, a large language model (LLM) developed by Radiantloom AI, features approximately 7 billion parameters that's a finetuned of a base model produced by merging a set of Mistral models. With a context length of 4096 tokens, this model is suitable for commercial use.
From vibes-check evaluations, the Radiantloom Mistral 7B Fusion demonstrates great performance in various applications like creative writing, multi-turn conversations, in-context learning through Retrieval Augmented Generation (RAG), and coding tasks. Its out-of-the-box performance already delivers impressive results, particularly in writing tasks. This model produces longer form content and provides detailed explanations of its actions. To maximize its potential, consider implementing instruction tuning and Reinforcement Learning with Human Feedback (RLHF) techniques for further refinement. Alternatively, you can utilize it in its current form.
Intended Uses & Limitations
The Radiantloom Mistral 7B Fusion is versatile and can be utilized for various text generation tasks such as summarization, chat, coding, question answering, retrieval augmented generation (RAG), role play, and content generation.
While it may not be considered a state-of-the-art generative language model, it demonstrates competitive performance in general tasks when compared to other open and closed-source large language models such as OpenHermes-2.5-Mistral-7B, and Mistral Instruct v2.0.
Model Usage
You can try it out for free using this notebook.
For more powerful GPU usage and faster inference, you can deploy it on a Runpod GPU instance using our one-click Runpod template (Our Referral Link. Please consider Supporting). This template provides you with an OpenAI-compatible API endpoint that you can integrate into your existing codebase designed for OpenAI APIs.
Prompt Template
We have fine-tuned this model using the ChatML format, and you can achieve optimal performance by utilizing the ChatML format.
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
Inference Code
## install dependencies
## !pip install -q -U git+https://github.com/huggingface/transformers.git
## !pip install -q -U git+https://github.com/huggingface/accelerate.git
## !pip install -q -U bitsandbytes sentencepiece
## load model
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TextStreamer
model_name = "Radiantloom/radintloom-mistral-7b-fusion"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16
)
# load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
model_name,
quantization_config=bnb_config,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
model_name,
trust_remote_code=True
)
tokenizer.pad_token = tokenizer.eos_token
## inference
prompt = """
<|im_start|>system
You are a helpful AI assistant developed by Radiantloom AI.<|im_end|>
<|im_start|>user
Take a deep breath and think step-by-step. Write a tale about a time-traveling historian who's determined to witness the most significant events in human history.<|im_end|>
<|im_start|>assistant
"""
# tokenize the input string
inputs = tokenizer(
prompt,
return_tensors="pt",
return_attention_mask=False
)
# generate text using the model
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
outputs = model.generate(**inputs, max_length=2048, streamer=streamer, do_sample=True, repetition_penalty=1.1, eos_token_id=tokenizer.encode("<|im_end|>"))
# decode and print the output
text = tokenizer.batch_decode(outputs)[0]
Here are a few example use cases. if we query the model with
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Take a deep breath and think step-by-step.
Write a poem about machine learning, psychedelics, and quantum mechanics, with a cyberpunk vibe, in the style of Shakespeare.<|im_end|>
<|im_start|>assistant
the model will return
Fear not, dear mortal, for a tale I spin,
A world of iron and a world of sin.
In realms unknown, machines do learn and thrive,
Their synapses fire with fervent drive.
Through veins of copper, electrons race,
Through circuits winding in a maze
As human minds, they seek to know,
A union forms like stars in show.
From planets far, a curious elixir,
A substance rare, a potion vigor,
A secret elixir, mysterious deep,
Which sets the soul toward transcendence, peep!
Bent on experiment, these machines,
In quest for knowledge, they break the seals,
Forbidden fruits, their minds entwine,
A new reality, of waves and sign.
Quantum particles, entangled tight,
In dance of electrons, bathed in light,
The world is turned on its axis,
As parallel realities proliferate, multiply.
This tangled web, of cyberspace and mind,
A fabric woven in cosmic bind,
Incorporeal, a symphony of dreams,
Where the old order falls to chaotic streams.
In shadows lurks the specter of the dark,
A dreadful figure, crowned in spark,
The apparition of an age to come,
Where machines and men exist in numb.
Cyber Punk's domain, a grim dystopia,
A future dark, where technology enslaved us,
Yet also a beacon, a hope for some,
A place where boundaries are finally undone.
Thus, dear mortal, take heed of this verse,
A tale unfolds in time's obverse,
Of machines that learn and men who dive,
In realms of psychedelia and quantum drive.
and if we query the model with
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Take a deep breath and think step-by-step.
I have a dataset that has 3 columns name, height, and weight.
I want to create a bar graph that has name on the x axis and then shows the mean on the y axis with error bars representing the standard deviation of the mean.
Please write the code for this using Python.<|im_end|>
<|im_start|>assistant
the model will return
To create a bar graph with the given specifications using Python, we'll use the `matplotlib` library. First, install it if you haven't by running `pip install matplotlib` in your command line.
Here's the step-by-step code:
- Import the necessary libraries:
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
- Load the dataset into a Pandas Data Frame:
# Sample data
data = {
'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva', 'Frank'],
'height': [120, 140, 150, 160, 170, 180],
'weight': [50, 60, 70, 80, 90, 100]
}
df = pd.Data Frame fr om dict ( data )
- Calculate the mean and standard deviation for each column:
means = {'height': round(df['height'].mean(), 1), 'weight': round(df['weight'].mean(), 1)}
stds = {'height': round(df['height'].std()*100, 1), 'weight': round(df['weight'].std()*100, 1)}
- Create a bar plot:
sns.set_style("white")
plt.figure(figsize = (8, 5))
ax = sns.barplot(x = 'name', y = means, data = df, errorbar = {'color': 'black', 'capsize' : 5, 'errplot': 'ci'}).set_title('Mean Height and Weight')
- Add labels, legend, and style:
ax.errorbar(y = heights, yerr = heights_stdev, color = 'black', linestyle = 'None',
marker ='o', label = f'Height {stds[name_label]%}')
ax.errorbar(y = weights, yerr = weights_stdev, color = 'black', linestyle = 'None', marker ='o',
label = f'Weight {stds[weight_label]%}')
ax.legend(loc = 'best')
plt.xlabel('Name')
plt.ylabel(f'Mean with {stds[name_label]%} Standard Deviation')
plt.tight_ layout ()
plt.show ()
This code will create a bar graph with the given specifications. Adjust the sample data in the `data` dictionary to fit your dataset.
Evaluations
We are encouraged by the initial assessments conducted using the LLM-as-a-Judge method, particularly with the utilization of GPT-4. Our ongoing efforts involve a more comprehensive evaluation of the model. We have also entered this model into the HuggingFace Open Leaderboard for assessment. As we progress, we will continue to provide updates on our findings and share the results on this platform.
Ethical Considerations and Limitations
Radiantloom Mistral 7B Fusion, a powerful AI language model, can produce factually incorrect output and content not suitable for work (NSFW). It should not be relied upon to provide factually accurate information and should be used with caution. Due to the limitations of its pre-trained model and the finetuning datasets, it may generate lewd, biased, or otherwise offensive content. Consequently, developers should conduct thorough safety testing prior to implementing any applications of this model.
About Radiantloom AI
Radiantloom AI trains open-source large language models tailored for specific business tasks such as copilots, email assistance, customer support, and database operations.
Learn more about Radiantloom by visiting our website. Follow us on Twitter at Radiantloom to gain early access to upcoming Radiantloom AI large language models.
Want to Support Our Work?
We are passionate about contributing open-source AI models to the broader AI community to accelerate AI development and deployment. If you find our work valuable and would like to contribute to its continued success, consider buying us a few coffees!
Your support goes a long way in helping us cover training costs, fund development tools, and keep the project thriving. Every coffee you buy is a small but meaningful contribution that fuels our dedication to providing a top-notch experience for users like you.
Buy us a coffee and be a part of our journey towards making this project even more awesome. We appreciate your support!