library_name: transformers
tags:
- trl
- sft
datasets:
- nenad1002/quantum_science_research_dataset
language:
- en
metrics:
- rouge
- bertscore
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
license: mit
Model Card for Model ID
Quantum Research Bot is a chat model fined tuned over the latest research data in quantum science. It contains data from the second half of 2024 making it more accurate than general-purpose models.
Model Details
Model Description
- Developed by: Nenad Banfic
- Language(s) (NLP): English
- License: MIT
- Finetuned from model [optional]: meta-llama/Meta-Llama-3.1-8B-Instruct
Uses
You can use the model to ask questions about the latest developments in quantum science. Below are examples of questions that general-purpose models may answer incorrectly or inadequately, but this model should provide accurate responses.
| Question | Expected answer |
|---|---|
| On top of what platform is TensorKrowch built on and where was it created? | TensorKrowch is built on top of the PyTorch framework and was created at the University of Madrid |
| What algorithms does the quantum FIPS 205 deal with? | The FIPS 205 deals with the stateless hash-based digital signature algorithm (SLH-DSA). |
| What is the variance which you can get with polynomial bond dimension in pure quantum states in one dimensional systems? | The variance that you can get with polynomial bond dimension in pure quantum states in one dimensional systems is as small as ∝ 1 / log N. |
| As if September 2024, how many qubits has the quantum Krylov algorithm been demonstrated on experimentally? | The quantum Krylov algorithm has been demonstrated on up to 56 qubits experimentally. |
| In the analysis of noise effects in controlled-swap gate circuits, what percentage of errors were eliminated with a dephasing error probability of 10% when using two noisy copies of a quantum state? | 67% of errors were eliminated when using two copies of a quantum state with a dephasing error probability of 10%. , |
Out-of-Scope Use
Although this model should be able to generalize well, the quantum science terminology and context is very complex, so it might struggle with corrent simplification, hence, should not be used in that context.
[More Information Needed]
Bias, Risks, and Limitations
[More Information Needed]
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
How to Get Started with the Model
Please refer to the instructions for the Meta Instruct models; the principle is the same.
[More Information Needed]
Training Details
Training Data
Initially trained on a bit less than 3k entries, it was later expanded t 5k high quality questions and answers to make the best of supervised fine tuning.
The dataset was generated by crawling the https://quantum-journal.org/ site, and passing data into the OpenAI gpt-4-turbo model with various prompts to ensure high quality data generation.
[More Information Needed]
Training Procedure
Many training procedures were tried alongside with multiple models.
Over the course of time multiple models and fine tuning approaches have been tried as the base model. The best performace was achieved with Lllama 3.1 70B Instruct and qLORA, but the model was very long to train, and finding the best hyperparameter would be too challenging.
The other two base models that were tries were the mistral 7B v0.1 base model, meta-llama/Llama-2-7b-chat-hf, and the base model of this model.
I've performed the grid search with several optimization techniques such as LORA, DORA, LORA+, REFT, and qLORA After exensive grid search, supervised fine tuning of Llama 3.1-8B with LORA+ resulted in the best training and evaluation cross entropy.
Preprocessing [optional]
[More Information Needed]
Training Hyperparameters
- Training regime: [More Information Needed]
- bfloat16 precision
- LORA rank: 8
- LORA alpha: 16
- LORA droput: 0.1
- Unfreezed nodes are attention, MLP, and embeddings
- Optimizer: AdamW
- LR: 1e-4
- LR scheduler: cosine
- NEFT enabled: true
- Batch size: 8
- Number of epochs: 3
Speeds, Sizes, Times [optional]
[More Information Needed]
Evaluation
Metrics
Since the fine-tuned model is designed to summarize newly learned data, ROUGE and BERTScore metrics were measured on a sample of 50 manually crafted questions. The reference answers were constructed during the creation of the training and evaluation sets. Given that GPT-4-turbo was already used in this context, I did not compare my model against it. Instead, I chose to compare it against the following models:
| Metric | quantum-research-bot-v1.0 | Meta-Llama-3.1-8B | gemini-1.5-pro |
|---|---|---|---|
| BERTScore F1 | 0.5821 | 0.3305 | 0.4982 |
| ROUGE-1 | 0.6045 | 0.3152 | 0.5029 |
| ROUGE-2 | 0.4098 | 0.1751 | 0.3104 |
| ROUGE-L | 0.5809 | 0.2902 | 0.4856 |
[More Information Needed]
Results
[More Information Needed]
Summary
Model Examination [optional]
[More Information Needed]
Environmental Impact
Carbon emissions are estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: RTX A6000
- Hours used: ~20h in total, although most trainings took a bit more than 30 minutes with rare exceptions
- Cloud Provider: Runpod
- Compute Region: West US
- Carbon Emitted: 1.5 kg CO2
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
For most workloads:
1 x RTX A6000 16 vCPU 62 GB RAM
However, when fine tuning meta-llama/Meta-Llama-3-70B-Instruct quantization was applied, and I've used 4xA100. Since this did not yield much improvements, and it was very costly, I decided to stick to model with fewer parameters.
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation [optional]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]