Update README.md

README.md

@@ -72,9 +72,9 @@ The dataset was generated by crawling the https://quantum-journal.org/ site, and
 
 ### Training Procedure
 
-Various training procedures were explored alongside multiple models, however, all of them were parameter efficient.
+Various training procedures were explored alongside multiple models, however, all of them were parameter efficient. The general idea was to freeze most of the original model's parameters and only allow a small subset of parameters to be trainable.
 
-Over time, several models and fine-tuning approaches were tested as the base model. The best accuracy was achieved with [Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) and qLoRA, but the training duration was extensive, and optimizing hyperparameters proved to be highly challenging.
+Over time, several base models and fine-tuning approaches were tested. The best accuracy was achieved with [Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) and qLoRA, but the training duration was extensive, and optimizing hyperparameters proved to be highly challenging.
 
 Other base models were also tested: [Mistral 7B v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), [Meta-Llama/Llama-2-7b-chat-hf](Meta-Llama/Llama-2-7b-chat-hf), and the base model of this experiment.
 
@@ -137,6 +137,7 @@ The table below shows the best evaluation cross-entropy (across all params) for
 | **qLORA (for 8b model)**| 0.5471         |  |
 | **(LO)ReFT**| 0.4824         |  |
 
+The loss mask was applied during training, but it wasn't particularly useful since the model doesn't involve function calling or external data fetching.
 
 #### Metrics