Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,8 @@ It achieves the following results on the evaluation set:
|
|
24 |
|
25 |
## Spectrum Fine-tuning:
|
26 |
I have used the **Spectrum Fine-tuning** method as described in [Eric Hartford et. al 2024](https://arxiv.org/abs/2406.06623), which selectively targets some ***t%*** of the model layers with the highest **Signal-to-Noise Ratio (SNR)**. By focusing on the most information-dense layers, this approach maximizes fine-tuning efficiency while minimizing compute resources.
|
27 |
-
|
|
|
28 |
|
29 |
The 25% layer selection ensures minimal computational overhead for fine-tuning.
|
30 |
|
|
|
24 |
|
25 |
## Spectrum Fine-tuning:
|
26 |
I have used the **Spectrum Fine-tuning** method as described in [Eric Hartford et. al 2024](https://arxiv.org/abs/2406.06623), which selectively targets some ***t%*** of the model layers with the highest **Signal-to-Noise Ratio (SNR)**. By focusing on the most information-dense layers, this approach maximizes fine-tuning efficiency while minimizing compute resources.
|
27 |
+
|
28 |
+
**The key goal of Spectrum Fine-tuning is:** *minimize the memory footprint and accelerate LLM training without sacrificing performance.*
|
29 |
|
30 |
The 25% layer selection ensures minimal computational overhead for fine-tuning.
|
31 |
|