Delta-Vector commited on
Commit
771850d
·
verified ·
1 Parent(s): 5c30790

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -49
README.md CHANGED
@@ -193,52 +193,3 @@ The training was done for 1 epochs We used 4 x [RTX 3090s](https://www.nvidia.c
193
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/bL0o_4bvbkmzAvK3W8gu2.png)
194
 
195
 
196
- optimizer: paged_adamw_8bit
197
- lr_scheduler: constant_with_warmup
198
- learning_rate: 1e-6
199
- max_grad_norm: 0.01
200
-
201
- train_on_inputs: false
202
- group_by_length: false
203
- bf16: auto
204
- fp16:
205
- tf32: true
206
-
207
- gradient_checkpointing: unsloth
208
- early_stopping_patience:
209
- resume_from_checkpoint:
210
- local_rank:
211
- logging_steps: 1
212
- xformers_attention:
213
- flash_attention: true
214
-
215
- warmup_steps: 25
216
- evals_per_epoch: 4
217
- eval_table_size:
218
- eval_max_new_tokens: 128
219
- saves_per_epoch: 1
220
- debug:
221
- deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16_cpuoffload_params.json
222
- weight_decay: 0.0
223
- fsdp:
224
- fsdp_config:
225
-
226
-
227
- ```
228
-
229
- </details><br>
230
-
231
- ## Credits
232
-
233
- Thank you to [Lucy Knada](https://huggingface.co/lucyknada), [Intervitens](https://huggingface.co/intervitens),[Cgato](https://huggingface.co/cgato), [Kubernetes Bad](https://huggingface.co/kubernetes-bad) and the rest of [Anthracite](https://huggingface.co/anthracite-org)
234
-
235
-
236
- ## Training
237
- The training was done for 1 epochs We used 4 x [RTX 3090s](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/) GPUs graciously provided by [Intervitens](https://huggingface.co/intervitens) for the fine-tuning of the model.
238
-
239
- [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
240
-
241
- ## Safety
242
-
243
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/bL0o_4bvbkmzAvK3W8gu2.png)
244
-
 
193
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/66c26b6fb01b19d8c3c2467b/bL0o_4bvbkmzAvK3W8gu2.png)
194
 
195