Update README.md
Browse files
README.md
CHANGED
@@ -108,7 +108,7 @@ Current SEA-LION models, including this commercially permissive release, have no
|
|
108 |
|
109 |
## Technical Specifications
|
110 |
### Fine-Tuning Details
|
111 |
-
Llama3.1 70B CPT SEA-LIONv3 Instruct was tuned using a combination of a full parameter fine-tune, on-policy alignment, and model merges of the best performing checkpoints. The training process for fine-tuning was approximately
|
112 |
|
113 |
## Data
|
114 |
Llama3.1 70B CPT SEA-LIONv3 Instruct was trained on a wide range of synthetic instructions, alongside publicly available instructions hand-curated by the team with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data sources.
|
|
|
108 |
|
109 |
## Technical Specifications
|
110 |
### Fine-Tuning Details
|
111 |
+
Llama3.1 70B CPT SEA-LIONv3 Instruct was tuned using a combination of a full parameter fine-tune, on-policy alignment, and model merges of the best performing checkpoints. The training process for fine-tuning was approximately 3200 GPU hours, on a single node of 8x H100-80GB GPUs.
|
112 |
|
113 |
## Data
|
114 |
Llama3.1 70B CPT SEA-LIONv3 Instruct was trained on a wide range of synthetic instructions, alongside publicly available instructions hand-curated by the team with the assistance of native speakers. In addition, special care was taken to ensure that the datasets used had commercially permissive licenses through verification with the original data sources.
|