Large-Scale Data Selection for Instruction Tuning
Datasets and models associated with the paper "Large-Scale Data Selection for Instruction Tuning" (https://arxiv.org/abs/2503.01807)
- Paper • 2503.01807 • Published • 9
hamishivi/tulu-2-multitask-rrmax-326k-sft
Updated • 6Note Above is our Llama 2 7b model trained on the multitask mixture linked below on Tulu 2 data.
hamishivi/rds-sels-multitask-rrmax-top326k
Viewer • Updated • 326k • 11
hamishivi/llama-3.1-tulu-3-multitask-rrmax-939k-sft
Updated • 9Note Above is our Llama 3 8b model trained on the multitask mixture linked below on Tulu 3 data.
hamishivi/rds-sels-tulu-3-multitask-rrmax-939k
Viewer • Updated • 939k • 43Note Below is our unfiltered datasets, multi-million size instruction tuning datasets made up of all the data considered for Tulu 2 and 3 respectively.
hamishivi/tulu-2-unfiltered
Viewer • Updated • 3.54M • 12
hamishivi/tulu-3-unfiltered
Viewer • Updated • 4.88M • 16Note Below are other multitask trained models and the data they were trained on. Tulu 2 models are based on Llama 2 7b and Tulu 3 models on Llama 3 8b.
hamishivi/llama-3.1-tulu-3-arena-hard-939k-sft
Updated • 10hamishivi/rds-sels-tulu-3-arena-hard-939k
Viewer • Updated • 939k • 42hamishivi/tulu-2-arena-hard-326k-sft
Updated • 10hamishivi/rds-sels-arena-hard-top326k
Viewer • Updated • 326k • 10hamishivi/tulu-2-wildchat-326k-sft
Updated • 3
hamishivi/rds-sels-wildchat-top326k
Viewer • Updated • 326k • 10Note Below is the data selected by RDS+ with Llama 2 7b from the Tulu 2 unfiltered dataset, selecting for the evaluation in the dataset name.
hamishivi/rds-sels-alpacafarm-top326k
Viewer • Updated • 326k • 12hamishivi/rds-sels-gsm8k-shots-top326k
Viewer • Updated • 326k • 47hamishivi/rds-sels-codex-top326k
Viewer • Updated • 326k • 10hamishivi/rds-sels-bbh-shots-top326k
Viewer • Updated • 326k • 10hamishivi/rds-sels-mmlu-shots-top326k
Viewer • Updated • 326k • 10hamishivi/rds-sels-squad-top326k
Viewer • Updated • 326k • 10hamishivi/rds-sels-tydiqa-shots-top326k
Viewer • Updated • 326k • 11hamishivi/lsds_data
Preview • Updated • 63