Updated open-sci-ref baselines. Re-training without dropout. Re-training on DCLM, FineWeb-Edu, Nemotron, HPLT-2, Pile. Further ref datasets included.
AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
View all activity
models 119
open-sci/sft_ot30k_1.7b-MixtureVitae-300BT-v1-decontaminated-16k-SFT-Tulu3-decontaminated_v0
Feature Extraction • 2B • Updated
open-sci/sft__ot30k_open-sci-ref-v0.02-1.7b-nemotron-hq-300B-16k-SFT-Tulu3-decontaminated
Feature Extraction • 2B • Updated • 44
open-sci/sft__ot30k_1.7b-MixtureVitae-300BT-v1-decontaminated-16k-SFT-Tulu3-decontaminated
Feature Extraction • 2B • Updated • 46
open-sci/sft__ot30k_1.7b-Comma0.1-300BT-longsft_16k-DPO-Tulu3-decontaminate
Feature Extraction • 2B • Updated • 45
open-sci/sft_ot30k_1.7b-MixtureVitae-300BT-v1-decontaminated-16k_base
Feature Extraction • 2B • Updated • 41
open-sci/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-long_sft_16k
Feature Extraction • 2B • Updated • 27
open-sci/sft__ot30k_SmolLM2-1.7B-Instruct-16k
Text Generation • 2B • Updated • 313
open-sci/sft__ot30k_SmolLM2-1.7B-16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 317
open-sci/sft__ot30k_Qwen3-1.7B-Base-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 310
open-sci/sft__ot30k_Qwen3-1.7B-Base-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 311