Releasing H2O-Danube2-1.8b - a new and improved language model being the current top model on the Open LLM Leaderboard.
The model is trained on additional 2T tokens (total of 3T tokens) with various data mix stages as a result of extensive experimentation. Focus is on natural language tasks.
Happy to share H2O-Danube-1.8b, a small 1.8b model based trained on only 1T natural language tokens showing competitive metrics across benchmarks in the <2B model space.