Aarushhh
/

SEWY2-640M-untrained

Text Generation

Model card Files Files and versions Community

Sewy2 (untrained) 640m

It is a new MoE architecture which uses the following:

DeepseekV3
nGPT
ResFormer
NeuTRENO (as in resformer)
Tanh logit softcapping (as in Gemma2)

Architecture:

32 Layers
32 Heads
32 KV heads
64 experts
8 experts per token

Downloads last month: 25

Safetensors

Model size

640M params

Tensor type

F32

·

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support