TeichAI/Gemma-4-31B-Fable-5-Agent-Distill-GGUF
31B • Updated • 7.81k • 6
Rod
Viac
AI & ML interests
playing around with a tool
Recent Activity
liked a model about 2 hours ago
TeichAI/Gemma-4-31B-Fable-5-Agent-Distill-GGUF liked a model 7 days ago
deepreinforce-ai/Ornith-1.0-35B-GGUF liked a model 24 days ago
BennyDaBall/Z-Image-Engineer-V6Organizations
None yet
liked a
model about 2 hours ago
liked a
model 7 days ago
liked a
model 24 days ago
liked a
model 27 days ago
liked a
model about 1 month ago
liked a
model about 2 months ago
liked a
model 2 months ago
liked 10
models 3 months ago
Qwen/Qwen3.6-27B
Image-Text-to-Text • 28B • Updated • 4.85M • • 1.91k
moonshotai/Kimi-K2.6
Image-Text-to-Text • 1.1T • Updated • 1.99M • • 1.51k
lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-IQ4_XS-GGUF
Text Generation • 35B • Updated • 3.01k • 31
Qwen/Qwen3.6-35B-A3B-FP8
Image-Text-to-Text • 36B • Updated • 6.86M • 301
nvidia/Lyra-2.0
Image-to-3D • Updated • 535 • 331
varjosoft/GLM-4.7-Flash-TQ3
13B • Updated • 8 • 1
Youssofal/MiniMax-M2.7-Abliterated-Heretic-GGUF
Text Generation • 229B • Updated • 1.63k • 46
MiniMaxAI/MiniMax-M2.7
Text Generation • 229B • Updated • 1.18M • • 1.23k
nwzjk/GLM-5.1-Open-TQ3
Text Generation • 289B • Updated • 8 • 2
ruv/ruvltra-claude-code
Text Generation • 0.5B • Updated • 8.46k • 187
liked a
dataset 3 months ago
reacted to reaperdoesntknow's post with 👍 3 months ago
Post
3296
We present a methodology for training small language models on CPU at FP32 precision
that achieves capability-per-dollar efficiency orders of magnitude beyond GPU-based training.
Across15modelsspanningfournovelarchitecturefamilies—MixtureofAttentions(MoA),cross-
architecture fusion (Qemma), swarm intelligence (SAGI), and metric-space causal language
models (DiscoverLM)—total compute cost was $24 on a single AMD EPYC 9454P proces-
sor. We introduce seven methodological pillars: (1) FP32 precision preservation, with exper-
iments demonstrating 5,810×single-operation error and 23,225×compounding error ratio for
FP16 at network depth; (2) sparse cognitive architectures where 0.02–7% of parameters activate
per token, matching CPU branching rather than GPU SIMD; (3) developmental curriculum
training progressing from language to logic to transfer to depth; (4) continuous belt-fed data
ingestion eliminating truncation waste; (5) hardware-native optimization for AMD Zen 4 via
AOCL/OpenMP/NUMA-aware allocation; (6) self-regulating thermodynamic governance with
emergent temperature measurement grounded in L2-star discrepancy; and (7) open-standard
compute (AVX2 SIMD at FP32) free of proprietary vendor dependency. We argue that transformers were designed for GPU hardware rather than mathematical optimality, and that architecture designed for geometric correctness—metric-space attention, triangle inequality enforcement, sparse expert routing—naturally favor CPU execution. For sub-2B parameter models, CPU training produces more capable models at a fraction of the cost.
that achieves capability-per-dollar efficiency orders of magnitude beyond GPU-based training.
Across15modelsspanningfournovelarchitecturefamilies—MixtureofAttentions(MoA),cross-
architecture fusion (Qemma), swarm intelligence (SAGI), and metric-space causal language
models (DiscoverLM)—total compute cost was $24 on a single AMD EPYC 9454P proces-
sor. We introduce seven methodological pillars: (1) FP32 precision preservation, with exper-
iments demonstrating 5,810×single-operation error and 23,225×compounding error ratio for
FP16 at network depth; (2) sparse cognitive architectures where 0.02–7% of parameters activate
per token, matching CPU branching rather than GPU SIMD; (3) developmental curriculum
training progressing from language to logic to transfer to depth; (4) continuous belt-fed data
ingestion eliminating truncation waste; (5) hardware-native optimization for AMD Zen 4 via
AOCL/OpenMP/NUMA-aware allocation; (6) self-regulating thermodynamic governance with
emergent temperature measurement grounded in L2-star discrepancy; and (7) open-standard
compute (AVX2 SIMD at FP32) free of proprietary vendor dependency. We argue that transformers were designed for GPU hardware rather than mathematical optimality, and that architecture designed for geometric correctness—metric-space attention, triangle inequality enforcement, sparse expert routing—naturally favor CPU execution. For sub-2B parameter models, CPU training produces more capable models at a fraction of the cost.
- 6 replies
liked a
model 4 months ago
