strangerzonehf/Flux-C7-Sketch-LoRA
Text-to-Image
•
Updated
•
53
•
•
9
Martin Viewegger
Viewegger
AI & ML interests
None yet
Recent Activity
liked
a model
5 days ago
strangerzonehf/Flux-C7-Sketch-LoRA
liked
a model
5 days ago
strangerzonehf/Flux-Sketch-Smudge-LoRA
new activity
5 days ago
tensorart/stable-diffusion-3.5-medium-turbo:Benefit using Lora + finetuned model
Organizations
None yet
Viewegger's activity
liked
2
models
5 days ago
Benefit using Lora + finetuned model
1
#1 opened 5 days ago
by
Viewegger
liked
2
models
about 1 month ago
Model quality on cca 200hour of audio
2
#8 opened about 1 month ago
by
Viewegger
Dataset size and output quality
#2 opened about 1 month ago
by
Viewegger
Training process details
4
#2 opened about 1 month ago
by
Nils11
liked
a
Space
about 1 month ago
reacted to
m-ric's
post with 🔥
about 1 month ago
Post
787
𝗔𝗿𝗲 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 𝗹𝗮𝘄𝘀 𝗼𝘃𝗲𝗿? 𝗔 𝗿𝗲𝗽𝗼𝗿𝘁 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗻𝗼𝘂𝗻𝗰𝗲𝗱 𝘁𝗵𝗮𝘁 𝗢𝗽𝗲𝗻𝗔𝗜 𝗶𝘀 𝘀𝗲𝗲𝗶𝗻𝗴 𝗱𝗶𝗺𝗶𝗻𝗶𝘀𝗵𝗶𝗻𝗴 𝗿𝗲𝘁𝘂𝗿𝗻𝘀 𝗳𝗿𝗼𝗺 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 𝘂𝗽 𝘁𝗵𝗲 𝗻𝗲𝘅𝘁 𝗚𝗣𝗧 𝗺𝗼𝗱𝗲𝗹𝘀.
📊 What are scaling laws? These are empiric laws that say "Every time you increase compute spent in training 10-fold, your LLM's performance will go up by a predictable tick". Of course, they apply only if you train your model with the right methods.
The image below illustrates it: they're from a paper by Google, "Scaling Autoregressive Models for Content-Rich Text-to-Image Generation", and they show how quality and instruction following of models improve when you scale the model up (which is equivalent to scaling up the compute spent in training).
➡️ These scaling laws have immense impact: they triggered the largest gold rush ever, with companies pouring billions into scaling up theiur training. Microsoft and OpenAI spent 100B into their "Startgate" mega training cluster, due to start running in 2028.
🤔 So, what about these reports of scaling laws slowing down?
If they are true, they would mean a gigantic paradigm shift, as the hundreds of billions poured by AI companies into scaling could be a dead-end. ⛔️
But I doubt it: until the most recent publications, scaling laws showed no signs of weakness, and the researchers at the higher end of the scale-up seems to imply the scaling up continues.
Wait and see!
📊 What are scaling laws? These are empiric laws that say "Every time you increase compute spent in training 10-fold, your LLM's performance will go up by a predictable tick". Of course, they apply only if you train your model with the right methods.
The image below illustrates it: they're from a paper by Google, "Scaling Autoregressive Models for Content-Rich Text-to-Image Generation", and they show how quality and instruction following of models improve when you scale the model up (which is equivalent to scaling up the compute spent in training).
➡️ These scaling laws have immense impact: they triggered the largest gold rush ever, with companies pouring billions into scaling up theiur training. Microsoft and OpenAI spent 100B into their "Startgate" mega training cluster, due to start running in 2028.
🤔 So, what about these reports of scaling laws slowing down?
If they are true, they would mean a gigantic paradigm shift, as the hundreds of billions poured by AI companies into scaling could be a dead-end. ⛔️
But I doubt it: until the most recent publications, scaling laws showed no signs of weakness, and the researchers at the higher end of the scale-up seems to imply the scaling up continues.
Wait and see!
- 1 reply
liked
a
model
about 1 month ago
upvoted
a
paper
about 2 months ago
reacted to
yongchanghao's
post with 🔥
about 2 months ago
Post
3747
We just released a paper (NeuZip) that compresses VRAM in a lossless manner to run larger models. This should be particularly useful when VRAM is insufficient during training/inference. Specifically, we look inside each floating number and find that the exponents are highly compressible (as shown in the figure below).
Read more about the work at NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks (2410.20650)
Read more about the work at NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks (2410.20650)
liked
a
dataset
about 2 months ago
Size of the dataset?
8
#1 opened about 2 months ago
by
Viewegger
liked
2
models
about 2 months ago
Thank you!
1
#1 opened about 2 months ago
by
Viewegger
liked
a
model
about 2 months ago
Hyperaparameters
#1 opened 2 months ago
by
Viewegger