Hugging Face's logo
Danielus's picture
13 10 127

Danielus

danielus
maddes8cht's profile picture 21world's profile picture 2seb2's profile picture
Ā·

AI & ML interests

None yet

Recent Activity

reacted to YerbaPage's post with šŸ‘€ 8 days ago
Curated list of **Repository-level Code Generation** papers & benchmarks! šŸ”„ Stay ahead with the latest in: āœ… Repo-level Issue Resolution āœ… Repo-level Code Completion āœ… Datasets & Benchmarks šŸ‘‰ Check it out: https://github.com/YerbaPage/Awesome-Repo-Level-Code-Generation šŸ”„
reacted to m-ric's post with šŸ”„ 11 days ago
š—£š—¼š˜š—²š—»š˜š—¶š—®š—¹ š—½š—®š—暝—®š—±š—¶š—“š—ŗ š˜€š—µš—¶š—³š˜ š—¶š—» š—Ÿš—Ÿš— š˜€: š—»š—²š˜„ š—½š—®š—½š—²š—æ š—Æš˜† š— š—²š˜š—® š—°š—¹š—®š—¶š—ŗš˜€ š˜š—µš—®š˜ š˜„š—² š—°š—®š—» š—“š—²š˜ š—暝—¶š—± š—¼š—³ š˜š—¼š—øš—²š—»š—¶š˜‡š—²š—暝˜€! šŸ„³ Current LLMs process text by first splitting it into tokens. They use a module named "tokenizer", that -spl-it-s- th-e- te-xt- in-to- arbitrary tokens depending on a fixed dictionnary. On the Hub you can find this dictionary in a model's files under tokenizer.json. āž”ļø This process is called BPE tokenization. It is suboptimal, everyone says it. It breaks text into predefined chunks that often fail to capture the nuance of language. But it has been a necessary evil in language models since their inception. šŸ’„ In Byte Latent Transformer (BLT), Meta researchers propose an elegant solution by eliminating tokenization entirely, working directly with raw bytes while maintaining efficiency through dynamic "patches." This had been tried before with different byte-level tokenizations, but it's the first time that an architecture of this type scales as well as BPE tokenization. And it could mean a real paradigm shift! šŸ‘šŸ‘ šŸ—ļø š—”š—暝—°š—µš—¶š˜š—²š—°š˜š˜‚š—暝—²: Instead of a lightweight tokenizer, BLT has a lightweight encoder that process raw bytes into patches. Then the patches are processed by the main heavy-duty transformers as we do normally (but for patches of bytes instead of tokens), before converting back to bytes. šŸ§© š——š˜†š—»š—®š—ŗš—¶š—° š—£š—®š˜š—°š—µš—¶š—»š—“: Instead of fixed tokens, BLT groups bytes based on their predictability (measured by entropy) - using more compute for complex sequences and efficiently handling simple ones. This allows efficient processing while maintaining byte-level understanding. I hope this breakthrough is confirmed and we can get rid of all the tokenizer stuff, it will make model handling easier! Read their paper here šŸ‘‰ https://dl.fbaipublicfiles.com/blt/BLT__Patches_Scale_Better_Than_Tokens.pdf
reacted to julien-c's post with šŸ”„ 15 days ago
After some heated discussion šŸ”„, we clarify our intent re. storage limits on the Hub TL;DR: - public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible - private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise) docs: https://huggingface.co/docs/hub/storage-limits We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community šŸ”„ cc: @reach-vb @pierric @victor and the HF team
View all activity

Organizations

None yet

danielus's activity

upvoted 2 collections 6 months ago
upvoted an article 7 months ago
view article
Article

Orchestration of Experts: The First-Principle Multi-Model System

By alirezamsh ā€¢
ā€¢ 15
upvoted a collection 7 months ago
upvoted an article 7 months ago
upvoted an article 8 months ago
upvoted a collection 8 months ago
upvoted an article 8 months ago
view article
Article

Welcome Llama 3 - Meta's new open LLM

ā€¢ 281
upvoted 2 papers 10 months ago