INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Paper
•
2411.19799
•
Published
•
11
BigScience Workshop
non-profit
AI & ML interests
A one-year long research workshop on large language models: the Summer of Language Models 21 🌸
Recent Activity
View all activity
bigscience's activity
Jekaterina
authored
2
papers
12 days ago
Adding `safetensors` variant of this model
#2 opened 22 days ago
by
SFconvertbot
Adding `safetensors` variant of this model
#4 opened 23 days ago
by
SFconvertbot
NohTow
authored
3
papers
23 days ago
Multitask Prompted Training Enables Zero-Shot Task Generalization
Paper
•
2110.08207
•
Published
•
2
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Paper
•
2211.05100
•
Published
•
27
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper
•
2412.13663
•
Published
•
121
mgalle
authored
a
paper
24 days ago
Post
2099
🇪🇺 Policy Thoughts in the EU AI Act Implementation 🇪🇺
There is a lot to like in the first draft of the EU GPAI Code of Practice, especially as regards transparency requirements. The Systemic Risks part, on the other hand, is concerning for both smaller developers and for external stakeholders.
I wrote more on this topic ahead of the next draft. TLDR: more attention to immediate large-scale risks and to collaborative solutions supported by evidence can help everyone - as long as developers disclose sufficient information about their design choices and deployment contexts.
Full blog here, based on our submitted response with @frimelle and @brunatrevelin :
https://huggingface.co/blog/yjernite/eu-draft-cop-risks#on-the-proposed-taxonomy-of-systemic-risks
There is a lot to like in the first draft of the EU GPAI Code of Practice, especially as regards transparency requirements. The Systemic Risks part, on the other hand, is concerning for both smaller developers and for external stakeholders.
I wrote more on this topic ahead of the next draft. TLDR: more attention to immediate large-scale risks and to collaborative solutions supported by evidence can help everyone - as long as developers disclose sufficient information about their design choices and deployment contexts.
Full blog here, based on our submitted response with @frimelle and @brunatrevelin :
https://huggingface.co/blog/yjernite/eu-draft-cop-risks#on-the-proposed-taxonomy-of-systemic-risks
- 2 replies
Post
1088
Performance leap: TGI v3 is out. Processes 3x more tokens, 13x faster than vLLM on long prompts. Zero config !
3x more tokens.
By reducing our memory footprint, we’re able to ingest many more tokens and more dynamically than before. A single L4 (24GB) can handle 30k tokens on llama 3.1-8B, while vLLM gets barely 10k. A lot of work went into reducing the footprint of the runtime and its effect are best seen on smaller constrained environments.
13x faster
On long prompts (200k+ tokens) conversation replies take 27.5s in vLLM, while it takes only 2s in TGI. How so ? We keep the initial conversation around, so when a new reply comes in, we can answer almost instantly. The overhead of the lookup is ~5us. Thanks @Dani ël de Kok for the beast data structure.
Zero config
That’s it. Remove all the flags your are using and you’re likely to get the best performance. By evaluating the hardware and model, TGI carefully selects automatic values to give best performance. In production, we don’t have any flags anymore in our deployments. We kept all existing flags around, they may come in handy in niche scenarios.
Read more: https://huggingface.co/docs/text-generation-inference/conceptual/chunking
3x more tokens.
By reducing our memory footprint, we’re able to ingest many more tokens and more dynamically than before. A single L4 (24GB) can handle 30k tokens on llama 3.1-8B, while vLLM gets barely 10k. A lot of work went into reducing the footprint of the runtime and its effect are best seen on smaller constrained environments.
13x faster
On long prompts (200k+ tokens) conversation replies take 27.5s in vLLM, while it takes only 2s in TGI. How so ? We keep the initial conversation around, so when a new reply comes in, we can answer almost instantly. The overhead of the lookup is ~5us. Thanks @Dani ël de Kok for the beast data structure.
Zero config
That’s it. Remove all the flags your are using and you’re likely to get the best performance. By evaluating the hardware and model, TGI carefully selects automatic values to give best performance. In production, we don’t have any flags anymore in our deployments. We kept all existing flags around, they may come in handy in niche scenarios.
Read more: https://huggingface.co/docs/text-generation-inference/conceptual/chunking
Davlan
authored
10
papers
about 1 month ago
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Paper
•
2303.03915
•
Published
•
6
SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)
Paper
•
2304.06845
•
Published
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
Paper
•
2305.06897
•
Published
•
8
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages
Paper
•
2305.13989
•
Published
AfriMTE and AfriCOMET: Empowering COMET to Embrace Under-resourced African Languages
Paper
•
2311.09828
•
Published
•
1
The Effect of Domain and Diacritics in Yorùbá-English Neural Machine Translation
Paper
•
2103.08647
•
Published
MasakhaNER: Named Entity Recognition for African Languages
Paper
•
2103.11811
•
Published
NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis
Paper
•
2201.08277
•
Published
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects
Paper
•
2309.07445
•
Published
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning
Paper
•
2204.06487
•
Published