Mohamed Aymane Farhi's picture

Mohamed Aymane Farhi

ayymen

AI & ML interests

NLP

Recent Activity

Organizations

1337fil's profile picture Tamazight NLP's profile picture Moroccan Data Scientists's profile picture Hugging Face Discord Community's profile picture

ayymen's activity

🚩 Report: Not working

2
#1 opened about 2 months ago by
ayymen
reacted to thomwolf's post with πŸš€ 15 days ago
view post
Post
4336
We are proud to announce HuggingFaceFW/fineweb-2: A sparkling update to HuggingFaceFW/fineweb with 1000s of πŸ—£οΈlanguages.

We applied the same data-driven approach that led to SOTA English performance in🍷 FineWeb to thousands of languages.

πŸ₯‚ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.

The dataset is released under the permissive πŸ“œ ODC-By 1.0 license, and the πŸ’» code to reproduce it and our evaluations is public.

We will very soon announce a big community project, and are working on a πŸ“ blogpost walking you through the entire dataset creation process. Stay tuned!

In the mean time come ask us question on our chat place: HuggingFaceFW/discussion

H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
  • 2 replies
Β·