Egor Pakhomov

egor-pakhomov's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

New activity in airtrain-ai/fineweb-edu-fortified 6 months ago

Deduped version of fineweb on HuggingFace yields "This dataset has 218 files that have been marked as unsafe."

1

#103 opened 6 months ago by

egor-pakhomov

New activity in HuggingFaceFW/fineweb 6 months ago

Exact copy of this dataset on HuggingFace yields "This dataset has 218 files that have been marked as unsafe."

1

#50 opened 6 months ago by

egor-pakhomov

Simple exact deduplication removes 2/3 of data.

4

#49 opened 6 months ago by

egor-pakhomov

New activity in mlfoundations/dclm-baseline-1.0 7 months ago

Can't read with Spark

4

#1 opened 7 months ago by

egor-pakhomov