data-is-better-together/fineweb-c-progress
Viewer
•
Updated
•
762
•
546
•
3
Data Is Better Together
community
AI & ML interests
Building better datasets together
Recent Activity
View all activity
data-is-better-together's activity
davanstrien
updated
a
dataset
about 1 hour ago
davanstrien
updated
a
dataset
about 2 hours ago
davidberenstein1957
posted
an
update
about 22 hours ago
Post
1084
🔦 What? The Hub as a vector search backend!
code: https://gist.github.com/davidberenstein1957/f0157a471ec59d9dd44ae6957f1d52ec
build on DuckDB: https://huggingface.co/docs/hub/en/datasets-duckdb
code: https://gist.github.com/davidberenstein1957/f0157a471ec59d9dd44ae6957f1d52ec
build on DuckDB: https://huggingface.co/docs/hub/en/datasets-duckdb
davanstrien
posted
an
update
2 days ago
Post
2531
Introducing scandi-fine-web-cleaner
davanstrien/scandi-fine-web-cleaner, the first model trained on FineWeb-C community annotations!
FineWeb2 is a massive multilingual dataset for pre-training language models. Like any web-scale dataset, it contains low-quality content. How can we improve it?
Over the past months, an amazing community of 400+ annotators has been labelling content quality (using Argilla) across 23 languages through the FineWeb-C initiative.
Today, I'm happy to share the first classifier trained on this data.
🔍 What we've built:
- A lightweight classifier that efficiently removes low-quality content
- 90%+ precision demonstrated on Danish & Swedish
- Can process the 43M+ documents in Danish FineWeb2 with minimal compute
🌍 Why this matters: The approach can be reproduced for any of the 23 languages in FineWeb-C ( data-is-better-together/fineweb-c). We can improve training data quality at scale without massive compute resources by starting with community annotations and training small, efficient classifiers.
Want to build a classifier for your language? Check out the full blog post with code examples and implementation details: https://danielvanstrien.xyz/posts/2025/FineWeb-c/scandinavian-content-filtering-fineweb.html
FineWeb2 is a massive multilingual dataset for pre-training language models. Like any web-scale dataset, it contains low-quality content. How can we improve it?
Over the past months, an amazing community of 400+ annotators has been labelling content quality (using Argilla) across 23 languages through the FineWeb-C initiative.
Today, I'm happy to share the first classifier trained on this data.
🔍 What we've built:
- A lightweight classifier that efficiently removes low-quality content
- 90%+ precision demonstrated on Danish & Swedish
- Can process the 43M+ documents in Danish FineWeb2 with minimal compute
🌍 Why this matters: The approach can be reproduced for any of the 23 languages in FineWeb-C ( data-is-better-together/fineweb-c). We can improve training data quality at scale without massive compute resources by starting with community annotations and training small, efficient classifiers.
Want to build a classifier for your language? Check out the full blog post with code examples and implementation details: https://danielvanstrien.xyz/posts/2025/FineWeb-c/scandinavian-content-filtering-fineweb.html
-
- 1 reply
davanstrien
posted
an
update
5 days ago
Post
2015
The
data-is-better-together/fineweb-c dataset is growing!
This week a few more languages have got 1,000 annotations for the educational quality of data from HuggingFaceFW/fineweb-2.
Why should you care?
The quality of pre-training data can have a big impact on the performance of downstream language models trained on that data ( HuggingFaceFW/blogpost-fineweb-v1).
Being able to filter by educational quality is on way of improving the quality of the data you use for training an LLM. Very importantly this approach can also reduce the amount of data needed for pertaining.
Why not use an LLM?
LLMs can be used to annotate educational quality for a subset of data. This data can then be used to train a smaller encoder only model to label the full dataset. However, this may not work well for languages outside of english. This is where fineweb-c (community) comes in.
The community is annotating the educational quality of fineweb2 data. Currently 114 languages have some annotations. These annotations will enable a number of things:
- Evaluate whether an LLM can label the educational quality for texts in that language well
- Directly be used for training quality classifiers
- Help discover other rules and huerisitcs for refining fineweb2 further for different languages.
This week the following languages where done:
Swedish thanks to: @Lauler @AntonVic @ohallstrom @bjarlestam @menbom @Ekgren @apsod
Ukrainian thanks to: @hannayukhymenko @robinhad @realPivo @RabotiahovDmytro @reciprocate
Assamese thanks to: @moyoor97 @Arpanjyoti @nawaf-helmi123 @pahigogoi1 @aelhence @kishorekashyap
Want to learn more: https://huggingface.co/blog/davanstrien/fineweb2-community
Contribute yourself here: data-is-better-together/fineweb-c
This week a few more languages have got 1,000 annotations for the educational quality of data from HuggingFaceFW/fineweb-2.
Why should you care?
The quality of pre-training data can have a big impact on the performance of downstream language models trained on that data ( HuggingFaceFW/blogpost-fineweb-v1).
Being able to filter by educational quality is on way of improving the quality of the data you use for training an LLM. Very importantly this approach can also reduce the amount of data needed for pertaining.
Why not use an LLM?
LLMs can be used to annotate educational quality for a subset of data. This data can then be used to train a smaller encoder only model to label the full dataset. However, this may not work well for languages outside of english. This is where fineweb-c (community) comes in.
The community is annotating the educational quality of fineweb2 data. Currently 114 languages have some annotations. These annotations will enable a number of things:
- Evaluate whether an LLM can label the educational quality for texts in that language well
- Directly be used for training quality classifiers
- Help discover other rules and huerisitcs for refining fineweb2 further for different languages.
This week the following languages where done:
Swedish thanks to: @Lauler @AntonVic @ohallstrom @bjarlestam @menbom @Ekgren @apsod
Ukrainian thanks to: @hannayukhymenko @robinhad @realPivo @RabotiahovDmytro @reciprocate
Assamese thanks to: @moyoor97 @Arpanjyoti @nawaf-helmi123 @pahigogoi1 @aelhence @kishorekashyap
Want to learn more: https://huggingface.co/blog/davanstrien/fineweb2-community
Contribute yourself here: data-is-better-together/fineweb-c
-
- 1 reply
nataliaElv
posted
an
update
6 days ago
Post
505
Do you want to easily save annotations to a Dataset in the Hub?
In the last version of Argilla (v2.6.0), you can export your data directly from the UI to the Hub.
Check all the changes and update to the latest version: https://github.com/argilla-io/argilla/releases/tag/v2.6.0
In the last version of Argilla (v2.6.0), you can export your data directly from the UI to the Hub.
Check all the changes and update to the latest version: https://github.com/argilla-io/argilla/releases/tag/v2.6.0
davidberenstein1957
posted
an
update
11 days ago
Post
1920
Fine-tune a SmolLM on domain-specific synthetic data from a LLM
Blog: https://huggingface.co/blog/davidberenstein1957/fine-tune-a-smollm-on-synthetic-data-of-llm
Blog: https://huggingface.co/blog/davidberenstein1957/fine-tune-a-smollm-on-synthetic-data-of-llm
- 1 reply
davidberenstein1957
posted
an
update
16 days ago
Post
1983
Fine-tuning ModernBERT for text classification using synthetic data generation
From prompt to model in 3 steps.
1 dataset description
20 minutes of generating
60 minutes of fine-tuning on my Macbook Pro
Tutorial: https://nbsanity.com/static/552eb50cbd91bedb4e5b73fddca2664a/fine-tune-modernbert-classifier.html
From prompt to model in 3 steps.
1 dataset description
20 minutes of generating
60 minutes of fine-tuning on my Macbook Pro
Tutorial: https://nbsanity.com/static/552eb50cbd91bedb4e5b73fddca2664a/fine-tune-modernbert-classifier.html
davanstrien
posted
an
update
19 days ago
Post
3158
🇸🇰 Hovorte po slovensky? Help build better AI for Slovak!
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
-
- 3 replies
Post
4017
Commits speak louder than words 🤪
* 4 new video models
* Multiple image models, including SANA & Flux Control
* New quantizers -> GGUF & TorchAO
* New training scripts
Enjoy this holiday-special Diffusers release 🤗
Notes: https://github.com/huggingface/diffusers/releases/tag/v0.32.0
* 4 new video models
* Multiple image models, including SANA & Flux Control
* New quantizers -> GGUF & TorchAO
* New training scripts
Enjoy this holiday-special Diffusers release 🤗
Notes: https://github.com/huggingface/diffusers/releases/tag/v0.32.0
davanstrien
posted
an
update
26 days ago
Post
1765
Introducing FineWeb-C 🌐🎓, a community-built dataset for improving language models in ALL languages.
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! 🌍
data-is-better-together/fineweb-c
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! 🌍
data-is-better-together/fineweb-c
burtenshaw
posted
an
update
27 days ago
Post
2671
People are flexing their end of year stats, so I made this app to show hub stats in a tidy design!
Thanks @Ameeeee and @jfcalvo for the feature from Argilla!
burtenshaw/recap
Thanks @Ameeeee and @jfcalvo for the feature from Argilla!
burtenshaw/recap
- 1 reply
davidberenstein1957
posted
an
update
27 days ago
Post
1355
🐇 Tumble down the AI rabbit hole without any technical knowledge!
Explore AI models on the Hub by a simple and quick search
Demo: davidberenstein1957/transformers-pipeline-playground
Explore AI models on the Hub by a simple and quick search
Demo: davidberenstein1957/transformers-pipeline-playground
Post
1911
In the past seven days, the Diffusers team has shipped:
1. Two new video models
2. One new image model
3. Two new quantization backends
4. Three new fine-tuning scripts
5. Multiple fixes and library QoL improvements
Coffee on me if someone can guess 1 - 4 correctly.
1. Two new video models
2. One new image model
3. Two new quantization backends
4. Three new fine-tuning scripts
5. Multiple fixes and library QoL improvements
Coffee on me if someone can guess 1 - 4 correctly.
- 1 reply
nataliaElv
posted
an
update
29 days ago
Post
1659
If you are still wondering how the FineWeb2 annotations are done, how to follow the guidelines or how Argilla works, this is your video!
I go through a few samples of the FineWeb2 dataset and classify them based on their educational content. Check it out!
https://www.youtube.com/watch?v=_-ORB4WAVGU
I go through a few samples of the FineWeb2 dataset and classify them based on their educational content. Check it out!
https://www.youtube.com/watch?v=_-ORB4WAVGU
davidberenstein1957
posted
an
update
30 days ago
Post
4193
Introducing the Synthetic Data Generator, a user-friendly application that takes a no-code approach to creating custom datasets with Large Language Models (LLMs). The best part: A simple step-by-step process, making dataset creation a non-technical breeze, allowing anyone to create datasets and models in minutes and without any code.
Blog: https://huggingface.co/blog/synthetic-data-generator
Space: argilla/synthetic-data-generator
Blog: https://huggingface.co/blog/synthetic-data-generator
Space: argilla/synthetic-data-generator
-
- 4 replies
nataliaElv
posted
an
update
about 1 month ago
Post
1285
How do your annotations for FineWeb2 compare to your teammates'?
I started contributing some annotations to the FineWeb2 collaborative annotation sprint and I wanted to know if my labelling trends were similar to those of my teammates.
I did some analysis and I wasn't surprised to see that I'm being a bit harsher on my evaluations than my mates 😂
Do you want to see how your annotations compare to others?
👉 Go to this Gradio space: nataliaElv/fineweb2_compare_my_annotations
✍️ Enter the dataset that you've contributed to and your Hugging Face username.
How were your results?
- Contribute some annotations: data-is-better-together/fineweb-c
- Join your language channel in Rocket chat: HuggingFaceFW/discussion
I started contributing some annotations to the FineWeb2 collaborative annotation sprint and I wanted to know if my labelling trends were similar to those of my teammates.
I did some analysis and I wasn't surprised to see that I'm being a bit harsher on my evaluations than my mates 😂
Do you want to see how your annotations compare to others?
👉 Go to this Gradio space: nataliaElv/fineweb2_compare_my_annotations
✍️ Enter the dataset that you've contributed to and your Hugging Face username.
How were your results?
- Contribute some annotations: data-is-better-together/fineweb-c
- Join your language channel in Rocket chat: HuggingFaceFW/discussion
burtenshaw
posted
an
update
about 1 month ago
Post
2430
Quick update from week 1 of smol course. The community is taking the driving seat and using the material for their own projects. If you want to do the same, join in!
- we have ongoing translation projects in Korean, Vietnamese, Portuguese, and Spanish
- 3 chapters are ready for students. On topics like, instruction tuning, preference alignment, and parameter efficient fine tuning
- 3 chapters are in progress on evaluation, vision language models, and synthetic data.
- around 780 people have forked the repo to use it for learning, teaching, sharing.
⏭️ Next step is to support people that want to use the course for teaching, content creation, internal knowledge sharing, or anything. If you're into this. Drop an issue or PR
REPO: https://buff.ly/3ZCMKX2
discord channel: https://buff.ly/4f9F8jA
- we have ongoing translation projects in Korean, Vietnamese, Portuguese, and Spanish
- 3 chapters are ready for students. On topics like, instruction tuning, preference alignment, and parameter efficient fine tuning
- 3 chapters are in progress on evaluation, vision language models, and synthetic data.
- around 780 people have forked the repo to use it for learning, teaching, sharing.
⏭️ Next step is to support people that want to use the course for teaching, content creation, internal knowledge sharing, or anything. If you're into this. Drop an issue or PR
REPO: https://buff.ly/3ZCMKX2
discord channel: https://buff.ly/4f9F8jA
Post
2086
Introducing a high-quality open-preference dataset to further this line of research for image generation.
Despite being such an inseparable component for modern image generation, open preference datasets are a rarity!
So, we decided to work on one with the community!
Check it out here:
https://huggingface.co/blog/image-preferences
Despite being such an inseparable component for modern image generation, open preference datasets are a rarity!
So, we decided to work on one with the community!
Check it out here:
https://huggingface.co/blog/image-preferences
-
- 7 replies
davidberenstein1957
posted
an
update
about 1 month ago
Post
2075
Open Preference Dataset for Text-to-Image Generation by the 🤗 Community
Open Image Preferences is an Apache 2.0 licensed dataset for text-to-image generation. This dataset contains 10K text-to-image preference pairs across common image generation categories, while using different model families and varying prompt complexities.
https://huggingface.co/blog/image-preferences
Open Image Preferences is an Apache 2.0 licensed dataset for text-to-image generation. This dataset contains 10K text-to-image preference pairs across common image generation categories, while using different model families and varying prompt complexities.
https://huggingface.co/blog/image-preferences