huggingface.co · Datasets
Hub of hundreds of thousands of datasets plus a library for loading and streaming them efficiently.
Hugging Face Datasets is one of 5 dataset tools on Axiomi. See all dataset tools or how we check facts.
// guides
The best LLM pretraining dataset depends on your goal. FineWeb-Edu for small models, DCLM-Baseline and Nemotron-CC for scale, Common Pile for licensed text.
How to fine tune an LLM for code generation: pick a coder base model, build an instruction or FIM dataset, train a LoRA with TRL or Unsloth, test with pass@k.
Train a small LLM from scratch with nanochat, and learn when NVIDIA Megatron-LM fits: model sizes, budgets, datasets, compute math and multi GPU parallelism.
Fine tune an LLM locally, step by step: when it pays off, picking a base model, formatting data, Unsloth vs Axolotl vs TRL vs mlx-lm, training and GGUF export.
// related
Dolmagithub.comAI2's open pretraining corpus and toolkit used to train OLMo, with tools for curating large text datasets.DatasetsOpen source★ 1.6k
LAIONlaion.aiNonprofit producing open multimodal datasets, models and tools for democratising machine learning research.DatasetsFreeHN 2
Common Crawlcommoncrawl.orgNonprofit that maintains a free, open archive of web crawl data used in most large scale pretraining corpora.DatasetsFreeHN 1FineWebhuggingface.coLarge, carefully filtered and deduplicated English web dataset for LLM pretraining, documented with ablations.DatasetsOpen sourceTransformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 168.3kHugging Face Hubhuggingface.coCentral hub for open models, datasets and demo Spaces, the GitHub of the machine learning community.Research PlatformsFreemium★ 4k