Common Crawl
commoncrawl.org · Datasets
Nonprofit that maintains a free, open archive of web crawl data used in most large scale pretraining corpora.
Common Crawl is one of 5 dataset tools on Axiomi. See all dataset tools or how we check facts.
Free
// guides
Guides that cover Common Crawl
Best LLM Pretraining Dataset in 2026: Top Picks
The best LLM pretraining dataset depends on your goal. FineWeb-Edu for small models, DCLM-Baseline and Nemotron-CC for scale, Common Pile for licensed text.
// related
Common Crawl alternatives and related tools
LAIONlaion.aiNonprofit producing open multimodal datasets, models and tools for democratising machine learning research.DatasetsFreeHN 2
Dolmagithub.comAI2's open pretraining corpus and toolkit used to train OLMo, with tools for curating large text datasets.DatasetsOpen source★ 1.6k
FineWebhuggingface.coLarge, carefully filtered and deduplicated English web dataset for LLM pretraining, documented with ablations.DatasetsOpen sourceHugging Face Datasetshuggingface.coHub of hundreds of thousands of datasets plus a library for loading and streaming them efficiently.DatasetsFree★ 22.1kMegatron-LMgithub.comNVIDIA's open source codebase for training transformers at scale: training scripts plus the Megatron Core library with tensor, pipeline, data, expert and context parallelism.Training & Fine tuningOpen source★ 18.3knanoGPTgithub.comAndrej Karpathy's minimal repository for training and fine tuning medium sized GPTs, a classic learning resource.Training & Fine tuningOpen source★ 64.2k