wandb.ai · Training & Fine tuning
Experiment tracking, model registry and dashboards used by many research teams to compare training runs; now part of CoreWeave.
Weights & Biases is one of 9 training and fine tuning tools on Axiomi. See all training and fine tuning tools or how we check facts.
// guides
Train a small LLM from scratch with nanochat, and learn when NVIDIA Megatron-LM fits: model sizes, budgets, datasets, compute math and multi GPU parallelism.
// related
Unslothunsloth.aiFine tuning toolkit that speeds up LoRA and full training while cutting GPU memory use, with free notebooks.Training & Fine tuningOpen source★ 78.7k
DeepSpeeddeepspeed.aiLibrary for large scale distributed training and inference, known for the ZeRO memory optimisation family.Training & Fine tuningOpen source★ 43.3k
PEFThuggingface.coParameter efficient fine tuning methods such as LoRA, letting large models adapt on modest hardware.Training & Fine tuningOpen source★ 21.9kTRLhuggingface.coHugging Face library for post training models with supervised fine tuning, DPO, GRPO and other preference optimisation methods.Training & Fine tuningOpen source★ 19.6k
Megatron-LMgithub.comNVIDIA's open source codebase for training transformers at scale: training scripts plus the Megatron Core library with tensor, pipeline, data, expert and context parallelism.Training & Fine tuningOpen source★ 18.3knanoGPTgithub.comAndrej Karpathy's minimal repository for training and fine tuning medium sized GPTs, a classic learning resource.Training & Fine tuningOpen source★ 64.2k