github.com · Interpretability
Library for mechanistic interpretability that exposes and edits the internal activations of GPT style models.
TransformerLens is one of 8 interpretability tools on Axiomi. See all interpretability tools or how we check facts.
// guides
What is mechanistic interpretability research? A plain guide to features, circuits, superposition, sparse autoencoders, key papers, open tools and how to start.
Neuronpedia explained (SAE features, Jacobian Lens, circuit tracer, API) and TransformerLens vs nnsight compared: TransformerLens 4, nnsight 0.8 and NDIF.
// related
Goodfiregoodfire.comInterpretability research company building tools to understand and edit the internal workings of neural networks.InterpretabilitySee siteHN 6
Transformer Circuitstransformer-circuits.pubAnthropic's interpretability publication thread, home to influential work on features, superposition and circuit tracing.InterpretabilityFreeHN 1
PyTorchpytorch.orgThe dominant deep learning framework in research, with eager execution, torch.compile and a huge ecosystem.ML FrameworksOpen source★ 105.1k
SHAPshap.readthedocs.ioGame theoretic approach to explaining the output of any machine learning model using Shapley values.InterpretabilityOpen source★ 25.8k
Captumcaptum.aiPyTorch library for model interpretability with integrated gradients, saliency and other attribution methods.InterpretabilityOpen source★ 5.7kSAELensgithub.comToolkit for training and analysing sparse autoencoders that decompose model activations into interpretable features.InterpretabilityOpen source★ 1.6k