SHAP
shap.readthedocs.io · Interpretability
Game theoretic approach to explaining the output of any machine learning model using Shapley values.
SHAP is one of 8 interpretability tools on Axiomi. See all interpretability tools or how we check facts.
Open source
// related
SHAP alternatives and related tools
Captumcaptum.aiPyTorch library for model interpretability with integrated gradients, saliency and other attribution methods.InterpretabilityOpen source★ 5.7k
TransformerLensgithub.comLibrary for mechanistic interpretability that exposes and edits the internal activations of GPT style models.InterpretabilityOpen source★ 4k
Goodfiregoodfire.comInterpretability research company building tools to understand and edit the internal workings of neural networks.InterpretabilitySee siteHN 6SAELensgithub.comToolkit for training and analysing sparse autoencoders that decompose model activations into interpretable features.InterpretabilityOpen source★ 1.6k
Neuronpedianeuronpedia.orgFree, open source (Apache 2.0) platform for browsing, searching and steering sparse autoencoder features, with circuit tracing graphs, the Jacobian Lens and a public API.InterpretabilityOpen source★ 1.2k
nnsightnnsight.netLibrary for inspecting and intervening on model internals, with remote execution on large models through NDIF.InterpretabilityOpen source★ 1.1k