Berkeley AI Research introduces SPEX and ProxySPEX to identify LLM interactions at scale

Berkeley AI Research describes SPEX and ProxySPEX, algorithms designed to identify critical, high-order interactions within Large Language Models (LLMs) while significantly reducing the computational cost of ablation. These methods apply ablation—measuring influence by component removal—to feature attribution, data attribution, and mechanistic interpretability. SPEX achieves fidelity even with thousands of features, and ProxySPEX reduces computational cost by approximately 10 times using hierarchical assumptions, improving the efficiency of analyzing features, training data, and internal model components.