This article explores the field of mechanistic interpretability, aiming to understand how large language models (LLMs) work internally by reverse-engineering their computations. It discusses techniques for identifying and analyzing the functions of individual neurons and circuits within these models, offering insights into their decision-making processes.
Researchers are studying large language models as if they were living things, discovering secrets by applying biological and neurological analysis techniques. This approach is revealing unexpected behaviors and limitations of LLMs.
This paper demonstrates that the inference operations of several open-weight large language models (LLMs) can be mapped to an exactly equivalent linear system for an input sequence. It explores the use of the 'detached Jacobian' to interpret semantic concepts within LLMs and potentially steer next-token prediction.
Sparse autoencoders (SAEs) have been trained on Llama 3.3 70B, releasing an interpreted model accessible via API, enabling research and product development through feature space exploration and steering.
Gemma Scope is an open-source, multi-scale, high-throughput microscope system that combines brightfield, fluorescence, and confocal microscopy, designed for imaging large samples like brain tissue.
DeepMind's Gemma Scope provides researchers with tools to better understand how Gemma 2 language models work through a collection of sparse autoencoders. This helps in understanding the inner workings of these models and addressing concerns like hallucinations and potential manipulation.
This post discusses a study that finds that refusal behavior in language models is mediated by a single direction in the residual stream of the model. The study presents an intervention that bypasses refusal by ablating this direction, and shows that adding in this direction induces refusal. The study is part of a scholars program and provides more details in a forthcoming paper.
