Steerling-8B is an interpretable causal diffusion language model that combines masked diffusion language modeling with concept decomposition, enabling generation, attribution, steering, and extraction of hidden representations. It offers features like block-causal attention and decomposition of hidden states into known and unknown concepts.
Researchers from the University of California San Diego have developed a mathematical formula that explains how neural networks learn and detect relevant patterns in data, providing insight into the mechanisms behind neural network learning and enabling improvements in machine learning efficiency.
