Steerling-8B is an interpretable causal diffusion language model that combines masked diffusion language modeling with concept decomposition, enabling generation, attribution, steering, and extraction of hidden representations. It offers features like block-causal attention and decomposition of hidden states into known and unknown concepts.
An article discussing the importance of explainability in machine learning and the challenges posed by neural networks. It highlights the difficulties in understanding the decision-making process of complex models and the need for more transparency in AI development.
