OpenMythos is an open-source PyTorch project by Kye Gomez that proposes a theoretical reconstruction of Anthropic's Claude Mythos architecture. Instead of standard transformer layers, it suggests a Recurrent-Depth Transformer (RDT) design where weights loop through multiple iterations to increase reasoning depth during inference. By combining Mixture-of-Experts with Multi-Latent Attention and stability constraints, the model achieves performance parity between 770M parameters and a 1.3B parameter standard transformer.
* open-source PyTorch reconstruction of claude mythos
* proposes recurrent-depth transformer architecture
* reasoning depth scales via inference-time loops rather than parameter count
* uses mixture-of-experts for domain breadth
* implements multi-latent attention to reduce memory usage
* employs lti injection and adaptive computation time for stability
* achieves 1.3b parameter performance with only 770m parameters
* open-source PyTorch reconstruction of claude mythos
* proposes recurrent-depth transformer architecture
* reasoning depth scales via inference-time loops rather than parameter count
* uses mixture-of-experts for domain breadth
* implements multi-latent attention to reduce memory usage
* employs lti injection and adaptive computation time for stability
* achieves 1.3b parameter performance with only 770m parameters
