This paper explores how reinforcement learning agents can use environmental features, termed artifacts, to function as external memory. By formalizing this intuition within a mathematical framework, the authors prove that certain observations can reduce the information required to represent an agent's history. Through experiments with spatial navigation tasks using both Linear Q-learning and Deep Q-Networks (DQN), the study demonstrates that observing paths or landmarks allows agents to achieve higher performance with lower internal computational capacity. Notably, this effect of externalized memory emerges unintentionally through the agent's sensory stream without explicit design for memory usage.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
A deep dive into the process of LLM inference, covering tokenization, transformer architecture, KV caching, and optimization techniques for efficient text generation.
A unified memory stack that functions as a memristor as well as a ferroelectric capacitor is reported, enabling both energy-efficient inference and learning at the edge.
