Zhening Li and colleagues introduce JAZ, an LLM agent framework designed to minimize the complexity of agent loops by treating them as a programming language primitive called `invoke`. Instead of relying on external specialized systems for memory or self-improvement, JAZ enables agents to achieve these capabilities through code execution where all interactions are treated as variables within the environment. This minimalist approach allows highly expressive workflows, such as long-horizon recall and continual self-improvement, using only prompting rather than manually designed tools or complex architectures.
- The `invoke` primitive allows for recursive calls, enabling LLMs to write arbitrary executable code that includes further iterations of itself.
- In testing on the StuLife dataset, JAZ outperformed MemGPT (Letta) by 8% in recall performance while costing half as much.
- On self-improvement tasks using AppWorld, JAZ demonstrated a 4% improvement over ACE at a lower computational cost.
- The `invoke` primitive allows for recursive calls, enabling LLMs to write arbitrary executable code that includes further iterations of itself.
- In testing on the StuLife dataset, JAZ outperformed MemGPT (Letta) by 8% in recall performance while costing half as much.
- On self-improvement tasks using AppWorld, JAZ demonstrated a 4% improvement over ACE at a lower computational cost.
This article examines the architectural implications of choosing between stateless and stateful designs when building agentic systems. It evaluates how an agent's approach to managing memory impacts deployment, horizontal scaling, and client-side complexity.
- Stateless agents allow for easy horizontal scaling since no user memory is stored on a backend server, but they require the client to send the full conversation history with every request, leading to increased token usage as conversations grow.
- Stateful agents manage their own context through a database layer using session identifiers, which simplifies client interactions and supports complex workflows, though it introduces challenges in distributed scaling and data persistence.
- Stateless agents allow for easy horizontal scaling since no user memory is stored on a backend server, but they require the client to send the full conversation history with every request, leading to increased token usage as conversations grow.
- Stateful agents manage their own context through a database layer using session identifiers, which simplifies client interactions and supports complex workflows, though it introduces challenges in distributed scaling and data persistence.
