Zhening Li, Joshua Liu, Mateja Vukelic, Nicole Shen, Supriya Lall, Amitayush Thakur, and colleagues at MIT CSAIL introduce JAZ, an agent framework that utilizes a single primitive called `invoke` to perform tasks typically requiring specialized memory or self-improvement systems. By treating the LLM as a runtime provider for function implementations through executable code, the framework allows all inputs and interaction histories to act as variables in the environment.
- The system uses "hooks" instead of dedicated subsystems like file systems or external memory stores to apply constraints and monitoring.
- JAZ outperformed Letta (MemGPT) by 8% at half the cost on recall-heavy tasks within the StuLife dataset.
- In self-improvement evaluations on AppWorld, JAZ exceeded ACE performance by 4% while maintaining a lower cost.
- The system uses "hooks" instead of dedicated subsystems like file systems or external memory stores to apply constraints and monitoring.
- JAZ outperformed Letta (MemGPT) by 8% at half the cost on recall-heavy tasks within the StuLife dataset.
- In self-improvement evaluations on AppWorld, JAZ exceeded ACE performance by 4% while maintaining a lower cost.
Zhening Li and colleagues introduce JAZ, an LLM agent framework designed to minimize the complexity of agent loops by treating them as a programming language primitive called `invoke`. Instead of relying on external specialized systems for memory or self-improvement, JAZ enables agents to achieve these capabilities through code execution where all interactions are treated as variables within the environment. This minimalist approach allows highly expressive workflows, such as long-horizon recall and continual self-improvement, using only prompting rather than manually designed tools or complex architectures.
- The `invoke` primitive allows for recursive calls, enabling LLMs to write arbitrary executable code that includes further iterations of itself.
- In testing on the StuLife dataset, JAZ outperformed MemGPT (Letta) by 8% in recall performance while costing half as much.
- On self-improvement tasks using AppWorld, JAZ demonstrated a 4% improvement over ACE at a lower computational cost.
- The `invoke` primitive allows for recursive calls, enabling LLMs to write arbitrary executable code that includes further iterations of itself.
- In testing on the StuLife dataset, JAZ outperformed MemGPT (Letta) by 8% in recall performance while costing half as much.
- On self-improvement tasks using AppWorld, JAZ demonstrated a 4% improvement over ACE at a lower computational cost.
