This paper explores how reinforcement learning agents can use environmental features, termed artifacts, to function as external memory. By formalizing this intuition within a mathematical framework, the authors prove that certain observations can reduce the information required to represent an agent's history. Through experiments with spatial navigation tasks using both Linear Q-learning and Deep Q-Networks (DQN), the study demonstrates that observing paths or landmarks allows agents to achieve higher performance with lower internal computational capacity. Notably, this effect of externalized memory emerges unintentionally through the agent's sensory stream without explicit design for memory usage.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
- Formalization of artifacts as observations that encode information about the past.
- The Artifact Reduction Theorem proving environmental artifacts reduce history representation requirements.
- Empirical evidence showing reduced internal capacity needs when spatial paths are visible.
- Observation that externalized memory can emerge implicitly in standard RL agents.
- Implications for agent design, suggesting performance gains may come from environment-agent coevolution rather than just scaling parameters.
AutoAgent is a revolutionary open-source library designed to automate the tedious process of agent engineering and prompt tuning. By employing a meta-agent, the library allows for the autonomous optimization of an agent's harness, including system prompts, tool definitions, and orchestration strategies, all without human intervention. During a 24-hour run, AutoAgent achieved impressive results, including the top score on SpreadsheetBench and a leading GPT-5 score on TerminalBench. This technology effectively transitions the human's role from a manual engineer to a high-level director, enabling rapid, self-improving agent development across various domains and benchmarks.
This project, `autoresearch-opencode`, is an autonomous experiment loop designed for use with OpenCode. It's a port of `pi-autoresearch`, but implemented as a pure skill, eliminating the need for an MCP server and relying solely on instructions the agent follows using its built-in tools. The skill allows users to automate optimization tasks, as demonstrated by the example of optimizing the BogoSort algorithm which achieved a 7,802x speedup by leveraging Python's `bisect` module for sorted-state detection.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
MIT researchers developed a new approach that rethinks how a classic method, known as Bayesian optimization, can be used to solve problems with hundreds of variables. In tests on realistic engineering-style benchmarks, like power-system optimization, the approach found top solutions 10 to 100 times faster than widely used methods.
Their technique leverages a foundation model trained on tabular data that automatically identifies the variables that matter most for improving performance, repeating the process to hone in on better and better solutions. The researchers’ tabular foundation model does not need to be constantly retrained as it works toward a solution, increasing the efficiency of the optimization process.
The technique also delivers greater speedups for more complicated problems, so it could be especially useful in demanding applications like materials development or drug discovery. The research will be presented at the International Conference on Learning Representations.
Their technique leverages a foundation model trained on tabular data that automatically identifies the variables that matter most for improving performance, repeating the process to hone in on better and better solutions. The researchers’ tabular foundation model does not need to be constantly retrained as it works toward a solution, increasing the efficiency of the optimization process.
The technique also delivers greater speedups for more complicated problems, so it could be especially useful in demanding applications like materials development or drug discovery. The research will be presented at the International Conference on Learning Representations.
DeepScientist is a goal-oriented, fully autonomous scientific discovery system. It uses Bayesian Optimization and a hierarchical 'hypothesize, verify, and analyze' process with a Findings Memory to balance exploration and exploitation. It generated and validated thousands of scientific ideas, surpassing human SOTA on three AI tasks.
A comprehensive guide covering the most critical machine learning equations, including probability, linear algebra, optimization, and advanced concepts, with Python implementations.
This article explains how derivatives, gradients, Jacobians, and Hessians fit together and shows examples of what they are used for, including optimization and rendering.
Optuna is an open-source hyperparameter optimization framework designed to automate the hyperparameter search process for machine learning models. It supports various frameworks like TensorFlow, Keras, Scikit-Learn, XGBoost, and LightGBM, offering features like eager search spaces, state-of-the-art algorithms, and easy parallelization.
"An example of simultaneously optimizing two policies for two adversarial agents, looking specifically at the cat and mouse game."
The article explores developing strategies for two players with conflicting goals, using methods like game trees, reinforcement learning, and hill-climbing optimization. The focus is on determining optimal policies for each player to either catch or evade capture, considering board configurations and player turn orders. The article further details how hill climbing is applied to improve strategies incrementally, using variations in policies to evaluate and enhance performance over numerous iterations.
The article explores developing strategies for two players with conflicting goals, using methods like game trees, reinforcement learning, and hill-climbing optimization. The focus is on determining optimal policies for each player to either catch or evade capture, considering board configurations and player turn orders. The article further details how hill climbing is applied to improve strategies incrementally, using variations in policies to evaluate and enhance performance over numerous iterations.
This article provides an overview of feature selection in machine learning, detailing methods to maximize model accuracy, minimize computational costs, and introduce a novel method called History-based Feature Selection (HBFS).
