The article details “autoresearch,” a project by Karpathy where an AI agent autonomously experiments with training a small language model (nanochat) to improve its performance. The agent modifies the `train.py` file, trains for a fixed 5-minute period, and evaluates the results, repeating this process to iteratively refine the model. The project aims to demonstrate autonomous AI research, focusing on a simplified, single-GPU setup with a clear metric (validation bits per byte).
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
Google has released a new command-line interface for Google Workspace apps, designed to make it easier for AI agents like OpenClaw to interface with Google apps like Docs, Drive, and Gmail. The tool offers over 100 Agent Skills to simplify agent actions and supports integrations with other AI agents beyond OpenClaw. While published by Google, it's not an officially supported product, so use it at your own risk.
We study two recurring phenomena in Transformer language models: massive activations, in which a small number of tokens exhibit extreme outliers in a few channels, and attention sinks, in which certain tokens attract disproportionate attention mass regardless of semantic relevance. Prior work observes that these phenomena frequently co-occur and often involve the same tokens, but their functional roles and causal relationship remain unclear. Through systematic experiments, we show that the co-occurrence is largely an architectural artifact of modern Transformer design, and that the two phenomena serve related but distinct functions. Massive activations operate globally: they induce near-constant hidden representations that persist across layers, effectively functioning as implicit parameters of the model. Attention sinks operate locally: they modulate attention outputs across heads and bias individual heads toward short-range dependencies. We identify the pre-norm configuration as the key choice that enables the co-occurrence, and show that ablating it causes the two phenomena to decouple.
discrawl mirrors Discord guild data into a local SQLite database, allowing you to search, inspect, and query server history independently of Discord. It’s a bot-token crawler – no user-token hacks – and keeps your data local. It discovers accessible guilds, syncs channels, threads, members, and message history, maintains FTS5 search indexes for fast text search (including small attachments), records mentions, and tails Gateway events for live updates with repair syncs. It provides read-only SQL access for analysis and supports multi-guild schemas with a simple single-guild default. Search defaults to all guilds, while sync and tail default to a configured default guild or fan out to all discovered guilds if none is set.
A new ETH Zurich study challenges the common practice of using `AGENTS.md` files with AI coding agents. LLM-generated context files decrease performance (3% lower success rate, +20% steps/costs).Human-written files offer small gains (4% success rate) but also increase costs. Researchers recommend omitting context files unless manually written with non-inferable details (tooling, build commands).They tested this using a new dataset, AGENTbench, with four agents.
RAG combines language models with external knowledge. This article explores context & retrieval in RAG, covering search methods (keywords, TF-IDF, embeddings/FAISS/Chroma), context length challenges (compression, re-ranking), and contextual retrieval (query & conversation history).
Timer-S1 is a scalable Mixture-of-Experts time series model with 8.3B parameters that uses serial scaling and novel TimeMoE blocks to improve long-term forecasting accuracy.
We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 will be released to facilitate further research.
We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 will be released to facilitate further research.
This article discusses how to effectively utilize Large Language Models (LLMs) by acknowledging their superior processing capabilities and adapting prompting techniques. It emphasizes the importance of brevity, directness, and providing relevant context (through RAG and MCP servers) to maximize LLM performance. The article also highlights the need to treat LLM responses as drafts and use Socratic prompting for refinement, while acknowledging their potential for "hallucinations." It suggests formatting output expectations (JSON, Markdown) and utilizing role-playing to guide the LLM towards desired results. Ultimately, the author argues that LLMs, while not inherently "smarter" in a human sense, possess vast knowledge and can be incredibly powerful tools when approached strategically.
Comprehensive guide to prompt engineering techniques for Claude's
latest models, including Claude Opus 4.6, Claude Sonnet 4.6, and
Claude Haiku 4.5. It covers foundational techniques, output
control, tool use, thinking, and agentic systems.
latest models, including Claude Opus 4.6, Claude Sonnet 4.6, and
Claude Haiku 4.5. It covers foundational techniques, output
control, tool use, thinking, and agentic systems.
This article explores five Python decorators that can be used to optimize LLM-based applications. These decorators leverage libraries like functools, diskcache, tenacity, ratelimit, and magnetic to address common challenges such as caching, network resilience, rate limiting, and structured output binding. The article provides code examples to illustrate how each decorator can be implemented and used to improve the performance and reliability of LLM applications.
