Jiahe Geng writes about RSM-full, an online clustered-memory pipeline designed to optimize the quality-to-token trade-off for long-horizon LLM deployments with limited prompt budgets. By utilizing a cosine-gated max-member merge rule and atom-aware grouped context packing, the method achieves significant performance gains in compact-memory regimes compared to existing baselines like Online K-Means and A-MEM. The approach is particularly effective when maintaining an answer quality of 83% for Full-Context level tasks while utilizing only 32% of the total token cost within a 4k budget.
- RSM-full outperforms Streaming-Proto by +2.97 percentage points on the RealMem benchmark.
- The performance gain is primarily driven by the merge rule and grouped packing rather than just flat concatenation or simple clustering.
- The method reaches its optimal utility in the 2k to 5k prompt token range.
- RSM-full outperforms Streaming-Proto by +2.97 percentage points on the RealMem benchmark.
- The performance gain is primarily driven by the merge rule and grouped packing rather than just flat concatenation or simple clustering.
- The method reaches its optimal utility in the 2k to 5k prompt token range.
- **Inference** – Platforms and engines for running models, plus user interfaces.
- **Models** – LLMs (general, coding, multimodal, image, audio), model providers, and specific model highlights.
- **RAG** – Retrieval-Augmented Generation tools.
- **Safeguards** – Safety and content filtering.
- **Agents & Tools** – Agent frameworks, Model Context Protocol, coding agents, computer/browser automation, memory management, and testing/evaluation.
- **Research, Training & Fine-tuning** – Security, sandboxing, and model development.
- **Hardware** – Local hardware options.
- **Tutorials** – Guides covering models, prompt/context engineering, inference, agents, and RAG.
- **Communities** – Places to connect and share knowledge.
- **Models** – LLMs (general, coding, multimodal, image, audio), model providers, and specific model highlights.
- **RAG** – Retrieval-Augmented Generation tools.
- **Safeguards** – Safety and content filtering.
- **Agents & Tools** – Agent frameworks, Model Context Protocol, coding agents, computer/browser automation, memory management, and testing/evaluation.
- **Research, Training & Fine-tuning** – Security, sandboxing, and model development.
- **Hardware** – Local hardware options.
- **Tutorials** – Guides covering models, prompt/context engineering, inference, agents, and RAG.
- **Communities** – Places to connect and share knowledge.
Adam Conway writes that running the Qwen 3.6 27B large language model locally on the Tines 3B platform demonstrated that context window constraints, not model capability, were the primary bottleneck in vibe coding. Despite never encountering the platform's specific architecture or documentation, the model successfully constructed a multi-step web application that combined RSS feeds, correctly following novel platform conventions and autonomously debugging its own output. The author notes that while the model occasionally stalled or lost progress due to a 100,000-token limit that maxed out his GPU's VRAM, it ultimately reasoned its way through complex architectural flaws and timeout issues by iteratively testing and refactoring code, proving highly capable when paired with attentive human oversight.
- Tines 3B injects API credentials through an external proxy, ensuring they never touch the generated code or the model's context window.
- The experiment ran Qwen 3.6 27B on a local Radeon RX 7900 XTX via llama.cpp with multi-token prediction, yielding 40-50 tokens per second.
- Platform behavior was governed by a 4,586-word AGENTS.md rulebook defining Docker volume modes, routing syntax, and cron configurations.
- Context overflow forced manual session forks, causing the model to lose previously verified fixes and inadvertently overwrite functional cache data during timeout retries.
- Tines 3B injects API credentials through an external proxy, ensuring they never touch the generated code or the model's context window.
- The experiment ran Qwen 3.6 27B on a local Radeon RX 7900 XTX via llama.cpp with multi-token prediction, yielding 40-50 tokens per second.
- Platform behavior was governed by a 4,586-word AGENTS.md rulebook defining Docker volume modes, routing syntax, and cron configurations.
- Context overflow forced manual session forks, causing the model to lose previously verified fixes and inadvertently overwrite functional cache data during timeout retries.
> Lessons from building a fast, reliable scientific agent with local open-weight models, vLLM, and long-context infrastructure
**Redis Iris** is a context and memory platform designed for agentic pull architectures. It replaces static RAG with dynamic, live-synced data, semantic tool access, and session management to handle high-frequency AI agent requests at scale.
* Delivers petabyte-scale retrieval with sub-millisecond latency by optimizing costs (99% flash/SSD, 1% RAM).
* Auto-generates MCP tools via Pydantic models, enabling agents to query business data directly with row-level access controls.
* Uses CDC pipelines for continuous synchronization with sources like Snowflake, Databricks, and Postgres.
* Delivers petabyte-scale retrieval with sub-millisecond latency by optimizing costs (99% flash/SSD, 1% RAM).
* Auto-generates MCP tools via Pydantic models, enabling agents to query business data directly with row-level access controls.
* Uses CDC pipelines for continuous synchronization with sources like Snowflake, Databricks, and Postgres.
This article explores the critical architectural decision of where to store conversation history when building AI agents. It examines how different storage strategies impact user experience, privacy, cost, and portability. The author compares service-managed versus client-managed storage models and details how modern APIs support both linear threads and forking/branching capabilities.
Key topics include:
* Service-Managed vs. Client-Managed storage tradeoffs
* Linear (single-threaded) vs. Forking-capable conversation models
* Strategies for context window management and compaction such as truncation, summarization, and sliding windows
* How Microsoft Agent Framework abstracts these patterns using AgentSession and ChatHistoryProvider to ensure provider-agnostic code
* Practical implementation examples for the Responses API in different modes
Key topics include:
* Service-Managed vs. Client-Managed storage tradeoffs
* Linear (single-threaded) vs. Forking-capable conversation models
* Strategies for context window management and compaction such as truncation, summarization, and sliding windows
* How Microsoft Agent Framework abstracts these patterns using AgentSession and ChatHistoryProvider to ensure provider-agnostic code
* Practical implementation examples for the Responses API in different modes
A new ETH Zurich study challenges the common practice of using `AGENTS.md` files with AI coding agents. LLM-generated context files decrease performance (3% lower success rate, +20% steps/costs).Human-written files offer small gains (4% success rate) but also increase costs. Researchers recommend omitting context files unless manually written with non-inferable details (tooling, build commands).They tested this using a new dataset, AGENTbench, with four agents.
RAG combines language models with external knowledge. This article explores context & retrieval in RAG, covering search methods (keywords, TF-IDF, embeddings/FAISS/Chroma), context length challenges (compression, re-ranking), and contextual retrieval (query & conversation history).
This research introduces Doc-to-LoRA (D2L), a method for efficiently processing long documents with Large Language Models (LLMs). D2L creates small, adaptable "LoRA" modules that distill key information from a document, allowing the LLM to answer questions without needing the entire document in memory. This significantly reduces latency and memory usage, enabling LLMs to handle contexts much longer than their original capacity and facilitating faster knowledge updates.
Here’s the simplest version — key sentence extraction:
<pre>
```
def extract_relevant_sentences(document, query, top_k=5):
sentences = document.split('.')
query_embedding = embed(query)
scored = »
for sentence in sentences:
similarity = cosine_sim(query_embedding, embed(sentence))
scored.append((sentence, similarity))
scored.sort(key=lambda x: x 1 » , reverse=True)
return '. '.join( s[0 » for s in scored :top_k » ])
```
</pre>
For each sentence, compute similarity to the query. Keep the top 5. Discard the rest
<pre>
```
def extract_relevant_sentences(document, query, top_k=5):
sentences = document.split('.')
query_embedding = embed(query)
scored = »
for sentence in sentences:
similarity = cosine_sim(query_embedding, embed(sentence))
scored.append((sentence, similarity))
scored.sort(key=lambda x: x 1 » , reverse=True)
return '. '.join( s[0 » for s in scored :top_k » ])
```
</pre>
For each sentence, compute similarity to the query. Keep the top 5. Discard the rest
