The Bitter Lesson (2919) by Rich Sutton explores a recurring pattern in the history of AI research, arguing that general methods leveraging massive computation are ultimately more effective than those relying on human-encoded domain knowledge. While incorporating human intuition can provide short-term gains, long-term breakthroughs are consistently driven by scaling computational power through search and learning as described by Moore's Law.
Key observations include:
- The historical shift in chess, Go, speech recognition, and computer vision from rule-based or human-centric models toward massive computation.
- The tendency for researchers to favor methods that reflect human understanding, which often plateaus compared to scalable learning processes.
- The necessity of developing meta-methods capable of discovering complex patterns rather than hardcoding existing human perceptions into agents.
Key observations include:
- The historical shift in chess, Go, speech recognition, and computer vision from rule-based or human-centric models toward massive computation.
- The tendency for researchers to favor methods that reflect human understanding, which often plateaus compared to scalable learning processes.
- The necessity of developing meta-methods capable of discovering complex patterns rather than hardcoding existing human perceptions into agents.
An exploration into the history of conversational technology, tracing its roots from Joseph Weizenbaum's 1966 ELIZA experiment at MIT to modern large language models like ChatGPT and Claude. The article examines how the evolution from rule-based symbolic AI to probabilistic deep learning has changed human interaction with machines, often leading users to attribute human qualities to code. It specifically addresses the risks of "chatbot psychosis" and the danger of individuals relying on general-purpose generative models for mental health support when these systems are prone to hallucinations or reinforcing delusional beliefs.
* The transition from symbolic AI's explicit rules to modern deep learning
* Joseph Weizenbaum’s warning against humanizing machines via the ELIZA effect
* The psychological impact and risks of using large language models for emotional support
* The transition from symbolic AI's explicit rules to modern deep learning
* Joseph Weizenbaum’s warning against humanizing machines via the ELIZA effect
* The psychological impact and risks of using large language models for emotional support
Researchers demonstrate that recurrent cortical circuits generate traveling waves acting as a computational engine in the visual cortex. These neural traveling waves allow the brain to build internal representations of the external world, enabling the prediction of upcoming sensory inputs and the replay of temporal memories. This biological mechanism is functionally analogous to how large language models learn statistical structures from text; instead, these waves encode environmental regularities into synaptic networks through experience.
- Traveling waves modulate moment-to-moment perception.
- They enable the inference of causes for noisy sensory inputs.
- These dynamics allow for short-term predictions and memory replay.
- Traveling waves modulate moment-to-moment perception.
- They enable the inference of causes for noisy sensory inputs.
- These dynamics allow for short-term predictions and memory replay.
Thinking Machines has released Inkling, an open-weights Mixture-of-Experts transformer model featuring 975B total parameters and a context window of up to 1M tokens. The model was trained on 45 trillion tokens across text, images, audio, and video to enable native multimodal reasoning. It is designed with controllable thinking effort to optimize the balance between performance and cost/latency, alongside strong capabilities for agentic coding and tool use.
- Native multimodality in vision, audio, and text
- Controllable computational effort settings
- High proficiency in agentic workflows and design tasks
- Available on Tinker for custom fine-tuning
- Native multimodality in vision, audio, and text
- Controllable computational effort settings
- High proficiency in agentic workflows and design tasks
- Available on Tinker for custom fine-tuning
An overview of Kolmogorov-Arnold Networks (KAN) as an alternative paradigm to traditional neural network design. Unlike Multi-Layer Perceptrons (MLPs) that use fixed activation functions on nodes, KANs employ learnable univariate functions—specifically B-splines—on the edges between neurons. This approach allows for more accurate representations with fewer parameters and offers improved interpretability and potential for continual learning due to the local nature of spline basis functions.
* The mathematical distinction between Universal Approximation Theorem and Kolmogorov-Arnold Representation Theorem
* How B-splines enable learnable non-linear transformations on edges through control points
* Comparison with MLPs regarding parameter efficiency, performance, and training speed
* Advantages in interpretability and the mitigation of catastrophic forgetting via locality properties
* The mathematical distinction between Universal Approximation Theorem and Kolmogorov-Arnold Representation Theorem
* How B-splines enable learnable non-linear transformations on edges through control points
* Comparison with MLPs regarding parameter efficiency, performance, and training speed
* Advantages in interpretability and the mitigation of catastrophic forgetting via locality properties
Google's release of Gemma 4 marks a major turning point for open-source AI, offering a versatile family of multimodal models under a permissive Apache 2.0 license. Built using Gemini 3 technology, these models demonstrate massive leaps in math and coding performance, rivaling much larger proprietary systems while remaining efficient enough to run on local hardware ranging from smartphones to high-end GPUs. This release positions Google as a formidable competitor in the open-weights ecosystem, prioritizing user ownership and deployment efficiency.
* Apache 2.0 license
* Multimodal intelligence
* Local hardware deployment
* Massive benchmark leaps
* Efficient MoE architecture
**Models**
* E2B: Mobile efficiency
* E4B: Edge specialist
* 26B MoE: Speed meets intelligence
* 31B Dense: Top-tier performance
* Apache 2.0 license
* Multimodal intelligence
* Local hardware deployment
* Massive benchmark leaps
* Efficient MoE architecture
**Models**
* E2B: Mobile efficiency
* E4B: Edge specialist
* 26B MoE: Speed meets intelligence
* 31B Dense: Top-tier performance
This article explains how to distinguish agentic workflows from autonomous agents by focusing on ownership of control flow. It argues that the primary distinction is a spectrum between predictability and autonomy, rather than just whether a large language model is involved. The author breaks down four key stages of system design:
* Deterministic workflows where humans define fixed sequences in advance
* Orchestrated workflows where models choose from predefined branches
* Reactive agents that use reasoning to decide actions at runtime via patterns like ReAct
* Autonomous multi-agent systems where agents act as nested tools for one another
While fully autonomous systems offer high flexibility, the piece concludes that production environments typically favor hybrid architectures that combine structured workflows with targeted autonomy.
* Deterministic workflows where humans define fixed sequences in advance
* Orchestrated workflows where models choose from predefined branches
* Reactive agents that use reasoning to decide actions at runtime via patterns like ReAct
* Autonomous multi-agent systems where agents act as nested tools for one another
While fully autonomous systems offer high flexibility, the piece concludes that production environments typically favor hybrid architectures that combine structured workflows with targeted autonomy.
gSMILE is a model-agnostic framework designed to provide interpretability for large language models by explaining how specific parts of a prompt influence the generated output. The system functions by making minor variations to input prompts and measuring subsequent changes in responses to identify high-impact words, which are then presented as visual heat maps. This approach aims to demystify black-box systems like GPT, Llama, and Claude for use cases where trust and accountability are essential.
- Model-agnostic interpretability specifically for generative AI solutions.
- Identification of influential tokens through input perturbation.
- Visualization of prompt significance via heat maps.
- Empirical validation using accuracy, consistency, stability, and fidelity metrics.
- Model-agnostic interpretability specifically for generative AI solutions.
- Identification of influential tokens through input perturbation.
- Visualization of prompt significance via heat maps.
- Empirical validation using accuracy, consistency, stability, and fidelity metrics.
Google DeepMind has released the Gemma 4 12B, a dense multimodal model featuring an encoder-free architecture. Unlike previous iterations that used separate vision and audio encoders, this model allows these modalities to flow directly into the LLM backbone. This streamlined design reduces latency and memory overhead, allowing the model to perform agentic reasoning tasks on consumer laptops with as little as 16 GB of VRAM while approaching the performance levels of much larger models like the 26B MoE variant.
- Unified decoder-only architecture for text, image, video, and native audio input.
- Encoder-free design using a 35M vision embedder and direct raw audio wave projection.
- Optimized to run locally on Apple Silicon Macs and consumer GPU laptops.
- Released under an Apache 2.0 license with support for llama.cpp, MLX, vLLM, and Ollama.
- Unified decoder-only architecture for text, image, video, and native audio input.
- Encoder-free design using a 35M vision embedder and direct raw audio wave projection.
- Optimized to run locally on Apple Silicon Macs and consumer GPU laptops.
- Released under an Apache 2.0 license with support for llama.cpp, MLX, vLLM, and Ollama.
This article explores the feasibility of running Large Language Models (LLMs) locally using only a CPU, challenging the assumption that expensive GPUs are strictly necessary. By testing eight different models on an older Intel i5 laptop with 12GB of RAM via Ollama, the author identifies which models offer practical usability for everyday tasks.
Key points include:
- Using tokens per second as a more critical metric for usability than model size or RAM usage alone.
- Why 1B to 2B parameter models provide the best balance of responsiveness and reasoning on low-end hardware.
- The effectiveness of GGUF quantization (specifically Q4_K_M) in reducing resource demands.
- A comparison of various model tiers, from ultra-fast tiny models like Qwen 0.6B to slower, high-capability models like Ministral 3 8B.
Key points include:
- Using tokens per second as a more critical metric for usability than model size or RAM usage alone.
- Why 1B to 2B parameter models provide the best balance of responsiveness and reasoning on low-end hardware.
- The effectiveness of GGUF quantization (specifically Q4_K_M) in reducing resource demands.
- A comparison of various model tiers, from ultra-fast tiny models like Qwen 0.6B to slower, high-capability models like Ministral 3 8B.
