This article explores the feasibility of running Large Language Models (LLMs) locally using only a CPU, challenging the assumption that expensive GPUs are strictly necessary. By testing eight different models on an older Intel i5 laptop with 12GB of RAM via Ollama, the author identifies which models offer practical usability for everyday tasks.
Key points include:
- Using tokens per second as a more critical metric for usability than model size or RAM usage alone.
- Why 1B to 2B parameter models provide the best balance of responsiveness and reasoning on low-end hardware.
- The effectiveness of GGUF quantization (specifically Q4_K_M) in reducing resource demands.
- A comparison of various model tiers, from ultra-fast tiny models like Qwen 0.6B to slower, high-capability models like Ministral 3 8B.
Key points include:
- Using tokens per second as a more critical metric for usability than model size or RAM usage alone.
- Why 1B to 2B parameter models provide the best balance of responsiveness and reasoning on low-end hardware.
- The effectiveness of GGUF quantization (specifically Q4_K_M) in reducing resource demands.
- A comparison of various model tiers, from ultra-fast tiny models like Qwen 0.6B to slower, high-capability models like Ministral 3 8B.
AMD CEO Dr. Lisa Su addressed concerns that the rise of agentic AI might cannibalize the GPU market, arguing instead that the demand is largely additive. While GPUs are essential for running foundational models, CPUs play a critical role in orchestration, data movement, and parallel execution required by autonomous agents. This shift could fundamentally change industry-standard CPU-to-GPU ratios, potentially moving from traditional 1:8 configurations toward a more balanced 1:1 ratio as agentic workloads expand.
AMD now supports Google’s Gemma 4 models (2B–31B parameters) across its entire hardware lineup, including Instinct GPUs (datacenters), Radeon GPUs (workstations), and Ryzen AI processors (PCs). The integration is compatible with vLLM, SGLang, llama.cpp, Ollama, and Lemonade Server, aiming to optimize AI performance for both cloud and local deployment.
A deep dive into the clever barrel shifter design used in the Intel 80386 processor, explaining how it efficiently performs bitwise shifts and rotations. The article details the logic behind the shifter, its construction using multiplexers, and how it contributes to the 80386's performance.
M5 delivers over 4x the peak GPU compute performance for AI compared to M4, featuring a next-generation GPU with a Neural Accelerator in each core, a more powerful CPU, a faster Neural Engine, and higher unified memory bandwidth.
A 120 billion parameter OpenAI model can now run on consumer hardware thanks to the Mixture of Experts (MoE) technique, which significantly reduces memory requirements and allows processing on CPUs while offloading key parts to modest GPUs.
The article shows how to check if a Linux CPU supports AES‑NI, Intel’s hardware‑accelerated AES instruction set. It explains what AES‑NI is, why it speeds up encryption, and then lists three easy methods: use cpuid and grep for “aes”, grep the /proc/cpuinfo file, or run lscpu and look for the “aes” flag. If none of these commands report AES‑NI, the CPU relies on slower software encryption, which is still secure. The first CPUs to expose this feature were Intel’s Westmere chips in 2010. In the CPUID specification the flag is simply called AES (bit 25 of ECX). The “NI” (New Instructions) part is just a marketing name for the feature set. There isn’t a distinct “aes_ni” bit in the CPUID leaf. So, when you run <tt>lscpu | grep -i aes or cat /proc/cpuinfo | grep aes</tt>, the presence of aes tells you that the CPU supports AES‑NI. There is no separate aes_ni flag because the kernel already uses the more concise aes.
LocalScore is an open benchmark to evaluate local AI task performance across various hardware configurations, measuring Prompt Processing speed, Token Generation speed, Time-to-First-Token (TTFT), and a combined LocalScore.
NVIDIA DGX Spark is a desktop-friendly AI supercomputer powered by the NVIDIA GB10 Grace Blackwell Superchip, delivering 1000 AI TOPS of performance with 128GB of memory. It is designed for prototyping, fine-tuning, and inference of large AI models.
6502.sh is a 6502 emulator and debugger written in busybox ash compliant shell script, featuring 32k RAM, 16k ROM, an interactive debugger, and STDIO directed to an ACIA compatible serial port.
