An exploration of high-performing small language models with under 7 billion parameters that can run locally on consumer hardware like laptops and smartphones. The article explains how advancements in training data quality, model distillation from larger frontier models, and architectural improvements like Mixture-of-Experts have enabled these compact models to compete with much larger versions on reasoning benchmarks. It provides a curated guide of top available models on Hugging Face, detailing their specific strengths, benchmark performance, and providing Python code for implementation.
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
The open-source AI landscape is rapidly evolving, and recent developments surrounding GGML and Llama.cpp are significant for those interested in running large language models locally. GGML, a C library for machine learning, has joined Hugging Face, ensuring its continued development and accessibility. Meanwhile, Llama.cpp, a project focused on running Llama models on CPUs, remains open-source and is finding a stable home. This article details these changes, the implications for local AI enthusiasts, and the benefits of an open ecosystem.
OpenAI releases gpt-oss-safeguard, an open-source AI model for content moderation that allows developers to define their own safety policies instead of relying on pre-trained models. It operates by reasoning about content based on custom policies, offering a more flexible and nuanced approach to moderation.
This article details five major updates to Gradio MCP servers, including seamless local file support, real-time progress notifications, OpenAPI spec to MCP conversion, improvements to authentication, and modifying tool descriptions. It highlights how these improvements enhance the development and hosting of AI-powered MCP servers on Hugging Face Spaces.
Leveraging MCP for automating your daily routine. This article explores the Model Context Protocol (MCP) and demonstrates how to build a toolkit for analysts using it, including creating a local MCP server with useful tools and integrating it with AI tools like Claude Desktop.
This course provides an introduction to the Model Context Protocol (MCP), covering its theory, design, and practical application. It includes foundational units, hands-on exercises, use case assignments, and collaboration opportunities. The course aims to equip students with the knowledge and skills to build AI applications leveraging external data and tools using MCP standards.
A library for working with prompt templates locally or on the Hugging Face Hub. It aims to provide a standardized way of sharing and using prompt templates, with a focus on interoperability and modularity.
This article details the creation of a simple, 50-line agent using Model Context Protocol (MCP) and Hugging Face's tools, demonstrating how easily agents can be built with modern LLMs that support function/tool calling.
1. **MCP Overview**: MCP is a standard API for exposing tools that can be integrated with Large Language Models (LLMs).
2. **Implementation**: The author explains how to implement a MCP client using TypeScript and the Hugging Face Inference Client. This client connects to MCP servers, retrieves tools, and integrates them into LLM inference.
3. **Tools**: Tools are defined with a name, description, and parameters, and are passed to the LLM for function calling.
4. **Agent Design**: An agent is essentially a while loop that alternates between tool calling and feeding tool results back into the LLM until a specific condition is met, such as two consecutive non-tool messages.
5. **Code Example**: The article provides a concise 50-line TypeScript implementation of an agent, demonstrating the simplicity and power of MCP.
6. **Future Directions**: The author suggests experimenting with different models and inference providers, as well as integrating local LLMs using frameworks like llama.cpp or LM Studio.
1. **MCP Overview**: MCP is a standard API for exposing tools that can be integrated with Large Language Models (LLMs).
2. **Implementation**: The author explains how to implement a MCP client using TypeScript and the Hugging Face Inference Client. This client connects to MCP servers, retrieves tools, and integrates them into LLM inference.
3. **Tools**: Tools are defined with a name, description, and parameters, and are passed to the LLM for function calling.
4. **Agent Design**: An agent is essentially a while loop that alternates between tool calling and feeding tool results back into the LLM until a specific condition is met, such as two consecutive non-tool messages.
5. **Code Example**: The article provides a concise 50-line TypeScript implementation of an agent, demonstrating the simplicity and power of MCP.
6. **Future Directions**: The author suggests experimenting with different models and inference providers, as well as integrating local LLMs using frameworks like llama.cpp or LM Studio.
Google releases Gemma 3, a new iteration of their Gemma family of models. It ranges from 1B to 27B parameters, supports up to 128k tokens, accepts images and text, and supports 140+ languages. This article details its technical enhancements (longer context, multimodality, multilinguality) and provides information on inference with Hugging Face transformers, on-device deployment, and evaluation.
This Space demonstrates a simple method for embedding text using a LLM (Large Language Model) via the Hugging Face Inference API. It showcases how to convert text into numerical vector representations, useful for semantic search and similarity comparisons.
