This repository provides access to Apple's built-in large language models via Node.js and Python packages, requiring only macOS 26+ on Apple Silicon and the Xcode Command Line Tools. It offers two tiers of interaction: an on-device tier using a small sparse model (AFM 3 Core Advanced) that ensures privacy and guaranteed JSON output through constrained decoding, and a cloud tier via Apple's Private Cloud Compute which provides more powerful reasoning capabilities but sends prompts off the machine.
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
Simon Willison explores "vibe coding" - building macOS apps with SwiftUI using large language models like Claude Opus 4.6 and GPT-5.4, without extensive coding knowledge. He successfully created two apps, Bandwidther (network bandwidth monitor) and Gpuer (GPU usage monitor), demonstrating the potential of this approach. The process involved minimal prompting and iterative development, leveraging the LLMs' capabilities for both code generation and feature suggestions.
While acknowledging the need for caution regarding the apps' accuracy, Willison highlights the efficiency and accessibility of building macOS applications in this manner.
While acknowledging the need for caution regarding the apps' accuracy, Willison highlights the efficiency and accessibility of building macOS applications in this manner.
LlamaBarn is a macOS menu bar app for running local LLMs. It provides a simple way to install and run models locally, connecting to apps via an OpenAI-compatible API.
Fast and easy LLM serving for the mac.
LM Studio has released lms, a command-line interface (CLI) tool to load/unload models, start/stop the API server, and inspect raw LLM input. It is developed on GitHub and is MIT Licensed.
Hallux.ai is a platform offering open-source, LLM-based CLI tools for Linux and MacOS. These tools aim to streamline operations, enhance productivity, and automate workflows for professionals in production engineering, SRE, and DevOps. They also improve Root Cause Analysis (RCA) capabilities and enable self-sufficiency.
An optional component to enable AI features in iTerm2, providing network request functionality, ensuring secure data transmission.
Ultrainbox is a unified inbox email client (currently for mac) that connects all your email accounts in one inbox and helps automate management of this inbox by delegating organisation, tasks and actions to GPT.
