This repository provides access to Apple's built-in large language models via Node.js and Python packages, requiring only macOS 26+ on Apple Silicon and the Xcode Command Line Tools. It offers two tiers of interaction: an on-device tier using a small sparse model (AFM 3 Core Advanced) that ensures privacy and guaranteed JSON output through constrained decoding, and a cloud tier via Apple's Private Cloud Compute which provides more powerful reasoning capabilities but sends prompts off the machine.
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
