This repository provides access to Apple's built-in large language models via Node.js and Python packages, requiring only macOS 26+ on Apple Silicon and the Xcode Command Line Tools. It offers two tiers of interaction: an on-device tier using a small sparse model (AFM 3 Core Advanced) that ensures privacy and guaranteed JSON output through constrained decoding, and a cloud tier via Apple's Private Cloud Compute which provides more powerful reasoning capabilities but sends prompts off the machine.
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
Fast and easy LLM serving for the mac.
On Linux and macOS you can find the user base binary directory by running python -m site --user-base and adding bin to the end. F
You can fix the problem on Mac OS X Mavericks and Yosemite by doing the following:
cd ~/.matplotlib/
fc-list # Should take a couple minutes.
After fc-list finishes running you will be able to import the package using import matplotlib.pyplot as plt.
cd ~/.matplotlib/
fc-list # Should take a couple minutes.
After fc-list finishes running you will be able to import the package using import matplotlib.pyplot as plt.
