This repository provides access to Apple's built-in large language models via Node.js and Python packages, requiring only macOS 26+ on Apple Silicon and the Xcode Command Line Tools. It offers two tiers of interaction: an on-device tier using a small sparse model (AFM 3 Core Advanced) that ensures privacy and guaranteed JSON output through constrained decoding, and a cloud tier via Apple's Private Cloud Compute which provides more powerful reasoning capabilities but sends prompts off the machine.
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
- The library includes both npm (`apple-llm`) and PyPI (`apple-llm`) packages.
- On-device models are noted to be poor at code generation and long reasoning tasks.
- Structured JSON output is guaranteed on-device via a specific "GenerationSchema" dialect used by Apple's decoder.
- The cloud tier uses Shortcuts as an intermediary because the direct Private Cloud Compute API requires restricted developer entitlements.
- Users can utilize built-in tools like OCR, barcode reading, and Spotlight semantic search for local RAG (macOS 27+).
Strands Agents Tools is a community-driven Python package that hands LLM-based agents a ready-made set of capabilities—file operations, shell integration, web search, Python execution, persistent memory, and multi-agent coordination—so developers building on the Strands Agents SDK don't have to write each integration from scratch.
- Memory backends include Mem0, Amazon Bedrock Knowledge Bases, Elasticsearch, and MongoDB Atlas
- Multi-agent primitives (swarm intelligence, agent-as-tool with model switching, multi-agent graphs) live in the same package as basic file tools, reducing glue code
- Python execution requires user confirmation as a first-class safety measure
- Modular design: pull in only the tools you need without dragging in video processing, cron scheduling, or Slack
- Memory backends include Mem0, Amazon Bedrock Knowledge Bases, Elasticsearch, and MongoDB Atlas
- Multi-agent primitives (swarm intelligence, agent-as-tool with model switching, multi-agent graphs) live in the same package as basic file tools, reducing glue code
- Python execution requires user confirmation as a first-class safety measure
- Modular design: pull in only the tools you need without dragging in video processing, cron scheduling, or Slack
@githubprojects writes about Quivr-core, a Python package extracted from Quivr.com's production retrieval-augmented generation pipeline that lets developers get a working system in five lines of code. It centers on a `Brain` class that ingests files and answers questions, with a YAML-configurable workflow (filter history → rewrite → retrieve → generate) that keeps the pipeline inspectable rather than a black box. It supports OpenAI, Anthropic, Mistral, and local Ollama models, and handles PDFs, Markdown, and TXT files out of the box.
- Integrates with Megaparse for more sophisticated document ingestion without switching frameworks.
- The opinionated philosophy is the core differentiator: sensible defaults over an infinite configuration surface.
- Integrates with Megaparse for more sophisticated document ingestion without switching frameworks.
- The opinionated philosophy is the core differentiator: sensible defaults over an infinite configuration surface.
paoloap wrote companion code for a Medium article on agent harnesses, including five layers: execution boundary, sandboxing, memory persistence, verification loops, and context pipelines, each with failure and guard scripts running alongside.
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
Anurag Singh replaced five Python scripts (backup, organizer, renamer, cleaner, watchdog) with a local LLM agent, which made errors the scripts didn't (wrong directories, skipped steps, false success reports).Each of the original scripts followed explicit rules through a scheduler; the agent instead added a longer inference chain (inspect, interpret, choose a tool, build a command, execute, review) to tasks that fixed logic already described completely, while also holding a loaded model in memory between runs.
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
Abid Ali Awan writes that a Jupyter Notebook pipeline can turn a webpage into a lightweight LLM-powered QA engine by fetching HTML with requests, stripping noisy elements with BeautifulSoup, converting the cleaned DOM to Markdown with markdownify and ftfy, then asking an OpenAI model to answer a specific user query using only the compact Markdown, which reduces token use by removing navigation, scripts and repeated marketing text.
- Uses gpt-5.4-nano for cost-efficient answers
- Removes script, style, nav, header, footer, form, button tags and class/id names containing popup, cookie, navbar, modal, etc.
- Demonstrates queries on olostep.com home and pricing pages and saves output to ai_scraper_result.md
- Notes running costs and cites commercial alternatives such as Olostep, Firecrawl and Exa
- Uses gpt-5.4-nano for cost-efficient answers
- Removes script, style, nav, header, footer, form, button tags and class/id names containing popup, cookie, navbar, modal, etc.
- Demonstrates queries on olostep.com home and pricing pages and saves output to ai_scraper_result.md
- Notes running costs and cites commercial alternatives such as Olostep, Firecrawl and Exa
The NOOA framework provides a way to build LLM agents using standard Pythonic object-oriented patterns. By treating agents as objects, developers can map state to typed fields and capabilities to methods where docstrings serve as prompts; specifically, an ellipsis in a method body triggers the runtime for an LLM-driven execution loop.
- Includes separate packages for CLI tools, memory management, and benchmarking.
- Supports various local and hosted models via LiteLLM integration.
- Offers automated tracing with an interactive web viewer for debugging.
- Necessitates OS-level isolation to safely execute LLM-generated code.
- Includes separate packages for CLI tools, memory management, and benchmarking.
- Supports various local and hosted models via LiteLLM integration.
- Offers automated tracing with an interactive web viewer for debugging.
- Necessitates OS-level isolation to safely execute LLM-generated code.
Asif Razzaq writes that NVIDIA Labs has open-sourced NOOA, a model-agnostic Python framework designed to streamline agentic development by consolidating prompt templates, tool schemas, and state into single class structures. By treating LLM-driven actions as standard methods with docstrings serving as prompts, the framework allows developers to build autonomous workflows that can be tested, traced, and version-controlled like ordinary software.
- Achieves 82.2% on SWE-bench Verified while using roughly half the tokens required by existing open harnesses.
- Employs a "pass by reference" mechanism for live Python objects via bounded previews to conserve context window space.
- Features an optional memory subsystem that utilizes SQLite and ACT-R activation ranking for record retrieval.
- Achieves 82.2% on SWE-bench Verified while using roughly half the tokens required by existing open harnesses.
- Employs a "pass by reference" mechanism for live Python objects via bounded previews to conserve context window space.
- Features an optional memory subsystem that utilizes SQLite and ACT-R activation ranking for record retrieval.
PandasAI is a Python library that allows users to query datasets using natural language. By leveraging large language models (LLMs), it assists both technical and non-technical individuals in performing data analysis, executing complex queries, and creating visualizations through simple conversation.
- Cross-dataframe query support
- Secure Docker sandbox option
- Multiple LLM provider compatibility via LiteLLm
- Cross-dataframe query support
- Secure Docker sandbox option
- Multiple LLM provider compatibility via LiteLLm
Emmimal P Alexander writes that while prompt engineering focuses on optimizing LLM inputs, managing these templates within evolving codebases often leads to production crashes when variables are renamed or removed. To solve this, she created `promptctl`, a Python tool that applies static analysis—similar to database schema migrations—to ensure prompt variable contracts match their call sites in the codebase.
- Performs PromptDiff (detects changes), Contract Validation (checks mismatches), and Impact Analysis (traces dependencies).
- Operates strictly via AST parsing, requiring zero LLM calls or API keys.
- Detects errors that unit tests often miss by mocking away the actual string formatting step.
- Performs PromptDiff (detects changes), Contract Validation (checks mismatches), and Impact Analysis (traces dependencies).
- Operates strictly via AST parsing, requiring zero LLM calls or API keys.
- Detects errors that unit tests often miss by mocking away the actual string formatting step.
