This demo showcases how the Arch can be used to build an HR agent to manage workforce-related inquiries, workforce planning, and communication via Slack. It intelligently routes incoming prompts to the correct targets, providing concise and useful responses tailored for HR and workforce decision-making.
Katanemo Labs introduces Arch-Router, a 1.5B parameter model that intelligently maps user queries to the most suitable LLM, achieving 93% accuracy without the need for costly retraining. It uses a preference-aligned routing framework based on a Domain-Action Taxonomy, allowing for flexible adaptation to evolving models and use cases.
This paper introduces Arch-Router, a preference-aligned routing framework for large language models (LLMs). It addresses limitations in existing routing approaches by focusing on matching queries to user-defined preferences (domain and action types) rather than solely relying on benchmark performance. The framework includes a 1.5B parameter model, Arch-Router, and a data creation pipeline. Experiments demonstrate state-of-the-art results in matching queries with human preferences and improved adaptability.
Arch is an intelligent gateway for agents, designed to securely handle prompts, integrate with APIs, and provide rich observability, built on Envoy Proxy.
The ArchGW project focuses on simplifying the development of **agentic applications** – applications powered by Large Language Models (LLMs) that can perform actions and interact with tools. Here's a breakdown of the use cases and examples highlighted:
**Core Use Cases:**
* **Routing:** Intelligent routing of prompts to the correct agents or tools.
* **Tools Use:** Simplifying the integration of prompts with tools/APIs for common tasks.
* **Guardrails:** Centralized configuration for safety and preventing harmful outcomes.
* **LLM Access:** Centralized access and management of LLMs with retries for reliability.
* **Observability:** Providing W3C-compatible tracing and metrics for monitoring LLM interactions.
**Specific Examples & Demos:**
* **Weather Forecast Agent:** A sample application demonstrating core function calling capabilities.
* **Network Operator Agent:** An agent that can interact with network devices (retrieve statistics, reboot).
* **Connecting to SaaS APIs:** Demonstrates integrating 3rd party SaaS APIs into agentic chat experiences.
* **LLM Router:** Using Arch as a gateway to route requests to different LLMs (GPT-4o, Mistral) based on configuration or headers. The example shows how to switch between LLMs using the `x-arch-llm-provider-hint` header.
* **Currency Exchange Agent:** A quickstart guide builds an agent that fetches currency exchange rates from an API (Frankfurter.app). This demonstrates setting up configuration files, starting the gateway, and interacting with the agent via curl.
**Overall, ArchGW aims to address common challenges in building agentic apps:**
* Managing complex routing logic.
* Integrating with various LLMs and tools.
* Ensuring safety and reliability.
* Providing observability into LLM interactions.
The ArchGW project focuses on simplifying the development of **agentic applications** – applications powered by Large Language Models (LLMs) that can perform actions and interact with tools. Here's a breakdown of the use cases and examples highlighted:
**Core Use Cases:**
* **Routing:** Intelligent routing of prompts to the correct agents or tools.
* **Tools Use:** Simplifying the integration of prompts with tools/APIs for common tasks.
* **Guardrails:** Centralized configuration for safety and preventing harmful outcomes.
* **LLM Access:** Centralized access and management of LLMs with retries for reliability.
* **Observability:** Providing W3C-compatible tracing and metrics for monitoring LLM interactions.
**Specific Examples & Demos:**
* **Weather Forecast Agent:** A sample application demonstrating core function calling capabilities.
* **Network Operator Agent:** An agent that can interact with network devices (retrieve statistics, reboot).
* **Connecting to SaaS APIs:** Demonstrates integrating 3rd party SaaS APIs into agentic chat experiences.
* **LLM Router:** Using Arch as a gateway to route requests to different LLMs (GPT-4o, Mistral) based on configuration or headers. The example shows how to switch between LLMs using the `x-arch-llm-provider-hint` header.
* **Currency Exchange Agent:** A quickstart guide builds an agent that fetches currency exchange rates from an API (Frankfurter.app). This demonstrates setting up configuration files, starting the gateway, and interacting with the agent via curl.
**Overall, ArchGW aims to address common challenges in building agentic apps:**
* Managing complex routing logic.
* Integrating with various LLMs and tools.
* Ensuring safety and reliability.
* Providing observability into LLM interactions.
