The article discusses the emergence of 'agentic traffic' – outbound API calls made by autonomous AI agents – and the need for a new infrastructure layer, an 'AI Gateway', to govern and secure this traffic. It outlines the components of an AI Gateway and the importance of security, compliance, and observability in managing agentic AI.
Solo.io donated Kagent, its open source framework for AI agents in Kubernetes, to the CNCF, and introduced MCP Gateway. They also unveiled automated zero-downtime migration and cost-analysis tools for Ambient Mesh.
Portkey AI Gateway allows application developers to easily integrate generative AI models, seamlessly switch among models, and add features like conditional routing without changing application code.
The author discusses the development of a function calling large language model (LLM) that significantly improves latency for agentic applications. This LLM matches or even exceeds the performance of other frontier LLMs. It is integrated into an open-source intelligent gateway for agentic applications, allowing developers to focus on more differentiated aspects of their projects. The model and the gateway are available on Hugging Face and GitHub, respectively.
