Articles on logging, tracing, and observability including Echopraxia, Blindsight, structured log analysis, and more.
This article details the Model Context Protocol (MCP), a new approach to integrating Large Language Models (LLMs) like Azure OpenAI with tools. MCP focuses on structured data exchange to improve reliability, observability, and functionality, moving beyond simple text-in, text-out interactions. It aims to standardize how LLMs interact with tools, enhancing their ability to utilize those tools effectively.
This Splunk Lantern article outlines the steps to monitor Gen AI applications with Splunk Observability Cloud, covering setup with OpenTelemetry, NVIDIA GPU metrics, Python instrumentation, and OpenLIT integration to monitor GenAI applications built with technologies like Python, LLMs (OpenAI's GPT-4o, Anthropic's Claude 3.5 Haiku, Meta’s Llama), NVIDIA GPUs, Langchain, and vector databases (Pinecone, Chroma) using Splunk Observability Cloud. It outlines a six-step process:
- Access Splunk Observability Cloud: Sign up for a free trial if needed.
- Deploy Splunk Distribution of OpenTelemetry Collector: Use a Helm chart to install the collector in Kubernetes.
- Capture NVIDIA GPU Metrics: Utilize the NVIDIA GPU Operator and Prometheus receiver in the OpenTelemetry Collector.
- Instrument Python Applications: Use the Splunk Distribution of OpenTelemetry Python agent for automatic instrumentation and enable Always On Profiling.
- Enhance with OpenLIT: Install and initialize OpenLIT to capture detailed trace data, including LLM calls and interactions with vector databases (with options to disable PII capture).
- Start Using the Data: Leverage the collected metrics and traces, including features like Tag Spotlight, to identify and resolve performance issues (example given: OpenAI rate limits).
The article emphasizes OpenTelemetry's role in GenAI observability and highlights how Splunk Observability Cloud facilitates monitoring these complex applications, providing insights into performance, cost, and potential bottlenecks. It also points to resources for help and further information on specific aspects of the process.
The article discusses the challenges faced due to differing observability tools and naming conventions, and how OpenTelemetry's standard naming schemas can streamline workflows and enhance interoperability.
Grafana Loki version 3.4 introduces enhancements such as standardized storage with Thanos, a sizing guidance page, merging of Promtail into Grafana Alloy, and support for out-of-order logs.
This skill path by Bryce Yu guides users through the basics of managing databases on Kubernetes using KubeBlocks. It covers installation, deployment, upgrades, backup, observability, and auto-tuning of database clusters.
Sawmills AI has introduced a smart telemetry data management platform aimed at reducing costs and improving data quality for enterprise observability. By acting as a middleware layer that uses AI and ML to optimize telemetry data before it reaches vendors like Datadog and Splunk, Sawmills helps companies manage data efficiently, retain data sovereignty, and reduce unnecessary data processing costs.
OpenInference is a set of conventions and plugins that complements OpenTelemetry to enable tracing of AI applications, with native support from arize-phoenix and compatibility with other OpenTelemetry-compatible backends.
Arize Phoenix is an open-source observability library for AI experimentation, evaluation, and troubleshooting, built by Arize AI.
