Anurag Singh replaced five Python scripts (backup, organizer, renamer, cleaner, watchdog) with a local LLM agent, which made errors the scripts didn't (wrong directories, skipped steps, false success reports).Each of the original scripts followed explicit rules through a scheduler; the agent instead added a longer inference chain (inspect, interpret, choose a tool, build a command, execute, review) to tasks that fixed logic already described completely, while also holding a loaded model in memory between runs.
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
Anurag Singh replaced his home lab cron scripts with Qwen3.5 9B using an agent harness with shell access. He expected contextual reasoning to be superior to rigid automation. The local model succeeded in identifying ballooned directories or judging if a container restart was needed, but it failed more often, sometimes stalling or silently skipping checks.He concluded that deterministic scripts remain the more dependable choice for routine tasks and pointed to n8n as a sensible middle ground when the friction is writing and maintaining code rather than the logic itself.
- A 9-billion-parameter local model needs several GB of RAM just to load weights, which is painful on a home server already running Docker, DNS, and other services.
- Singh's specific hardware ceiling: roughly 14B parameters on a 16 GB MacBook, maybe 32B on an M5 Pro, beyond which you need a dedicated rig.
- His suggested hybrid: let the local model read an error log and draft a short explanation, then have n8n relay that summary without granting the model permission to restart or modify anything.
The model's failure mode was not wrong commands but an inconsistent process—the same prompt and the same system state, yet different execution paths on successive runs.
- A 9-billion-parameter local model needs several GB of RAM just to load weights, which is painful on a home server already running Docker, DNS, and other services.
- Singh's specific hardware ceiling: roughly 14B parameters on a 16 GB MacBook, maybe 32B on an M5 Pro, beyond which you need a dedicated rig.
- His suggested hybrid: let the local model read an error log and draft a short explanation, then have n8n relay that summary without granting the model permission to restart or modify anything.
The model's failure mode was not wrong commands but an inconsistent process—the same prompt and the same system state, yet different execution paths on successive runs.
Splunk .conf in Denver Sept 2026: The core theme focuses on "Agentic AI"—the transition toward autonomous workflows in security, observability, and operations to manage increasing data complexity and digital risk.
**Sessions**
* **Powering the Rise of the Agentic Enterprise:** Transforming security/ops with scalable, trust-based AI workflows.
* **See Inside Your Agentic AI Applications:** Tracing Python agent interactions to detect hallucinations and tool failures.
* **The Future is Here: What’s New in Splunk Security:** Transitioning to an "Agentic SOC" via automated orchestration.
* **The Platform for Your Agentic Enterprise:** Unlocking cross-domain insights through unified data and AI.
* **What's New in Platform:** Using a unified data fabric for predictive, agentic operations.
* **5 ML Algorithms in Under 5 Lines of Code:** Reducing alert fatigue using simple SPL/MLTK implementations.
* **Accelerate Digital Resilience (Cisco & Splunk):** Maximizing ROI through integrated ecosystem use cases.
* **AI Troubleshooting and Remediation Agents:** Using AI SRE agents to accelerate root cause analysis.
* **Accelerate SmartStore Searches with VAST Data:** Reducing storage costs via InfiniteCache integration.
* **The Cisco & Splunk Partner Playbook:** Strategies for scaling business in the AI era.
* **Accelerating Your Journey to Agentic Observability:** Navigating 4 steps toward agent-driven monitoring.
* **A Compromised Account Alerting System:** Automating detection of suspicious user behavior.
* **Advanced Threat Detection (DSDL & RBA):** Using machine learning for behavioral risk scoring.
* **Advancing Security Operations with ESCU Innovations:** Leveraging new high-fidelity threat intelligence content.
* **Agentic Data Ingestion (Anthropic's Claude):** Automating Splunk Add-On development using GenAI.
* **Agentic Detection Posture Management:** Using AI to rapidly identify and close detection gaps.
* **Agentic Incident Command:** Deploying coordinated AI agent teams for full incident lifecycles.
* **Hands-on Agent Attack Analysis:** Building anomaly detections from raw agent telemetry.
* **Build Your Own SOC AI Agent (Workshop):** Hands-on guide to building custom agents with guardrails.
* **Triage and Response at Scale:** Using Triage and SOP Agents for automated incident response.
* **Agents Are Easy, Trust Is Hard:** Implementing governance in cohesive agentic operations via Cisco Data Fabric.
* **AI Assistant V2 Unlocked (Workshop):** Hands-on with Agent Mode and organizational knowledge integration.
**Sessions**
* **Powering the Rise of the Agentic Enterprise:** Transforming security/ops with scalable, trust-based AI workflows.
* **See Inside Your Agentic AI Applications:** Tracing Python agent interactions to detect hallucinations and tool failures.
* **The Future is Here: What’s New in Splunk Security:** Transitioning to an "Agentic SOC" via automated orchestration.
* **The Platform for Your Agentic Enterprise:** Unlocking cross-domain insights through unified data and AI.
* **What's New in Platform:** Using a unified data fabric for predictive, agentic operations.
* **5 ML Algorithms in Under 5 Lines of Code:** Reducing alert fatigue using simple SPL/MLTK implementations.
* **Accelerate Digital Resilience (Cisco & Splunk):** Maximizing ROI through integrated ecosystem use cases.
* **AI Troubleshooting and Remediation Agents:** Using AI SRE agents to accelerate root cause analysis.
* **Accelerate SmartStore Searches with VAST Data:** Reducing storage costs via InfiniteCache integration.
* **The Cisco & Splunk Partner Playbook:** Strategies for scaling business in the AI era.
* **Accelerating Your Journey to Agentic Observability:** Navigating 4 steps toward agent-driven monitoring.
* **A Compromised Account Alerting System:** Automating detection of suspicious user behavior.
* **Advanced Threat Detection (DSDL & RBA):** Using machine learning for behavioral risk scoring.
* **Advancing Security Operations with ESCU Innovations:** Leveraging new high-fidelity threat intelligence content.
* **Agentic Data Ingestion (Anthropic's Claude):** Automating Splunk Add-On development using GenAI.
* **Agentic Detection Posture Management:** Using AI to rapidly identify and close detection gaps.
* **Agentic Incident Command:** Deploying coordinated AI agent teams for full incident lifecycles.
* **Hands-on Agent Attack Analysis:** Building anomaly detections from raw agent telemetry.
* **Build Your Own SOC AI Agent (Workshop):** Hands-on guide to building custom agents with guardrails.
* **Triage and Response at Scale:** Using Triage and SOP Agents for automated incident response.
* **Agents Are Easy, Trust Is Hard:** Implementing governance in cohesive agentic operations via Cisco Data Fabric.
* **AI Assistant V2 Unlocked (Workshop):** Hands-on with Agent Mode and organizational knowledge integration.
As AI agents evolve from autocomplete tools to active contributors (opening PRs, managing infrastructure), DevOps must adapt. This playbook outlines the shift through these key strategic pillars:
* **Foundational Prerequisites:** Robust CI/CD, automated testing, and Infrastructure as Code are essential for agentic workflows.
* **Evolving Engineering Roles:** Engineers transition from code producers to system designers, agent operators, and quality stewards.
* **Structured Collaboration:** Integration across IDEs, PRs, pipelines, and production environments is required.
* **Repository Design:** Repositories must act as explicit interfaces using skill profiles and instruction files.
* **Development Methodology:** Shift from ephemeral prompt engineering to durable, specification-driven development.
* **Governance & Security:** Implement frameworks for custom agent consistency/auditability and transform CI/CD into active verifiers of semantic intent and security.
* **New Success Metrics:** Move from volume-based productivity counts to outcome-based and trust-boundary measurements.
* **Foundational Prerequisites:** Robust CI/CD, automated testing, and Infrastructure as Code are essential for agentic workflows.
* **Evolving Engineering Roles:** Engineers transition from code producers to system designers, agent operators, and quality stewards.
* **Structured Collaboration:** Integration across IDEs, PRs, pipelines, and production environments is required.
* **Repository Design:** Repositories must act as explicit interfaces using skill profiles and instruction files.
* **Development Methodology:** Shift from ephemeral prompt engineering to durable, specification-driven development.
* **Governance & Security:** Implement frameworks for custom agent consistency/auditability and transform CI/CD into active verifiers of semantic intent and security.
* **New Success Metrics:** Move from volume-based productivity counts to outcome-based and trust-boundary measurements.
This article examines the development of Microsoft’s Azure SRE Agent, designed to mitigate operational toil in mission-critical environments. By utilizing an "agentic workflow" of specialized AI agents, Microsoft has integrated automation across the entire software development lifecycle. This human-AI partnership has autonomously resolved over 35,000 incidents and saved more than 50,000 developer hours, accelerating root cause analysis and mitigation while maintaining rigorous governance and human oversight.
This article details how Google SREs are leveraging Gemini 3 and Gemini CLI to accelerate incident response, root cause analysis, and postmortem creation, ultimately reducing Mean Time To Mitigation (MTTM) and improving system reliability.
>When deployed strategically, agents can empower SREs to offload low-risk, toilsome tasks so they can focus on the most critical matters.
Agents in practice include:
* **Contextual Information:** Providing SREs with details from previously resolved incidents involving the same service, including responder notes.
* **Root Cause Analysis:** Suggesting potential origins of an issue and identifying recent configuration changes that might be responsible.
* **Automated Remediation:** Handling low-risk, well-defined issues without human intervention, with SRE review of after-action reports.
* **Diagnostic Suggestions:** Nudging SREs towards running specific diagnostics for partially understood incidents and supplying them automatically.
* **Runbook Generation:** Automatically creating and updating runbooks based on successful remediation steps, preventing recurring issues.
.
Agents in practice include:
* **Contextual Information:** Providing SREs with details from previously resolved incidents involving the same service, including responder notes.
* **Root Cause Analysis:** Suggesting potential origins of an issue and identifying recent configuration changes that might be responsible.
* **Automated Remediation:** Handling low-risk, well-defined issues without human intervention, with SRE review of after-action reports.
* **Diagnostic Suggestions:** Nudging SREs towards running specific diagnostics for partially understood incidents and supplying them automatically.
* **Runbook Generation:** Automatically creating and updating runbooks based on successful remediation steps, preventing recurring issues.
.
TraceRoot.AI is an AI-native observability platform that helps developers fix production bugs faster by analyzing structured logs and traces. It offers SDK integration, AI agents for root cause analysis, and a platform for comprehensive visualizations.
**Experiment Goal:** Determine if LLMs can autonomously perform root cause analysis (RCA) on live application
Five LLMs were given access to OpenTelemetry data from a demo application,:
* They were prompted with a naive instruction: "Identify the issue, root cause, and suggest solutions."
* Four distinct anomalies were used, each with a known root cause established through manual investigation.
* Performance was measured by: accuracy, guidance required, token usage, and investigation time.
* Models: Claude Sonnet 4, OpenAI GPT-o3, OpenAI GPT-4.1, Gemini 2.5 Pro
* **Autonomous RCA is not yet reliable.** The LLMs generally fell short of replacing SREs. Even GPT-5 (not explicitly tested, but implied as a benchmark) wouldn't outperform the others.
* **LLMs are useful as assistants.** They can help summarize findings, draft updates, and suggest next steps.
* **A fast, searchable observability stack (like ClickStack) is crucial.** LLMs need access to good data to be effective.
* **Models varied in performance:**
* Claude Sonnet 4 and OpenAI o3 were the most successful, often identifying the root cause with minimal guidance.
* GPT-4.1 and Gemini 2.5 Pro required more prompting and struggled to query data independently.
* **Models can get stuck in reasoning loops.** They may focus on one aspect of the problem and miss other important clues.
* **Token usage and cost varied significantly.**
**Specific Anomaly Results (briefly):**
* **Anomaly 1 (Payment Failure):** Claude Sonnet 4 and OpenAI o3 solved it on the first prompt. GPT-4.1 and Gemini 2.5 Pro needed guidance.
* **Anomaly 2 (Recommendation Cache Leak):** Claude Sonnet 4 identified the service restart issue but missed the cache problem initially. OpenAI o3 identified the memory leak. GPT-4.1 and Gemini 2.5 Pro struggled.
Five LLMs were given access to OpenTelemetry data from a demo application,:
* They were prompted with a naive instruction: "Identify the issue, root cause, and suggest solutions."
* Four distinct anomalies were used, each with a known root cause established through manual investigation.
* Performance was measured by: accuracy, guidance required, token usage, and investigation time.
* Models: Claude Sonnet 4, OpenAI GPT-o3, OpenAI GPT-4.1, Gemini 2.5 Pro
* **Autonomous RCA is not yet reliable.** The LLMs generally fell short of replacing SREs. Even GPT-5 (not explicitly tested, but implied as a benchmark) wouldn't outperform the others.
* **LLMs are useful as assistants.** They can help summarize findings, draft updates, and suggest next steps.
* **A fast, searchable observability stack (like ClickStack) is crucial.** LLMs need access to good data to be effective.
* **Models varied in performance:**
* Claude Sonnet 4 and OpenAI o3 were the most successful, often identifying the root cause with minimal guidance.
* GPT-4.1 and Gemini 2.5 Pro required more prompting and struggled to query data independently.
* **Models can get stuck in reasoning loops.** They may focus on one aspect of the problem and miss other important clues.
* **Token usage and cost varied significantly.**
**Specific Anomaly Results (briefly):**
* **Anomaly 1 (Payment Failure):** Claude Sonnet 4 and OpenAI o3 solved it on the first prompt. GPT-4.1 and Gemini 2.5 Pro needed guidance.
* **Anomaly 2 (Recommendation Cache Leak):** Claude Sonnet 4 identified the service restart issue but missed the cache problem initially. OpenAI o3 identified the memory leak. GPT-4.1 and Gemini 2.5 Pro struggled.
The Azure MCP Server implements the MCP specification to create a seamless connection between AI agents and Azure services. It allows agents to interact with various Azure services like AI Search, App Configuration, Cosmos DB, and more.
