Netflix uses an internal system called Service Topology to maintain a live, queryable dependency graph for thousands of microservices. The platform merges three distinct data sources—eBPF network flow logs (for kernel-level visibility), IPC metrics from instrumented services (for application context), and aggregated distributed traces (for request paths)—to provide engineers with a unified view of runtime connections. This architecture helps teams quickly identify the blast radius of failures, understand upstream dependencies, and resolve incidents more efficiently by visualizing how various components interact in real-time.
- Data integration from eBPF logs, IPC metrics, and distributed traces ensures comprehensive coverage even for uninstrumented services.
- A three-stage aggregation pipeline resolves multi-hop paths into direct application-to-application edges to simplify troubleshooting.
- The processing architecture leverages Apache Pekko Streams across multi-region Kafka consumers.
- The system supports sub-second response times and provides historical time-window aggregations for incident correlation.
- Data integration from eBPF logs, IPC metrics, and distributed traces ensures comprehensive coverage even for uninstrumented services.
- A three-stage aggregation pipeline resolves multi-hop paths into direct application-to-application edges to simplify troubleshooting.
- The processing architecture leverages Apache Pekko Streams across multi-region Kafka consumers.
- The system supports sub-second response times and provides historical time-window aggregations for incident correlation.
