Prashant Kumar Singh, Kshitij Gupta and Shivendra Srivastava write about how AWS Lambda replaced an aging network capture system with a new pipeline built using eBPF and Rust. The old architecture relied on kernel-side extensions that faced significant performance issues due to rule explosion in iptables as microVM density increased. By moving packet interception to the kernel via eBPF's traffic control (tc) hooks and handling aggregation in unprivileged, memory-efficient Rust processes, Lambda achieved constant-time lookups, supported IPv6, and significantly reduced CPU/memory overhead without dropping packets or modifying customer data.
- The new system uses a decoupled architecture: eBPF for kernel capture, Rust taggers for userspace aggregation, and an orchestrator to manage the fleet of taggers per host.
- To maintain security, unprivileged Rust processes receive access to ring buffers through file descriptors passed via Unix domain sockets using SCM_RIGHTS.
- The system achieves byte-for-byte identical Amazon Ion records compared to the legacy system, allowing for seamless migration without changing downstream consumers.
- Ring buffer sizes are mathematically derived from peak packet rates and drain intervals (100ms) to ensure no events are lost during bursts.
- The new system uses a decoupled architecture: eBPF for kernel capture, Rust taggers for userspace aggregation, and an orchestrator to manage the fleet of taggers per host.
- To maintain security, unprivileged Rust processes receive access to ring buffers through file descriptors passed via Unix domain sockets using SCM_RIGHTS.
- The system achieves byte-for-byte identical Amazon Ion records compared to the legacy system, allowing for seamless migration without changing downstream consumers.
- Ring buffer sizes are mathematically derived from peak packet rates and drain intervals (100ms) to ensure no events are lost during bursts.
