OpenLogParser, an unsupervised log parsing approach using open-source LLMs, improves accuracy, privacy, and cost-efficiency in large-scale data processing.
Approach:
- Log grouping: Clusters logs based on shared syntactic features.
- Unsupervised LLM-based parsing: Uses retrieval-augmented approach to separate static and dynamic components.
- Log template memory: Stores parsed templates for future use, minimizing LLM queries.
Results:
- Processes logs 2.7 times faster than other LLM-based parsers.
- Improves average parsing accuracy by 25% over existing parsers.
- Handles over 50 million logs from the LogHub-2.0 dataset.
- Achieves high grouping accuracy (87.2%) and parsing accuracy (85.4%).
- Outperforms other state-of-the-art parsers like LILAC and LLMParserT5Base in processing speed and accuracy.
Approach:
- Log grouping: Clusters logs based on shared syntactic features.
- Unsupervised LLM-based parsing: Uses retrieval-augmented approach to separate static and dynamic components.
- Log template memory: Stores parsed templates for future use, minimizing LLM queries.
Results:
- Processes logs 2.7 times faster than other LLM-based parsers.
- Improves average parsing accuracy by 25% over existing parsers.
- Handles over 50 million logs from the LogHub-2.0 dataset.
- Achieves high grouping accuracy (87.2%) and parsing accuracy (85.4%).
- Outperforms other state-of-the-art parsers like LILAC and LLMParserT5Base in processing speed and accuracy.
New Relic's Nic Benders discusses the importance of the Innovation Centre in Hyderabad, their vision for AI, the benefits of their technologies for Indian digital businesses, and more.
Honeycomb introduces a new tool, Honeycomb for Frontend Observability, to enhance web application performance. The tool integrates OpenTelemetry instrumentation as an NPM package to collect and analyze Core Web Vitals data, offering actionable insights to improve SEO and site performance.
With all the hype around AI/ML in observability, it's more likely than ever that companies benefit from storing and viewing data in one system and training ML models in another.
Trace2 is an open source performance logging/tracing framework built into Git that emits messages at key points in each command, such as process exit and expensive loops.
