Amazon S3 Batch Operations allows you to process hundreds, millions, or even billions of S3 objects efficiently. You can perform various actions such as copying objects, setting tags, restoring from Glacier, or invoking AWS Lambda functions on each object without writing custom code.
Arize Phoenix is an open-source observability library for AI experimentation, evaluation, and troubleshooting, built by Arize AI.
An article on building an AI agent to interact with Apache Airflow using PydanticAI and Gemini 2.0, providing a structured and reliable method for managing DAGs through natural language queries.
- Agent interacts with Apache Airflow via the Airflow REST API.
- Agent can understand natural language queries about workflows, fetch real-time status updates, and return structured data.
- Sample DAGs are implemented for demonstration purposes.
Breser stands for Business Rules & Expression Syntax for Easy Retrieval. It is a powerful and flexible query language designed for efficient log processing and structured data filtering.
How to ensure data quality and integrity using open-source tools for observability in data pipelines.
Data pipelines are essential for connecting data across systems and platforms. This article provides a deep dive into how data pipelines are implemented, their use cases, and how they're evolving with generative AI.
A guide to tracking in MLOps, covering code, data, and machine learning model tracking
Distributable streaming
