This paper provides a theoretical analysis of Transformers' limitations for time series forecasting through the lens of In-Context Learning (ICL) theory, demonstrating that even powerful Transformers often fail to outperform simpler models like linear models. The study focuses on Linear Self-Attention (LSA) models and shows that they cannot achieve lower expected MSE than classical linear models for in-context forecasting, and that predictions collapse to the mean exponentially under Chain-of-Thought inference.
This article explores how prompt engineering can be used to improve time-series analysis with Large Language Models (LLMs), covering core strategies, preprocessing, anomaly detection, and feature engineering. It provides practical prompts and examples for various tasks.
This article demonstrates how to use the attention mechanism in a time series classification framework, specifically for classifying normal sine waves versus 'modified' (flattened) sine waves. It details the data generation, model implementation (using a bidirectional LSTM with attention), and results, achieving high accuracy.
SigLLM is an extension of the Orion library, built to detect anomalies in time series data using LLMs. It provides two types of pipelines for anomaly detection: Prompter (directly prompting LLMs) and Detector (using LLMs to forecast time series).
This paper introduces Toto, a time series forecasting foundation model with 151 million parameters, and BOOM, a large-scale benchmark for observability time series data. Toto uses a decoder-only architecture and is trained on a large corpus of observability, open, and synthetic data. Both Toto and BOOM are open-sourced under the Apache 2.0 License.
Datadog announces the release of Toto, a state-of-the-art open-weights time series foundation model, and BOOM, a new observability benchmark. Toto achieves SOTA performance on observability metrics, and BOOM provides a challenging dataset for evaluating time series models in the observability domain.
SHREC is a physics-based unsupervised learning framework that reconstructs unobserved causal drivers from complex time series data. This new approach addresses the limitations of contemporary techniques, such as noise susceptibility and high computational cost, by using recurrence structures and topological embeddings. The successful application of SHREC on diverse datasets highlights its wide applicability and reliability in fields like biology, physics, and engineering, improving the accuracy of causal driver reconstruction.
A deep dive into time series analysis and forecasting methods, providing foundational knowledge and exploring various techniques used for understanding past data and predicting future outcomes.
Article discusses a study at MIT Data to AI Lab comparing large language models (LLMs) with other methods for detecting anomalies in time series data. Despite losing to other methods, LLMs show potential for zero-shot learning and direct integration in deployment, offering efficiency gains.
The article discusses an interactive machine learning tool that enables analysts to interrogate modern forecasting models for time series data, promoting human-machine teaming to improve model management in telecoms maintenance.
