This paper provides a theoretical analysis of Transformers' limitations for time series forecasting through the lens of In-Context Learning (ICL) theory, demonstrating that even powerful Transformers often fail to outperform simpler models like linear models. The study focuses on Linear Self-Attention (LSA) models and shows that they cannot achieve lower expected MSE than classical linear models for in-context forecasting, and that predictions collapse to the mean exponentially under Chain-of-Thought inference.
This article explores how prompt engineering can be used to improve time-series analysis with Large Language Models (LLMs), covering core strategies, preprocessing, anomaly detection, and feature engineering. It provides practical prompts and examples for various tasks.
IBM’s new foundation model, TSPulse, can go beyond standard forecasting tasks to detect anomalies, fill in missing values, classify data, and search recurring patterns. It’s also tiny enough to run on a laptop.
This paper introduces Toto, a time series forecasting foundation model with 151 million parameters, and BOOM, a large-scale benchmark for observability time series data. Toto uses a decoder-only architecture and is trained on a large corpus of observability, open, and synthetic data. Both Toto and BOOM are open-sourced under the Apache 2.0 License.
This article provides a roundup of notable time-series forecasting papers published between 2023 and 2024. It highlights five influential papers, including a case study from the online fashion industry, a review on forecasting reconciliation, and new deep learning models like TSMixer and CARD. The article emphasizes advancements in forecasting models, handling challenges in retail forecasting, and improvements in hierarchical forecasting methods.
The article discusses methods for data scientists to answer 'what if' questions regarding the impact of actions or events without having conducted prior experiments. It focuses on creating counterfactual predictions using machine learning techniques and compares a proposed method with Google's Causal Impact. The approach involves using historical data and control groups to estimate the effect of modifications, addressing challenges such as seasonality, confounders, and temporal drift.
TimesFM is a pretrained time-series foundation model developed by Google Research for time-series forecasting, focusing on point forecasts for univariate time series up to 512 time points with any horizon length and an optional frequency indicator.
This article discusses Time-MOE, an open-source time-series foundation model using Mixture-of-Experts (MOE) to improve forecasting accuracy while reducing computational costs. Key contributions include the Time-300B dataset, scaling laws for time series, and the Time-MOE architecture.
A deep dive into time series analysis and forecasting methods, providing foundational knowledge and exploring various techniques used for understanding past data and predicting future outcomes.
The article discusses an interactive machine learning tool that enables analysts to interrogate modern forecasting models for time series data, promoting human-machine teaming to improve model management in telecoms maintenance.
