Inverse scaling
A common approach to time-series forecasting is to provide the model with increasingly long histories. Unlike language, however, distant observations in a stochastic series may contain volatility or patterns irrelevant to the current forecast.
Across PatchTST and other continuous-context baselines on ETTh1, performance deteriorates as the context window grows. At 3,000 steps, forecasting error increases by more than 68% relative to the shorter-context setting. This degradation suggests that attention alone does not reliably suppress irrelevant historical information.
Retrieval instead of a longer window
Retrieval-Augmented Forecasting retains a fixed 720-step input window and retrieves historical segments that resemble the current query. The retrieved segments are then supplied as exogenous context.
On ETTh1, RAFT reaches an MSE of 0.379, compared with 0.647 for the strongest long-context baseline in our evaluation, while requiring less computation.
The underlying principle is simple: when useful history is sparse, the model should retrieve relevant segments rather than process the entire available history.
I presented this work as a poster at the ICLR 2026 TSALM Workshop. The paper, poster, and code are linked above.
Cite
@inproceedings{
ahuja2026retrieval,
title={Retrieval Mechanisms Surpass Long-Context Scaling in Time Series Forecasting},
author={Rishi Ahuja and Kumar Prateek and Simranjit Singh and Dr Vijay Kumar},
booktitle={1st ICLR Workshop on Time Series in the Age of Large Models},
year={2026},
url={https://openreview.net/forum?id=Qj96MlCmZw}
}