June 27, 2025
Classical and modern forecasting models based on numerical data have seen widespread success in a range of applications. While historical numerical data and engineered covariates provide a start, they fail to convey the complete context for reliable and accurate predictions. Human forecasters frequently rely on additional information, such as background knowledge and constraints, which can flexibly be communicated through natural language. My talk will be on this setting of “context-aided forecasting”, where the goal is to produce statistical forecasts by incorporating all relevant context in natural language. I will first discuss our efforts on building CiK (short for “Context is Key”), a forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context, with a strict requirement for models to integrate both modalities to produce accurate forecasts. I will discuss the Region-of-Interest CRPS, a proper scoring rule we propose, which prioritizes context-sensitive windows and accounts for constraint satisfaction. We evaluate a range of approaches on CiK, including statistical models, time series foundation models, and LLM-based forecasters, and propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark. Our experiments demonstrate surprising performance when using LLM-based forecasting models and also reveal some of their critical shortcomings, especially with respect to efficiency. I will also present new results on the benchmark, including simple methods that improve the zero-shot performance of LLMs through forecast-bootstrapping and in-context learning. Next, I will discuss other efforts from the community on benchmarks and methods for context-aided forecasting, and expand on our ongoing work on hybrid methods combining time series foundation models and LLMs for accurate and efficient context-aided forecasting. Finally, I will discuss the next big challenges in this space, centering around building interactive, agentic systems for forecasting and analysis and their potential in democratizing powerful forecasting tools, making them significantly more accurate and accessible.