Time Series and Temporal Data in ML

DS-GA 3001 - Fall 2026

Claudio Silva

NYU Center for Data Science

2026-11-17

Today

  1. Temporal data basics: events and measurements, time structures, aggregation
  2. Charts for time: line charts and aspect ratio, many series, heat maps, horizon charts
  3. Evaluating a forecast: prediction intervals, time-ordered validation, residuals, horizon
  4. Drift and monitoring: when a deployed model goes stale, and how to see it
  5. Anomaly detection: showing anomalies in context, and the threshold trade-off

Parts 1 and 2 are based on materials by Enrico Bertini (NYU Tandon).

Temporal Data Basics

What is temporal data?

Data in which values depend on time, and time is explicitly recorded.

Two kinds of temporal data

Event data

Time + object (with attributes)

“Something happened at time T”

  • A tweet, an email, an alarm, a model prediction request
  • Often irregular: events happen when they happen
  • Question: when, how often, in what order?

Measurement data

Time + measure(s)

“This was the value at time T”

  • Temperature, revenue, a stock price, daily model accuracy
  • Usually sampled at regular intervals
  • Question: how does the value change?

Counting events per interval turns event data into measurement data.

Three time structures

  1. Sequential: time as a line (Jan 1, …, Dec 31). Shows trends and change.
  2. Cyclic: time as a repeating cycle (Mon, Tue, …, Sun). Shows rhythms.
  3. Hierarchical: nested resolutions (year / month / week / day / hour). Allows drill-down.

Most real series have all three at once. You choose which one a chart emphasizes.

The same orders, two time structures

Sequential: sales by month, 2013 to 2016

Cyclic: number of orders by day of the week

Neither chart can show what the other shows.

  • Left answers: is the business growing, and when are the peaks?
  • Right answers: which day of the week is slow?
  • Same order records both times. Choosing the bin (month or weekday) is choosing the question.

Aggregation is a design decision

What aggregation buys

  • Less noise
  • Visible long-term trends
  • A manageable number of points

Daily to monthly averages makes a seasonal pattern readable.

What aggregation costs

  • Short events disappear
  • Extreme values are averaged away
  • The pattern can depend on the bin size

A one-hour outage is invisible in a monthly average.

Choose the granularity from the question: trends want coarse bins, anomalies want fine ones.

Charts for Time

Line charts: time on x, value on y

Position encodes both time and value; the line’s slope encodes the rate of change.

The same data at three aspect ratios

The aspect ratio (width / height) changes how steep every change looks.

Banking to 45°

  • We compare the slopes of line segments most accurately when they are near 45°
  • Banking to 45°: choose the aspect ratio so the typical segment has a 45° slope
  • Right: Canadian lynx trappings, 1821 to 1934
  • Wide and flat (bottom): each cycle rises slowly and falls quickly, which the tall version hides

Several series: color by category

Three categories on shared axes are easy to compare.

The spaghetti plot

Seventeen categories on shared axes: color runs out and lines hide each other.

Two fixes: filter, or highlight

Filter: show only the series you care about

Highlight: keep the rest as faded context

  • Filtering removes the context: are these three series typical?
  • Highlighting keeps the context, at the cost of some clutter
  • A third fix is small multiples: one small chart per series on shared axes. Nothing is hidden, but two series are harder to overlay.

Stacked areas are hard to read

Four regions as lines

The same four regions stacked

Only the bottom layer and the total have a flat baseline. Every other layer rides on the ones below it.

  • Left: comparing any two regions takes a glance, but the total is not drawn
  • Right: the total is the top edge, but try to say whether South (red) rose or fell in a given quarter
  • Stack when the total and the bottom layer are what matter

Heat maps: time on one axis, categories on the other

Color replaces height, so many series fit in the space of one line chart.

Measles in the United States, by state and year

Calendar heat maps

  • A heat map laid out by the calendar: rows are weekdays, columns are weeks
  • Weekly rhythm reads down a column, seasonal rhythm across a row
  • Here: daily bike-share rides, 2014 to mid-2017
  • Summers are bright, winters dark, every year

Horizon charts: many series in little space

Line charts squeezed into thin rows

The same rows as horizon charts

Same data, same space. The next two slides show how the right-hand version is built.

  • Left: about fifty line charts, each a few pixels tall. They all look flat.
  • Right: blue is above the baseline, red below, and darker is further from it

Building a horizon chart (1): cut into bands

Fill the area between the line and the baseline, then slice it into bands of equal height. Darker means further from the baseline.

Building a horizon chart (2): mirror and collapse

Mirror the negative values above the baseline, then slide every band down to the baseline. Height becomes color intensity.

Sparklines

“Small, intense, word-sized graphics with typographic resolution.” (Edward Tufte, Beautiful Evidence, 2006)

No axes and no labels: a sparkline sits beside a number and shows where it came from.

Evaluating a Forecast

Time series tasks in ML

What we ask a model to do

  • Forecast: predict the next values
  • Classify: label a whole series or a window
  • Detect anomalies: flag what does not belong
  • Monitor: watch a deployed model’s inputs and outputs over time

What time changes

  • Observations are not independent: today looks like yesterday
  • The order matters: you may only train on the past
  • The future differs from the past: trends, new behavior, drift

The evaluation tools from Week 3 assumed none of these three. Each one has a visual check.

A forecast is a line and a band

Synthetic daily demand. Linear model on a time index, a yearly cycle and day of week; band from the model’s errors on its last 180 training days.

A random train/test split is too optimistic

Same synthetic series; gradient-boosted trees on time index, day of week and day of year.

Backtesting: rolling-origin evaluation

Same synthetic series and the linear model from two slides back; six folds, each tested on the next 120 days.

Residuals show what the model missed

Same synthetic series; the linear model with the day-of-week inputs removed.

Error grows with the forecast horizon

Same synthetic series; a separate linear model per horizon, each given the 14 most recent days observed when the forecast is issued. All horizons are scored on the same 93 days, each forecast issued on or after the last training day.

Drift and Monitoring

Why deployed models go stale

A model learns \(P(y \mid x)\) from data drawn from \(P(x)\). After deployment either can change.

  • Data drift (covariate shift): the inputs change, \(P(x)\). New customers, a new sensor, a season the model never saw.
  • Concept drift: the relationship changes, \(P(y \mid x)\). The same input now deserves a different answer.

What we can watch

  • Inputs and predictions: available immediately, no labels needed
  • Accuracy: needs the true labels, which often arrive days or months later

Four shapes of drift

The shape decides how to respond: a sudden change calls for retraining, a recurring one for keeping the old model around.

Schematic, after Gama et al. (2014).

Accuracy over time

Synthetic stream: a two-feature classifier scored on 400 new labeled points each week. The labeling rule changes at week 44.

Two monitors, two different alarms

Same synthetic stream. Weeks 20 to 32: the inputs shift and the labeling rule is unchanged. Week 44: the rule changes and the inputs do not.

Designing a monitoring view

  • Overview first: one row per metric or feature, on a shared time axis (heat map, horizon chart, sparklines)
  • Show the reference: the training period, a baseline value, an alert threshold
  • Annotate events: deployments, retraining, data pipeline changes
  • Pick the bin size on purpose: weekly hides a bad afternoon, hourly is mostly noise
  • Link inputs to outcomes: selecting a time range in one view filters the others
  • Slice: overall accuracy can stay flat while one segment collapses
  • Say what is delayed: mark the last week for which labels are complete
  • Read: Yang et al., Diagnosing Concept Drift with Visual Analytics (DriftVis), IEEE VAST 2020

Anomaly Detection

Three kinds of anomaly

Schematic series. Dotted lines mark the highest and lowest values of the normal series.

Flagging what the forecast did not expect

Synthetic demand series, last 240 days, with 12 injected anomalies; σ is a robust estimate of the forecast’s typical error on the training data. A retrospective demonstration: the first 120 of these days are days the forecast model was trained on.

The threshold is a trade-off

Same detector and data as the previous slide; the dashed line marks k = 3.

What to take away

  • Pick the time structure first: sequential, cyclic or hierarchical. It decides the chart.
  • The aspect ratio and the bin size are choices, and both change what the reader sees
  • Many series: filter, highlight, small multiples, heat maps, horizon charts
  • Never evaluate a forecaster on a random split; backtest, and show the error per fold and per horizon
  • Plot residuals against time, weekday and lag; structure there is something the model missed
  • Input drift and accuracy loss are different events; watch both, and annotate what changed
  • An anomaly needs its context: show the expected band, and count false alarms as well as hits

References

Next: the last lecture before presentations

  • Tuesday, Nov 24: Interpretable ML and Fairness
  • Dec 1 and Dec 8: final project presentations
  • Dec 14: final project report due (8 pages)