Analytical forecasting

Analytical forecasting estimates a future business outcome using information available before that outcome occurs. A useful forecast names the measure, period, decision and uncertainty. Define the task, compare a simple benchmark with other methods on earlier periods, then keep a record of issued forecasts and outcomes.

Define the decision and measure

“Forecast demand” is too broad to assess. A workable brief might ask for completed service requests by week over the next month so a manager can plan staffing. Specify what counts as completed, which date assigns a request to a week, the reporting time zone, the latest complete source period and the lead time the decision requires.

Forecast at a level that matches the action. A monthly total may help a budget discussion but conceal a weekly peak that affects a roster. A highly detailed forecast may be too unstable to guide that decision. Record the audience and action before choosing the level of detail.

Keep the forecast separate from a target. The forecast estimates what may happen under stated assumptions; the target describes a desired result. A plan sets out actions to address any gap. Retain dated forecasts when assumptions or plans change so readers can see what was expected at each decision point.

Prepare the history

Plot the observations before modelling them. Look for trends, seasonal patterns, unusual periods and changes in how the measure was recorded. Confirm whether a missing period means zero activity, an incomplete delivery or a changed definition. Ask the source owner about outliers before removing them.

Assess a model against outcomes defined consistently with its input data. If a case-status rule changed during the history, make periods comparable under an approved rule where possible, or mark the break. Record the data cut-off and any source revisions relevant to each forecast.

Start with a benchmark

A simple benchmark shows whether added complexity helps. Candidates include the historical mean, the latest observation, the latest observation from the same season and a drift method that extends the average change between the first and last observations. None is automatically suitable. A seasonal method needs comparable seasonal history; repeating the latest value may miss a recurring cycle.

A more elaborate model may use past values, calendar effects or other predictors. Before adding a predictor, check whether its future value will be known when the forecast is issued. If it will not, the model needs a forecast or an explicit scenario for that predictor. Using its later-observed value in a historical comparison does not recreate an advance forecast.

Benchmark Methods vs. Advanced Models

Compare forecasts as they would have been made

At several earlier cut-off dates, fit each candidate using information that would have been available then and predict the same horizon the business now needs. Compare those predictions with subsequent outcomes. Apply the same cut-offs, population and outcome definition to the benchmark and challenger.

Choose error measures for the decision. Mean absolute error expresses the average size of a miss in the measure’s units. Root mean squared error gives larger misses more weight.

Percentage errors can be undefined or unstable when actual values are zero or close to zero. Inspect errors at the lead times and in the segments that matter; a single overall score can hide a weak result.

Fit to training data does not establish future accuracy. Record which historical periods could be compared fairly and any model settings chosen after looking at results.

Forecast Accuracy Metrics

Show uncertainty and assumptions

A point estimate alone can imply more precision than the evidence supports. Where a defensible forecast distribution is available, show a prediction interval for a named future outcome and coverage level. It depends on the model and its assumptions; it does not guarantee that the outcome will fall inside it. Intervals commonly widen at longer horizons, though this depends on the method.

Scenarios answer a different question: what would the model predict under specified future conditions? Their endpoints are not automatically prediction-interval bounds. In a regression scenario, an interval calculated while holding future predictors fixed may omit uncertainty about those predictors. State that limit beside the result.

Pros and Cons of Using Prediction Intervals

Maintain the decision record

Present the issue date, source cut-off, forecast period, unit, estimate, uncertainty basis and material assumptions. When new evidence or a planned intervention changes the outlook, issue a dated revision and keep the earlier version available.

Once outcomes arrive, compare them with the forecasts issued beforehand. Look for repeated bias, changing seasonal behaviour and intervals that appear too narrow or too wide. A single miss does not identify its cause.

Forecast Quality Assurance Checklist