What does an 80% prediction interval promise?
An 80% prediction interval promises that, across many forecasts, the actual outcome lands inside the interval 8 times out of 10. If outcomes land inside 9 times out of 10, the intervals are too wide; if 7 times, too narrow. Both are failures of calibration. The promise is about the long run: one outcome outside one interval tells you nothing on its own.
Hyndman and Athanasopoulos explain in section 5.9 of Forecasting: Principles and Practice that the lower limit of an 80% interval is the 10th percentile (the 0.1 quantile) of the forecast distribution, so the actual value should fall below it about 10% of the time and above the upper limit about 10% of the time.
How is an 80% interval calculated?
For a forecast with normally distributed errors, the 80% prediction interval is the point forecast plus or minus 1.28 times the forecast standard deviation, and the 95% interval uses a multiplier of 1.96. Both multipliers come from Table 5.1 in Forecasting: Principles and Practice, section 5.5. The book's worked example: a naïve forecast of 758.88 with a residual standard deviation of 11.19 gives an 80% interval of [744.5, 773.2] and a 95% interval of [736.9, 780.8]. A point forecast without an interval, the authors write, can be of almost no value.
| Coverage | Multiplier (normal errors) | Interval limits (percentiles) | Expected misses per 100 forecasts | Expected inside after 50 forecasts | Within 2 standard deviations |
|---|---|---|---|---|---|
| 80% | 1.28 | 10th and 90th | 20 | 40 | 34 to 46 |
| 95% | 1.96 | 2.5th and 97.5th | 5 | 47.5 | 44 to 50 |
Multipliers from Table 5.1 in section 5.5, checked 13 November 2026. The last two columns are computed with the binomial distribution: expected count = n × p and standard deviation = √(n × p × (1 − p)), so 2.83 for 80% and 1.54 for 95% with n = 50.
Why do intervals get wider further out?
Prediction intervals usually get wider as the forecast horizon increases, because the further ahead you forecast, the more uncertainty there is (section 5.5). An assistant that gives the same width for next week and next year is not showing you its real uncertainty.
The US National Hurricane Center publishes a clear example. Its track forecast cone is built from circles sized so that two-thirds of its official forecast errors over the previous five years fall inside, which makes each circle a 67% interval. The circles grow with every step ahead:
| Hours ahead | Atlantic circle radius (nautical miles) | Eastern and Central North Pacific radius (nautical miles) |
|---|---|---|
| 12 | 25 | 25 |
| 24 | 39 | 37 |
| 36 | 49 | 48 |
| 48 | 62 | 56 |
| 60 | 77 | 66 |
| 72 | 95 | 78 |
| 96 | 134 | 106 |
| 120 | 200 | 138 |
Source: NHC, Definition of the NHC Track Forecast Cone, radii for 2026 based on 2021 to 2025 errors, checked 13 November 2026. The Atlantic five-day circle is eight times the width of the 12-hour circle.
Why do 80% intervals often miss more than 20% of the time?
Many model-based intervals are too narrow. Hyndman and Athanasopoulos note in section 9.8 that ARIMA-based intervals, like most prediction interval calculations, tend to be too narrow because they only account for variation in the errors. They leave out variation in the parameter estimates and in the choice of model, and they assume the historical patterns will continue. The result is a stated 80% interval that, in practice, catches fewer than 8 outcomes in 10.
How do I check whether a forecaster is calibrated?
Keep a forecast log: for every forecast, record the stated interval, its coverage (80% or 95%) and the outcome once it is known. After 50 forecasts at 80%, a calibrated forecaster has about 40 outcomes inside the interval. By the binomial distribution the standard deviation of that count is about 2.8 (the square root of 50 × 0.8 × 0.2), so a count between roughly 34 and 46 (within 2 standard deviations) is consistent with calibration; a count far outside that range is evidence the intervals are miscalibrated. Run the same check separately for different horizons, because calibration at one week says little about calibration at one year.
What score should I use for intervals?
Use the Winkler score, proposed by Robert Winkler in 1972 and described in section 5.9 of Forecasting: Principles and Practice. When the outcome falls inside the interval, the Winkler score is the width of the interval, so narrower is better. When the outcome falls outside, the score adds a penalty proportional to how far outside it landed. Lower Winkler scores are better, and the score rewards intervals that are as narrow as possible without missing too often. For single quantiles, the same section describes the quantile score (pinball loss).
| Score | What it grades | How it works | Source |
|---|---|---|---|
| Winkler score | A prediction interval | Interval width, plus 2/α times the distance by which the outcome misses it (α = 0.2 for 80%) | Winkler (1972); FPP3 5.9 |
| Quantile score (pinball loss) | One quantile, such as the 90th percentile | An absolute error weighted by how likely the outcome is to fall on each side | FPP3 5.9 |
| CRPS | A whole forecast distribution | The quantile score averaged over all probabilities | FPP3 5.9 |
| Brier score | A probability for a yes-or-no event | Mean of (probability − outcome)², with outcome 1 or 0 | Brier (1950) |
Lower is better for all four. The forecast score calculator computes interval coverage, the Winkler score and the Brier score from your own log.
What score should I use for yes-or-no forecasts?
For probability forecasts of yes-or-no events, use the Brier score, introduced by Glenn Brier in "Verification of forecasts expressed in terms of probability" (Monthly Weather Review, 1950). The Brier score is the mean squared difference between the forecast probability and the outcome (1 if the event happened, 0 if not). It runs from 0 (perfect) to 1, and a forecaster who always says 50% scores 0.25.
When does calibration matter less?
Calibration needs many forecasts of the same kind to judge. For a one-off decision, look at whether the forecaster's reasons and inputs hold up instead. Calibration also says nothing about sharpness: a forecaster can be perfectly calibrated with intervals too wide to be useful, which is why the Winkler score also counts width.
Sources
Checked 13 November 2026:
- Hyndman, R. J. and Athanasopoulos, G., Forecasting: Principles and Practice, 3rd edition, OTexts: 5.5 Distributional forecasts and prediction intervals, 5.9 Evaluating distributional forecast accuracy, 9.8 Forecasting (ARIMA)
- Brier, G. W. (1950), Verification of forecasts expressed in terms of probability, Monthly Weather Review 78(1)
- Winkler, R. L. (1972), A decision-theoretic approach to interval estimation, Journal of the American Statistical Association 67(337), 187-191
- National Hurricane Center, Definition of the NHC Track Forecast Cone
Next step
Start a forecast log today with four columns (date, forecast and interval, coverage, outcome), check it with the forecast score calculator, then set up the routine in the forecast review loop.