Tag Archives: weather

Forecasting uncertainty in practice: Snowperbole

Example of snow forecast communicating levels of undertainty; image from the Capital Weather Gang

Example of snow forecast communicating levels of uncertainty; image from the Capital Weather Gang

Because making accurate predictions is extremely difficult, we can dramatically improve both the accuracy of forecasts and enable effective communication about the forecast by embracing the uncertainty involved in the forecast. This allows decision-makers to both use the information available while understanding the limits of those predictions.

Following forecasts for a “potentially historic” storm set to hit New York and New England, public officials in New York City went to great lengths to emphasize the dangers of the storm. The Governor closed down New York’s subways in anticipation of the storm (showing one of the quirks of New York’s transit governance, local transit is under state control).

There was just one problem: the storm mostly missed NYC.

In their forecast post-mortem, the Washington Post’s Capital Weather Gang highlighted the key shortcomings of the forecast – a failure to present the level of uncertainty in the forecast.

Why were the forecasts so bad?

It’s simple: Many forecasters failed to adequately communicate the uncertainty in what was an extremely complicated forecast. Instead of presenting the forecast as a range of possibilities, many outlets simply presented the worst-case scenario.

Especially for New York City, some computer model forecasts were extremely dire, predicting upwards of 30 inches of snow – shattering all-time snowfall records. The models producing these forecasts (the NAM model and European model) had a sufficiently good enough track record to take them seriously.

However, some model forecasts (e.g. the GFS model) signaled reason for caution. They predicted closer to a foot of snow.

Part of the challenge here is that most of the forecast was accurate. This was a historic storm; the storm simply tracked a bit further to the east. Areas like New York City were right on the margins, where a small change to the inputs can mean a large change in the outcome  – and the forecast did not adequately convey that uncertainty. Add in the fact that the forecast miss happened to be the largest city in the United States, and you have a very public error.

When a forecast is so sensitive to small changes (eastern Long Island, not far away, received 30-plus inches), it is imperative to loudly convey the reality that small changes could have profound effects on what actually happens.

It’s easy to second-guess public officials making key decisions like closing transit systems after the fact (and after the forecast bust), but they can only act on the information that they have in front of them. It’s easy to argue that it is better to be safe than sorry (and this is certainly true – it is better safe than sorry) but there is a real risk of eroding public confidence in these kinds of decisions when the forecast doesn’t pan out. (It doesn’t help that despite closing the subways, the MTA’s snow plan called for trains to remain in operation without passengers to keep the tracks clear of snow)

As some meteorologists suggest, conveying the uncertainty in their forecasts should be a larger element of both the forecast and communication. It’s not just a matter of using the best information available, but also understanding the uncertainty involved.

Snow perspective, in graph form.

Since we’ve now eclipsed the seasonal record, it’s worth noting how unusual it is for DC to get lots of snow in a season, to say nothing about snow storms coming back to back.  Let’s look at the history:

We’re now well above that blue star for the 1898-99 season record.  As Gabe Klein noted on Kojo yesterday, you can’t budget for those intervening years and then expect to deal with the extraordinary snowfalls.

Likewise, there are physical limits to how fast you can remove snow, and when the snow is accumulating faster than that rate, you’ve got a problem.

City Paper has a survey of ANC commissioners, asking about their street conditions.  Gary Thompson has some good perspective on snow removal:

Gary Thompson, of ANC 3G, who lives in the 2800 block of Northampton Street NW: “My own street’s been plowed very well,” he says. “They’ve done an excellent job plowing, especially under the circumstances. There have been a few forgotten streets here or there, like a dead end culdesac,” but the city is catching up. “People are very quick to complain,” he says. “But Mother Nature is what it is. It’s a pretty powerful storm.”

Ben Thomas, not so much:

Ben Thomas, of ANC 7E, who lives in the 1100 block of Chaplin Street SE: A plow came by sometime last night, he reports (the second of two times a plow has made an appearance during D.C.’s three snowstorms this season). “They don’t really do anything but pile the snow against peoples’ cars,” he says. But the street is at least passable. “I saw a car go by a little while ago,” he says.

That’s what plowing does – it moves snow around.  And that works pretty well when you get a little snow here and a little snow there.  When you get 40 inches in a matter of a few days, there’s only so much you can do within the realm of what’s physically possible.

Perspective, please.