WEATHER FORECASTING IS MORE GUESSING GAME THAN SCIENCE
Copyright 2003 by STEPHEN WINZENBURG
Associate Professor of Communications
Grand View College
Des Moines, Iowa
When most people get up in the morning to get ready for work, one of the first things they want to know is what the weather is going to be like that day. Will they need a jacket? An umbrella? Will they need to layer their clothing for a cool morning and a warm afternoon?
Yet the question that they should be asking is, Where is the best place to get an accurate weather forecast? Consumers that want weather information typically choose a medium based purely on habit without giving thought to there being differences when it comes to forecasts. In our household my teenage daughter wakes up to the forecast on the radio while my wife takes it from the local morning paper and I get my weather from the 6 a.m. television news shows. Others log online, watch the Weather Channel, listen to the National Weather Service or glance at USA Today. Do these media all do the same job when it comes to predicting the weather or is there one medium that is best?
There have been few studies conducted regarding the accuracy of media weather forecasting and almost no direct comparisons done of forecasts by each medium. For six weeks during March and April of 2003, I monitored a variety of media in the Des Moines market, including the morning news shows on the three major television stations, the 6 a.m. hour on the highest-rated radio stations, the Des Moines Register, USA Today, and the National Weather Service.
The simple answer to the question is that television does the best job predicting the weather, with radio a close second. Newspapers are the most inaccurate. But it also became obvious that the range of predictions for any one day varied dramatically, even within a medium. Audiences should not assume that all forecasts are similar and no media outlet should be trusted for an accurate forecast beyond a 48-hour period.
The average forecast high for the day was off around four degrees, high or low, meaning that if a meteorologist predicted a high of 68 the temperature would typically fall between 64 and 72 degrees. Is that acceptable accuracy? A windy 64 feels quite different from a sunny 72. Professor Mark Anderson of the University of Nebraska, who has conducted forecast studies for a number of television stations across the country, finds those numbers to be acceptable but the lay person at home may get the impression that the forecasters are constantly wrong if off by such a wide margin.
The best medium for predicting the days high temperatures was television, with stations averaging 3.6 degrees off the mark. Though they were statistically about the same, the markets lowest-rated TV station did slightly better in forecasting the daytime highs than did the much higher-rated and better-equipped competitors. Radio was almost as accurate as television at 3.8 degrees off, but less reliable were the National Weather Service at 4 degrees difference and USA Todays 4.11 (from information provided by the Weather Channel). These media sources were all within a half-a-degree of each other. By far the least accurate predictor of the days temperature was the Des Moines Register, which was off almost six degrees either direction!
There were no days during the six weeks when the media on average guessed the correct high, though they came within one degree on three occasions and within two degrees four times. Over three-fourths of the time the average was off three degrees or more. About 10% of the time the media were off in their predictions by double digits, including forecasts that were 12 to 15 degrees too high for the day! However, on seven days during the survey the actual high was accurately forecast by one media source, with six of those seven coming from television stations and the lowest-rated TV station in town getting it on-the-nose the most often.
Most disturbing, however, was the inability of any medium to accurately predict the sky conditions. On over 55% of the days, the forecasts of conditions were inaccurate. During this survey, when a meteorologist said it was going to rain or be partly sunny that day, over half the time he was wrong. Some days were better than others and Professor Mace Bentley of Northern Illinois University notes that in the spring due to complex storm systems, accuracy rates typically goes down. Yet being unable to predict simple sky conditions for the upcoming 12-hour period leads to justifiable skepticism from consumers.
Forecasting that evenings temperatures had about the same amount of accuracy as the daytime predictions, averaging four degrees off, and most of the time the predictions were too high. Over two-thirds of the predicted daytime and nighttime temperatures were over-estimated by three or four degrees, while one-third of the time the forecasters under-estimated the temperatures by four or five degrees. Radios nighttime forecast average was higher due to one radio station in the market that consistently was the farthest off of all the broadcast media compared to other radio stations in town that simply aired forecasts from local TV meteorologists.
Guessing the highs 24- and 48-hours ahead of time was less accurate, being off almost seven degrees either direction. So when tomorrows high was said to be 68, the actual temperature would end up anywhere between 61 and 75, which seems an incredibly broad range to a person at home who is making plans for the next day. Half of the time they were too high by almost eight degrees, the other half they were too low by around six degrees.
For the next-day forecast the highest-rated TV station in town did the best job at 6.2 degrees off, while the Des Moines Register was the farthest off at eight degrees. The most glaring examples of bad two- and three-day forecasts were the rainy predictions for Easter Sunday morning, for which television meteorologists boldly proclaimed two days ahead of time that there would be wet Easter morning services and a very soggy Easter Sunday, which would lead to sunshine in the afternoon. Easter ended up being cloudy all day and no rain fell except for a few trace sprinkles around 5 p.m.
||AVERAGE DEGREES HIGH OR LOW
||48 HOUR ACCURACY
In terms of which medium is most apt to come closest to predicting temperatures over a 48-hour period, television does the best job. Four-out-of-ten times TV stations got closest to guessing the actual temperatures and they were rarely the medium that was the farthest off. The newspapers, which received their information from the Weather Channel, were the farthest off half of the time.
So the conclusion in the Des Moines market is that television and radio are the best choices for same-day weather forecasts, with TV coming closest the most often even though over half the time they got that days conditions wrong. Most of the time temperature predictions were four degrees too high for the same day and seven degrees high or low for the next day. And newspapers were by far the least accurate and the farthest off in their forecasts.
Once you got beyond a 48-hour period, there were wildly divergent forecasts for the four-to-seven day period. The radio stations wisely chose not to predict beyond 48 hours and only the NWS and some television stations took a guess at six- or seven-day forecasts. The four-day was 7.5 degrees too high or low, the five-day was 9.2 degrees off, with the six-day wrong by an average 10.1 degrees and the seven-day forecast an amazing 11.3 degrees incorrect. If 68 degrees was predicted as the high seven days ahead of time, real temperatures a week later ranged from 57 to 79 degrees. And long-term sky conditions predictions were virtually worthless, meaning that most media outlets should stick with the short-term three day forecasts.
One of the biggest surprises was the large amount of conflicting information from the same media source and the lack of up-to-the-minute forecast information from web sites. For example, one television meteorologist in Des Moines does morning forecasts for his TV station while also voicing the forecasts for a couple of local radio stations and putting the forecast information on the web site. However, on some days none of the forecast information was the same on the three media sources even though the same weathercaster provided it! In the same 15-minute period he said on TV that the high would be 44, he went on the radio and said the high would be 45 and a check of the internet site showed he was predicting the high would be 49. This did not happen every day but it happened enough that it would be confusing to consumers.
The internet proved to be surprisingly outdated with its weather forecasts. Most local stations would give fresh predictions on the air but not update the web sites with the same forecasts. The Weather Channel would provide information to the web sites of USA Today and local papers that would contradict what was being shown on the cable channel. One TV station placed a crawl on the screen during the morning news that told viewers that if they wanted more weather information to go to the stations web site, yet the online forecast wasnt the same as what was being displayed on the TV screen. Often the internet was the least accurate source of weather forecasts and was six degrees off in its 24-hour predictions.
The least credible medium for forecasting is print. Since the morning paper information is usually gathered at 9 p.m. the night before, it is already old when people read it. Three-fourths of the time the Des Moines Register was the farthest off in its forecast high for the day compared to the rest of the media, while one television station was the farthest off only twice in the entire six-week study.
The National Weather Service also did not come out well in the study. It typically forecast ranges instead of specific predictions, but it was still unable to predict accurately. The NWS did worst in its long-range predictions beyond three days. However, the Weather Service did get storm watch and warning information on the air before local stations.
Radio forecasting had its own problems. Most forecasts were read too quickly, with no sense of communicating what was going to happen that day. One local station daily spent ten seconds mentioning the weather sponsor, then only three seconds doing the actual forecast. This station would usually only give the forecast for the daytime high and no other information. It was too brief and too fast. Another station daily gave one forecast on the air while a dramatically different forecast would be available online.
Radio, the medium which prides itself on timeliness, was also guilty giving old forecasts. During the morning radio shows the same forecast was usually given for four hours straight, even though on television weathercasters would adjust their forecasts based on changing information. On the other hand one radio station gave a 5:30 a.m. forecast from the NWS and then at 5:37 a different forecast that was five degrees higher. Two different forecasts within seven minutes? Maybe explain why to the audience. And during the afternoon hour radio stations were guilty of running outdated forecasts, with one station playing the afternoon highs in the 40s forecast at 5:15 one day, long after the temps had peaked only in the 30s.
Despite the success of television in this survey, there were number of frustrations when watching television meteorologists. They added personal comments with their forecasts, unlike the weather predictions given from any other medium:
Aware of the criticism they receive for being off on their forecasts, weathercasters on television fudge by leaving wiggle room. Instead of boldly predicting rain, they will say widely scattered showers are a possibility for some parts of the viewing area. That information does not help. I never heard one TV person give a specific amount of predicted rain nor give the percentage chance of rain even though those numbers were available from the National Weather Service.
TV weathercasters love to trumpet when they think they were right and rarely admit they are wrong. On April 1, most forecasters saw that it was going to be unusually hot, but they didnt want to stick their neck out too much so they said things like near-record highs and the record is in jeopardy. It turned out that the 120-year-old record was broken by four degrees and the next day these same weather people went on the air proclaiming that the record high was shattered just as they had predicted!
On Thursday, April 17, a local forecaster looked at the weekend predictions and said, Im trying hard to buy into the models of widespread heavy rainfall but I just cant do it. The next day he came back to say the models changed so now it looked promising for weekend rainfall. There were giant thunderstorms by 10 p.m. that night! He was wrong in his original forecast and didnt admit it, instead blaming it on the models (which he is supposed to interpret) and stating, I was a little bit pessimistic about it yesterday. I feel much more confident today. Huh? Just tell us you were wrong and decided to change your mind.
One Friday a local weatherman predicted 40 degrees and a rain/snow mix, adding We hope youre ready for this because we expected this. Oh no he didntjust 24 hours earlier the same weatherman predicted that the next day it would be sunny and 50! But who would remember that other than a researcher keeping track for a study?
One unique admission of error was done by the meteorologist at a local station while doing his radio weather forecast. That day the station had predicted a high of only 43, yet it turned out to be sunny and 54. This is a weather guy who doesnt like to be wrong, said the meteorologist, but this heat is the way to do it. Namely, he messed up but found a way to turn it into a positive!
Weathercasters love hype. On Friday, April 4, just three days after the record high was set, the TV stations began predicting a giant snow storm for the weekend. One station predicted significant snow without giving numbers. Another said heavy snow and still another significant accumulations for Sunday. Then on Saturday night the stations began running crawls across the bottom of the screen for a Winter Storm Warning that would not take place until Sunday night and Monday morning! They used prime time TV to predict that ten inches was coming in another 24 to 36 hours. What ended up happening? Nothing unusual arrived by Sunday evening and a little over two inches fell on Monday morning, light stuff that melted by mid-morning.
The expensive local satellite technology did not do any better at predicting the conditions than did the NWS or The Weather Channels information in the newspapers. There was no need to put people into a panic and paste Winter Storm Warning on the screen 24 hours ahead of time. It was television weathercasting at its worst and the inaccuracy of the storm prediction left the viewing audience skeptical again.
They give contradictory information. They will put on the screen a predicted high of 68 but will verbally add, I dont think well get that high. One TV station showed the high that day being 59 but the meteorologist said, I only think well get into the mid-50s. One put on the screen a prediction of clearing skies for the weekend while saying, I think our futurecast is a little too optimistic. Then why put the information on the screen if you dont agree with it?
One weatherwoman said, Well be right at 50 degrees todaya high of 48. Well, which one is it? A weatherman said, This afternoon will be drybut not completely dry. Some drizzle, some haze, some fog. Oh, thats helpful. Or one day a weathercaster called 42 degrees at 5:30 a.m. very mild but a half hour later claimed that it was cold out despite no change in temperature. This was the same weatherman who on another day said 62 degrees was also mild.
One station put on the crawl that the high would be 58, but when the weatherman put the forecast on the full screen it said the high would be 56. Another weathercaster often stated the incorrect time, contradicting the clock on the TV screen. Sloppy numbers result in lack of credibility with the audience.
They often provide unnecessary information, such as the Des Moines television forecaster who loved to focus on the days weather in small towns in upstate New York. If youre flying into Watertown, New York, expect a foot of snow. Three days earlier the same weatherman mentioned they were expecting snow in Binghamton, New York. Did he used to work in that market or did he grow up there? Whatever the case, the information was of no importance to central Iowa viewers.
Or the sports-fan weathercaster who always talked about the weather for that days baseball game, which would be of little interest to 95% of the viewers. Or the forecast for Iraq, which may have had some interest during the war but ended up getting more air time than the statewide predictions. Or Live Doppler is picking up rain getting ready to move into downtown Quincy in the next 15 minutes, a small Illinois town that is 300 miles outside the main viewing area. Who cares? Instead TV people should spend more time to focus on the region around this market.
Weather people often state the obvious. The temperature is going up, reported one morning weatherman. Most days it does. Grab an umbrella all the weathercasters love to ad lib, though we can probably figure out that if they are predicting rain we should take one (and from this study we know that if they predict rain there is a 55% chance that it wont happen!). One weathercaster said grab your umbrella three times in less than ten minutesonce at the opening of the newscast, once in the tease to the weather and once during the weathercastbut it was already raining out and his three suggestions seemed unnecessary. Even two cans of hairspray wont do you good today. You mean its really going to be windy?
Partly sunny is not partly cloudy. Trained professional meteorologists confuse the terms. They will be positive by predicting partly sunny, when in reality partly cloudy is the more optimistic term, meaning the skies are more sun than clouds. The terms and screen symbols are incorrectly used interchangeably.
Normal is not average. Weather people claim that the normal high is 68, when in reality what they mean is the average temperature is 68those two are not the same thing. Lets say the following high temperatures occurred on April 15th over the past decade: 73, 72, 72, 72, 72, 66, 66, 65, 62, 62. A weathercaster would claim (as the NWS also incorrectly states) that the normal high for that date is 68. Yet 68 never was a high on that date! They mean the average is 68. So when weathercasters ad lib about todays predicted high being ten degrees above normal, they really should state that its above the average for this date. I heard that phrase used only once in the six weeks of this study, in the statement, Our average low is 38 degrees, thats our normal low.
While television proved itself to be the best medium to use in weather forecasting, the track record of those involved proved to be weak. Audiences should not assume that media with better reputations and more expensive equipment are the most accuratethey arent. All the television stations did an acceptable job with the 24-hour temperature forecasts but were wrong over half the time about the conditions. And anything beyond a three day forecast lacks credibility.