The BOM gets it wrong

Well, we all know that they get it wrong, but that’s not surprising since it’s a tricky job to predict the future. However, since they don’t ever tell us how accurate they are, we never knew exactly how wrong they were. Until now.

Average of differences between temperature forecasts and observations (Melbourne, Sydney and Perth)

The chart above is the result of some analysis on the data I collected over a month (between 25th February and 25th March 2007) for the cities of Melbourne, Sydney and Perth. The first thing you’ll notice is that the 7-Day forecasts are not as accurate as the 1-Day (i.e. tomorrow) forecasts, and that as the forecasts head off into the future, they get less accurate. This is as you’d expect.

Other things to notice are that (i) Maximum temperature forecasts are generally less accurate than Minimum temperatures, (ii) The Melbourne Maximum temperatures are the least accurate, while Sydney Minimum temperatures are the most accurate, and (iii) none of the curves are heading towards zero, i.e. the forecasts for the following day are still a surprise.

Since the data is collected over the course of only one month, it’s hard to say if this sample is representative of all Bureau of Meteorology forecasting, but at least we now have some idea of their accuracy. The rule of thumb seems to be that the next day forecast will be out by on average 1.5 degrees, and the 4-day forecast will be out by on average at least 2 degrees. This is better than I thought it was going to be, to be honest.

I’ll probably continue to crunch the numbers and see if anything interesting comes out, but I think I’ve won my bet.

Forecasting Conspiracy?

Weather Station, MelbourneOn the weekend, I (perhaps impulsively) agreed to a bet with a friend who claimed that Melbourne’s weather forecasts are accurate, while I suggested that they were slightly better than totally random. I am to record the 7-day forecasts for the next month and see how accurate the 4-day forecast is for Melbourne (compared with say, Sydney or Perth).

Since the Bureau of Meteorology does not publish their historical forecasts on their website, or indicate their forecasts’s probabilities, I think they must be embarassed by how imperfect their art is (at least when it comes to Melbourne). In a month’s time, we’ll know if that’s true, or if there’s a more likely explanation…