On the weekend, I (perhaps impulsively) agreed to a bet with a friend who claimed that Melbourne’s weather forecasts are accurate, while I suggested that they were slightly better than totally random. I am to record the 7-day forecasts for the next month and see how accurate the 4-day forecast is for Melbourne (compared with say, Sydney or Perth).
Since the Bureau of Meteorology does not publish their historical forecasts on their website, or indicate their forecasts’s probabilities, I think they must be embarassed by how imperfect their art is (at least when it comes to Melbourne). In a month’s time, we’ll know if that’s true, or if there’s a more likely explanation…
So far (after collecting 11 days of forecasts), interesting results are emerging. I will wait until I get the full month before doing a complete analysis, but it seems the forecasts are more accurate than I had thought. Although, Melbourne is less accurate than Sydney or Perth.
The amount that the a day’s 4-day forecast maximum temperature differs from its 1-day forecast (3 days later) is, on average, 2.0 degrees in Melbourne (but at worst 5 degrees), 1.7 in Perth (at worst 3), and 1.5 in Sydney (at worst 4).
The average is calculated with only 8 data points right now, so I’m keen to see if the trend persists as I get more data.
Hi Andrew!
Don’t forget to measure it on a percentage basis! a 3 degree error in Perth on a 42 degree forecast is very different from a 3 degree error in Melbourne on a 12 degree forecast. Ahh Perth…
I’ve been thinking about that. However, is a 3 degree difference on a 30 degree forecast a 10% error? If you use Fahrenheit or Kelvin, you’ll get different percentages to Celsius. What is a fair way to normalise it?
sorry for barging in like this, I got to your blog while searching for UFTP information…
Anyway, I looked around and felt quite at home with your postings (I assume being kind of geek as well might add to that).
So, what I wanted to say actually is more about the weather…
“Guessing” the forecast in certain periods of the year is quite easy – it will be the same as it was today (it is right for my country). The chances of being mistaken in this way are quite small on the long run. Consider this, how many changes of weather do occur? most of the time it is a degree up or down (day to day comparison). Which reminds me that even a broken clock is showing the right time twice a day while a clock that runs late is always mistaken.
No probs Arik. Interesting question…
Well, looking at the 29 days of observations I collected, I dropped the decimal points from the maximum temperatures, and checked how many different values were recorded for each city. Melbourne had 16 unique values, Sydney had 11, and Perth had 16 also.
Then I crunched this a bit more, and found that on average the same temperature came up 1.8 times in Melbourne, 2.6 times in Sydney, and 1.8 times in Perth also. So, you had a reasonable chance of the same temperature striking twice in the same month, particularly if you live in Sydney (where it turns out only three temperatures didn’t reoccur).
D’oh. I just realised that of course the figures in the second paragraph follow from the first.