I look a lot at Copper. The main reason is that I originally started collecting data at Copper and therefore have a lot more data there. I added other resorts later. When looking at statistics, it’s useful to look at a lot of data, and sometimes it is helpful to look at outlier cases. Last storm had a day at Copper that recorded 16″ of snow. That definitely an outlier.
In this post, I’m again going to be focusing on Copper. First, let’s look again at the forecast error distribution. This is produced by taking by subtracting the actual snow value from the forecast. Therefore, if the actual is higher than the forecast, it will produce a negative number, and vice versa.
The most obvious thing about the distributions to the right is that both the National Weather Service and Opensnow do a good job keeping their average error around zero. Snowforecast.com does an okay job and Forecast.io and Snow-forecast.com routinely under-forecast.
There’s more data for NWS and Opensnow since I originally only tracked those two sources.
There are some huge outliers, both positive and negative.
To learn more about the distribution, I ran a few simple statistics which are displayed in the second image to the right.
If we look just at the mean of the error distribution, Opensnow has the best accuracy, having an average error of only 0.092″. That’s pretty good. The median value is 0.5″ though, so that means that the distribution is skewed. We see that the skewness is negative with a value of -1.622″. That means that distribution has a longer tail on the negative side.
All the sources have a negative distribution, though Opensnow’s is the largest. They also have the largest Kurtosis, meaning that the distribution is broader than the standard normal distribution. A standard normal distribution has a mean of zero, standard deviation of one, skewness of zero, and kurtosis of zero. Having large values of skewness and kurtosis mean that data have a bias and is indicative of poor forecasts.
NWS has smaller values for skewness and kurtosis than Opensnow, but a larger standard deviation. Trying to understand what this means in terms of answering the simple question “who has the best forecast?” is difficult. Having a small standard deviation is obviously good. But since the mean, standard deviation, skewness, and kurtosis are all related we can’t just say that Opensnow is the best because it has the smallest mean and standard deviation. If their distribution had a skewness of zero and kurtosis of zero then it would obviously be the best. But NWS is less skewed and has less shoulder.
One thing that can be unequivocally be said: Forecast.io and Snow-forecast.com are really bad at predicting snow. Snowforecast.com comes in a distant third to Opensnow and NWS. I would say that NWS and Opensnow do equally good jobs at predicting snow at Colorado resorts.
This is confirmed in the last two columns of the table. The Bust % is an admittedly arbitrary statistic of when I personally feel that a forecast is a bust. For example, if a forecast calls for 6″ of snow and it turns out there’s 16″, that’s a busted forecast. If a forecast calls for 2″ and it snow 3″, that’s a good forecast. We can see that NWS, Opensnow, and Snowforecast.com are pretty equal when it comes to busted forecasts. The t-test compares a distribution to the NWS as a null hypothesis. The only statistically significant difference at the 95% lever is Snow-forecast.com and that’s not a good thing since they’re obviously worse.
The post Obsession with Copper appeared first on Atmoz.