Uncertainty in NFL Projections

As I believe I have mentioned, I've been working on putting together a simulation of the NFL season, using a combination of the Football Outsiders win projections and the Vegas O/Us to determine team strength. That part is simple enough; I have a regression equation I derived from 2006-2008 data which weighs the various information appropriately.

It gets a big trickier when, in the spreadsheet kindly provided by xlsSports, I need to determine the standard deviation of each team's true talent. While boring, this is extremely important; the FO and Vegas projections provide only estimates of how good each team is, rather than gospel. To figure out what to use for the StDev, I looked at the last three years of data to see how far off, on average, the projections should be.

Here is a summary of the average misses (in wins) of the two different systems, as well as a combination of them ("Regression") over each of the last three years:

Even when combining the data in (what we believe is) the most efficient way possible, the average miss is 2.33 wins. In a season of only 16 games, that may seem rather high. But, as the graph below shows, even if we did know the the exact true talent of each team, our projections would still be far from perfect:

The left side of the graph shows that even if the standard deviation of true talent is 0.00, the average miss will be around 1.55; that's something you're going to have to deal with no matter how good your projections are. The black dot on the graph represents an average miss of 2.33 wins, which is where our regression has been over the last three years. So it looks like the proper StDev to use in the simulation is 0.150, which is what I'll be doing. To keep things reasonable, I've been capping each team's true talent at 0.150 on the low end, and 0.850 on the high end.

Okay, well that was the boring part. Over the next couple weeks I'll be using the sim to break down a wide variety of props, the majority of which can be found among BetUS' incredible selection of NFL offerings.