Evaluating April MLB Predictions

tag:blogger.com,1999:blog-7965585998403674176.post-89073422217363570122007-09-24T20:09:00.001-04:002008-08-15T00:57:03.716-04:002008-08-15T00:57:03.716-04:00Evaluating April MLB Predictions<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.baseball-almanac.com/players/pics/bill_pecota_autograph.jpg"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 219px; height: 306px;" src="http://www.baseball-almanac.com/players/pics/bill_pecota_autograph.jpg" alt="" border="0" /></a>Everybody makes baseball predictions in late March/early April. A lot of people just predict who will win each division, and who will advance to the World Series. Anyone can do this- you really only have to have a general knowledge of the top teams. There are also people who predict how many wins each of the 30 teams will have. There are various complications with this (Jayson Stark's predictions have the average team winning 83.6 games, which is quite unlikely), but the thing about this is you actually have to know what you are doing. People make these predictions differently- some rely strictly on numbers, others on "feel". I found 13 sets of these predictions- <a href="http://sports.espn.go.com/mlb/preview07/" target="_blank">10 from ESPN</a> (Gammons, Stark, Crasnick, Olney, Neyer, Kurkjian, Phillips, Law, Caple, Karabell), two from BP (<a href="http://baseballprospectus.com/article.php?articleid=6023" target="_blank">PECOTA</a> and <a href="http://www.baseballprospectus.com/article.php?articleid=6052" target="_blank">BP Hit List</a>), and also the <a href="http://sonsofsamhorn.net/lofiversion/index.php/t16348.html" target="_blank">over/unders</a> from <a href="http://www.sportsinteraction.com/sportsbook/index.cfm?CFID=29657207&CFTOKEN=496a78bb8ed84b57-3A6A595B-BDB9-4A4E-FD038FE61EEF71E0" target="_blank">SportsInteraction.com</a> (via <a href="http://sonsofsamhorn.net/" target="_blank">SoSH</a>). I thought I'd take a look at some of the best and worst individual predictions, as well as whose overall predictions were most accurate. (Note: These lists aren't just based on who was the closest- I also factored in how far off the other predictions were. So predicting at team within two games if the average prediction was eight games off would be higher than predicting a team exactly if the average was just three games off.) The Best 1. PECOTA, Chicago White Sox Predicted wins: 72 On pace for: 71.0 The over/under for the White Sox was 89.5, and the ESPN analysts average prediction was 84.6. Chicago won 90 games in '06 after winning 99 in 2005. <a href="http://www.firejoemorgan.com/2007/03/computers-dont-have-feelings.html" target="_blank">Much was made</a> of PECOTA's pessimism, but this turned out to be, pretty easily, the best prediction of the year. 2. Jayson Stark, Seattle Mariners Predicted wins: 85 On pace for: 86.6 The average for everyone else was 76.5, and PECOTA had them winning only 73 games. They only won 78 games in '06, while finishing last in the AL West. <a href="http://sports.espn.go.com/mlb/preview07/team?team=sea" target="_blank">ESPN's preview</a> had JJ Putz under "Bust", as they were worried about his elbow pains. I feel like that <a href="http://www.baseball-reference.com/pi/gl.cgi?n1=putzjj01&t=p&year=2007" target="_blank">turned out OK</a> for him. 3. Steve Phillips, Minnesota Twins<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.nicksfix.com/nypost_jan13.gif"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 200px;" src="http://www.nicksfix.com/nypost_jan13.gif" alt="" border="0" /></a> Predicted wins: 78 On pace for: 78.6 <a href="http://vegaswatch.blogspot.com/2007/09/steve-phillips-not-smart.html" target="_blank">Steve Phillips: <del>Not</del> Smart</a>! Well, for now at least. Nobody else at ESPN had the Twins winning less than 83 games, and PECOTA pegged them at 90. On the ESPN Message Boards, Twinsdude08 remarked that, "The Twins just have too much talent to not win the division." I don't know how Phillips came to 78 wins, but, as well see later, this accuracy certainly isn't a trend. 4. Rob Neyer, Washington Nationals Predicted wins: 69 On pace for: 71.1 People (especially Buster Olney), thought the Nationals were going to be really bad. The second most optimistic ESPN prediction was 64 wins; six had them losing over 100 games. Neyer, who always refers to his predictions as "running the numbers", was more realistic- it's hard to lose 100 games in the NL, since all the other teams are really bad too. 5. Peter Gammons, Colorado Rockies Predicted wins: 84 On pace for: 87.2 The Rockies have far exceeded all expectations- their over/under at SportsInteraction was 74.5 wins, and nobody else had them winning even 80 games. Even Gammons didn't see this coming, but everybody else was so far off that his prediction makes the list. The Rest (Predictor, Team, Prediction, Actual Pace) 6. Phillips, Orioles, 70, 69.4 7. Karabell, Pirates, 69, 68.9 8. Caple, Marlins, 68, 68.6 9. PECOTA, Oakland, 80, 77.3 10. Stark, Red Sox, 96, 96 Now for the fun part... The Worst <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_Xtn5bp8dZEg/RvhpIK_QeoI/AAAAAAAAAdA/eAaMeKvy4U0/s1600-h/Predictions2.bmp"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer;" src="http://3.bp.blogspot.com/_Xtn5bp8dZEg/RvhpIK_QeoI/AAAAAAAAAdA/eAaMeKvy4U0/s320/Predictions2.bmp" alt="" id="BLOGGER_PHOTO_ID_5113952965988743810" border="0" /></a> 1. Buster Olney, Washington Nationals Predicted wins: 49 On pace for: 71.7 Pretty much everyone was a little off on the Nats, but this one stands out. Sure, things didn't look good back in March, but 113 losses? No NL team lost more than 96 games in '05 or '06- it would be quite amazing if someone was actually that bad. Olney is a smart guy, but I'm not sure where he got 49 wins from. 2. Jim Caple, Kansas City Royals Predicted wins: 54 On pace for: 70 I don't know, maybe people just think it's funny to pick teams to be amusingly bad. I kind of see Caple's reasoning here, as he predicted the other four AL Central teams to average 89 wins. But seriously, how did he see this playing out? Did he figure they would all go like 16-3 against the Royals? Thats the only way they could average 89 wins, since they have to play each other so many times. 3. Steve Phillips, Boston Red Sox Predicted wins: 82 On pace for: 96 This only came out third in my little formula, but that may be generous. Boston was a mess in '06, and they still managed 86 wins. Nobody else had the Red Sox winning less than 90 games. Between this and repeatedly predicting the Yankees to miss the playoffs in August, I feel like Phillips just makes predictions for the shock value of them. 4. Keith Law, Seattle Mariners<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.businessweek.com/bschools/images/klaw_2004_118x158.jpg"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 122px; height: 164px;" src="http://www.businessweek.com/bschools/images/klaw_2004_118x158.jpg" alt="" border="0" /></a> Predicted wins: 65 On pace for: 86.6 Law and Stark didn't quite see eye to eye on this one, as their predictions were 20 wins apart, the highest such margin. Seattle has surprised people, but their over/under was 79.5 wins; there really wasn't any reason to think they would approach 100 losses. The Rest (Predictor, Team, Prediction, Actual Pace) 5. Philips, Diamondbacks, 78, 90.8 6. Phillips, White Sox, 92, 71 7. PECOTA, <del>Devil</del> Rays, 78, 66.4 8. Karabell, Cubs, 75, 86 9. Stark, Reds, 85, 74.2 10. Karabell, Astros, 88, 70.5 <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_Xtn5bp8dZEg/RvhmDK_QenI/AAAAAAAAAc4/4bkznU-yqCQ/s1600-h/Predictions.bmp"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer;" src="http://3.bp.blogspot.com/_Xtn5bp8dZEg/RvhmDK_QenI/AAAAAAAAAc4/4bkznU-yqCQ/s320/Predictions.bmp" alt="" id="BLOGGER_PHOTO_ID_5113949581554514546" border="0" /></a> Now, let's look at whose overall predictions were the most accurate. The table on the right is ranked by how close people were, on average of all 30 predictions. The top three are all predictions based on numbers. PECOTA is 100% quantitative, and both Neyer and the Hit List rely heavily on numerical predictions. Those are the only three that did better than Vegas. Neyer did really well- his picks are 19-10-1 against the over/unders so far. Even more impressive, of his seven predictions that had large discrepancies with Sports Interaction, he was right on six of them. On the other end of the spectrum is, not surprisingly, Mr. Phillips. If you watch Baseball Tonight and SportsCenter (or are a Mets fan...) this probably doesn't come as much of a surprise. Luckily, Steve Phillips isn't paid a lot of money to analyze baseball for a living- if he was, his incompetence would be pretty embarassing. Pictures: <a href="http://www.baseball-almanac.com/players/player.php?p=pecotbi01" target="_blank">Pecota</a>, <a href="http://www.nicksfix.com/nypost_jan13.gif" target="_blank">Phililps</a>, <a href="http://deadspin.com/sports/espn/buster-olney-responds-i-like-my-job-thank-you-137475.php" target="_blank">Olney</a>, <a href="http://www.businessweek.com/bschools/content/aug2004/bs20040824_9874_bs049_0.htm" target="_blank">Law</a>.<script type="text/javascript" src="http://widgets.ballhype.com/story/000/104/104173.js"></script> <noscript><a href="http://ballhype.com/story/evaluating_april_mlb_predictions/">Ballhype - Evaluating April MLB Predictions</a></noscript> <script type="text/javascript">ballhype_story_widget_104173(true);</script>Vegas Watchhttp://www.blogger.com/profile/02353166004125421683VegasWatch@gmail.com15