Evaluating April MLB Predictions (2008)

We did this last year, in a couple parts. It quickly became obvious that the computer projections are much more accurate than those of ESPN's analysts; not at all surprising, considering the various methods used. But PECOTA also fared well against the sabermetrically-inclined analysts, like Joe Sheehan, Keith Law, and Rob Neyer.

For now, I'll use 12 sets of predictions: six from ESPN (Stark, Kurkjian, Olney, Law, Phillips, Neyer), two from BPro (PECOTA, Joe Sheehan), three from Yahoo! (Steve Henson, Tim Brown, and Jeff Passan), and the over/unders. After the season is over, I'll have a post incorporating a few more computer projections, and comparing how everyone did both this year and over the last couple. For now, here are some of the best and worst individual predictions, as well as whose overall predictions were the most accurate.

(Note: These lists aren't just based on who was the closest- I also factored in how far off the other predictions were. So predicting at team within two games if the average prediction was eight games off would be higher than predicting a team exactly if the average was just three games off.)

The Best

1. PECOTA, Tampa Bay Rays
Predicted wins: 88
On pace for: 97.4

Well, this was a pretty good call, wasn't it. According to the metric I've come up with to compare these projections, this was the second best one over the last two years, behind only PECOTA's 2007 White Sox projection. Tampa Bay has surpassed even the most optimistic of expectations; even if you just look at their Pythagorean record, of 87-66, they're on pace to beat PECOTA's projection by four games.

Also close: Nobody in this field.

2. PECOTA, Seattle Mariners
Predicted wins: 75
On pace for: 60.0

As we'll see later, it was a very good year for PECOTA. Both the Tampa Bay and Seattle projections seemed somewhat insane six months ago, but as it turns out they actually weren't extreme enough. PECOTA may not have nailed this one exactly, but it saw at least some of this regression coming, and knew they were going to take a large step back from 2007. Others weren't quite as prescient.

Also close: Neyer, 77.

3. Steve Henson, Houston Astros
Predicted wins: 81
On pace for: 85.2

Henson was much more optimistic about the Astros than anyone else, except maybe the great Ed Wade. He ends up looking good here, but this was mostly luck, as Houston's Pythag (73-81) is the inverse of their actual record (81-73). While that's a good thing for Henson, it's probably bad for the Astros in the long run, as it gives them the mistaken impression that they're close to being a playoff-caliber squad.

Also close: Nobody.

4. Jayson Stark, Chicago White Sox
Predicted wins: 85
On pace for: 89.4

We're left to wonder why Stark thought the White Sox would bounce back--I kind of doubt it was because he thought Quentin would have a 148 OPS+ and Danks would suddenly transform into an ace--but this is impressive nonetheless. Chicago got a bit of "revenge" on PECOTA this year, which had them winning just 77 games.

Also close: Phillips (84)

Now for the fun part...

The Worst

1. Steve Phillips, Seattle Mariners
Predicted wins: 92
On pace for: 60.0

Wow. This is about as wrong as you can possibly be. Phillips missed by 32 games, which is just an incredible amount. Think about how many things had to happen for him to be this far off:

• Steve Phillips, despite doing a terrible job as GM of the Mets, gets hired by ESPN. They let him go on TV and give his "analysis", as well as make predictions. In fact, they pay him a large sum of money to do these things. Possibly the most confounding occurrence of this whole process.

• The Mariners outplay their Pythag by nine games in 2007, going 88-74 despite being outscored by 19 runs. Many people get the false impression that they're a contender heading into 2008.

• Seattle trades a large portion of their farm system for Erik Bedard, causing some to believe they're now the favorite in the AL West.

• Even compared to the most pessimistic expectation, everything goes horribly wrong for the Mariners. Silva has a 65 ERA+. Bedard makes just 15 starts, and is only slightly above average when he does pitch. Their DH hits .234/.274/.338. Miguel Cairo starts 36 games at first base. And so on.
  

• Blogger writes post on meaningless preseason predictions.

It really has been an incredible ride. Cherish this moment; it's possibly nobody will ever be this wrong again.

Also very wrong: Kurkjian (91), Passan (91).

2. Steve Phillips, Texas Rangers
Predicted wins: 64
On pace for: 78.4

I really have no idea how he came up with this one; it didn't make sense prior to the season, and it still doesn't in late September. They won 75 games in 2007, and were outscored by only 28 runs (79-83 Pythag). Phillips wasn't nearly as far off on this one, but it's almost as bad as his Seattle prediction, since it's not like the Rangers shocked the world by playing .480 baseball.

Also very wrong: Nobody.

3. Steve Henson, Tampa Bay Rays
Predicted wins: 72
On pace for: 97.4

So, Mr. Henson, what do you think of Tampa's 2008 outlook?

"The Rays are improving but are still middle-school level to the Red Sox graduate students."

Truly enlightening. At least he wasn't the only one who didn't see this coming.

Also very wrong: Brown (73), Kurkjian (75), Olney (75)

4. Buster Olney, Baltimore Orioles
Predicted wins: 56
On pace for: 70.9

Olney did the same thing with the Nationals in '07, predicting they'd lose 113 games, and claiming the top spot here when they lost just 89. It seems as though he just gets totally caught up in the story--"the Orioles are going to be really bad; how bad? well look, I'm going to predict they lose 106 games!"--without realizing how silly it is to make such extreme predictions.

Also very wrong: Sheehan (57)

Here are the overall standings, using RMSE (lower is better, obviously):

You may notice that all these numbers are a good deal higher than they were last year, when the average was 7.12. It turns out that 2007 was historically easy to predict; the average here (11.37) is a little higher than '05-'06, but not that much.

The top two are the same as last year, with Neyer and PECOTA ahead of the pack. This does not come as a surprise, as both use similar methodologies, which are certainly more advanced than most.

The two names at the bottom also haven't changed, although this year it's Olney who fared unbelievably poorly, while last year it was Phillips. 13.05 is incredibly bad; if you just used 2007 records, without looking at RS/RA or regressing or accounting for any offseason moves or doing anything, you get 12.46. So Olney's input actually detracted from the information given. Impressive.

Photos: StarTribune.com, The DiaTribe.