Prospect List Review, Part 3

This is the third and final portion of analysis on the comparison I've done for the three major prospect lists.  I should mention that some other sites have done interesting work compiling data similar to this, so I highly recommend you search that out if this stuff tickles your fancy.

The first two portions can be found here, and will be helpful to understanding the analysis in this post:

Post 1 - 2007

Post 2 - 2008

Once again, the data can be found here (or in excel format by e-mailing us):

Prospect List Review

Anyway, interesting notes about the 2009 prospect list data.  The list contains 130 players, 82 of which have produced a positive WAR to this point in their careers.  However, due to the more recent time frame that this data covers, the average WAR is only 1.91.  Matt LaPorta brings up the rear posting a -1.4 figure, and Andrew McCutchen paces the field producing 12.9 WAR to this point.  So how did the data break down?

The average differences are again in the 30s, with Baseball America leading at 32.30 and Goldstein coming in last at 37.25.  Keith Law, just as in 2008, splits the difference with an average difference between WAR ranking and his list of 34.21.  The 'MISSES' data shows yet another similar story with the 3 lists coming in the same order, ranging from BA's 33% to Goldstein's 42% with Law coming in at 38%.

It starts getting interesting though when you look at the number of times each analyst was closest to the actual rankings in their predictions.  Here Law paces the field at 42.31%, while BA and Goldstein finished with 41.54% and 33.08% respectively.  This is certainly interesting as it is the first time anyone has bested Baseball America's team of analysts in any of the categories.  Breaking it down even further, one can determine that Law simply predicted more players that produced positive WAR as all 3 analysts were similarly accurate (or inaccurate) in predicting which players wouldn't produce.  It should be noted that the difference between Law and BA here is only 1 prospect, so any arguments about the difference being negligible would be valid as well.

In summary we can determine a few things about the overall process of creating a prospect list.  Of the 383 prospects (including repeats) named by each of the sources, 258 or 67.4% have produced a positive WAR to this point in their careers.  That's actually pretty impressive considering the uncertain nature of developing prospects and all the issues I mentioned in the first post of this series.

Additionally, it seems that having more information is better.  Baseball America has a team of evaluators that determine their top 100 through individual ranking, discussion and voting.  Similarly, Keith Law uses his personal evaluations as well as opinions of scouts and executives he trusts around the game.  These approaches seem to give a more fitting view of a prospect, possibly due to The Wisdom of Crowds.

I'd love to hear your thoughts and opinions, as well as any takes you had on the data or process.  Let me know in the comments section.  I know there is a lot more to gain from using this data, not to mention much more data to gather.