Kevin Goldstein of Baseball Prospectus
Baseball America's Top 100
Keith Law of ESPN
What I've done is go back and look at the top 100 lists from each of these sources from 2007 through 2009. I then used WAR as a measure of MLB success or lack thereof to determine if each source 'hit' or 'missed' on a particular prospect. However, before getting into the data there are several limitations to doing such objective analysis of these prospect lists.
The first limitation is prospect age. Some prospects on this list can not be listed as a failed prospect merely because of their career performance thus far. For example, Domonic Brown, listed on both BA and Keith Law's top 100 lists for 2009 has produced -0.6 WAR thus far in his career. However, he is by no means a failed prospect.
Injury is another huge limitation to evaluating these lists. A player might be a top prospect until an injury limits them or even ruins their career entirely.
Prospect mismanagement could also be a huge issue for evaluating a player. One might argue that this could be the case with Domonic Brown, but it is a factor that is prevalent across baseball. A player may have immense upside, but his future is heavily dependent on the team for which he plays.
Finally, one must consider limitations in using this analysis to interpret future lists. One factor is that methodologies change over time. It is reasonable to believe that in evaluating each crop of prospects each source weighs performance and scouting, as well as upside and proximity to the majors differently.
Those are just a few of the limitations, and there are likely countless more. When using this data to come to a conclusion one must consider what outside forces are impacting the data, and understand how those impacts influence the interpretation.
That said, let's take a look at 2007.
Most of the data is pretty self explanatory. I should mention that a 'MISS' is defined as:
A prospect is on a top 100 list but has failed to provide a positive career WAR to this point. It could also mean that one of the other lists named a player that produced a positive WAR. For example, Brandon Wood is a 'MISS' for both BA and Kevin Goldstein as both men had him on their top 100, but he has produced negative WAR thus far. An example of the other type of 'MISS' would be Elvis Andrus, who was number 65 on BA's list, but went unmentioned by Kevin Goldstein. To this point in his career Andrus has accumulated 10.1 WAR.
Additionally, the 'Closest' column lists the source that most closely predicted a prospect's performance. For example, if a prospect ranked 8th in WAR, the list that had him closest to 8th would be named the closest. For the 'DIFF' columns then allow you to determine just how close their predictions were. When a prospect was not ranked on a list it was given a default value of 125, resulting in differences greater than 100.
Additionally, the 'Closest' column lists the source that most closely predicted a prospect's performance. For example, if a prospect ranked 8th in WAR, the list that had him closest to 8th would be named the closest. For the 'DIFF' columns then allow you to determine just how close their predictions were. When a prospect was not ranked on a list it was given a default value of 125, resulting in differences greater than 100.
Looking at 2007 from an objective point of view, one would likely come to the conclusion that Baseball America was more accurate overall. Baseball America missed on only 26% of prospects compared to Goldstein's 32%. Similarly, Baseball America was the closest to the prospects actual WAR ranking an impressive 60% of the time. Again, this data could be misleading for a variety of reasons, but it is interesting nonetheless.
No comments:
Post a Comment