Posted by admin on September 8, 2014
After reading Jeff Passan's article about WAR and his view of its failings, I got a little hot under the collar and intemperate in my discussion of the issue on twitter.
The defensive metrics are constantly critiqued. I agree that there may be issues with the defensive stats, but my issues aren't the ones brought up by the critics. I believe that the metrics do a decent job of measuring the percent of time a particular ball with a particular hang time has been caught in the past. All you need for that is a stopwatch and a way to mark the play's location on the field. Sure there may be biases in this, but there are biases in who batters and pitchers face or even what umpire they appear against. So this isn't the biggest issue with fielding stats.
It's also true that fielding stats don't correlate quite as strongly year to year as the batting stats do (see chart after break), but there is also a lot more variability in opportunity for fielders than for batters (and more variability in batting stats than people perceive). A batter is going to get 3-6 PA's per game every game. The distribution of balls hit to fielders is much more random. But even this isn't the big issue with fielding stats.
(Chart of 2012 to 2013 correlation of defensive and offensive runs for near full-timers who didn't change primary position.)
The main issue with defensive stats is how do we account for positioning versus fielding skill.
For example, our defensive numbers come from Baseball Info Solutions and for every ball put into play BIS looks at where the ball is caught and its hang time and compares it to similar plays from previous (or even future games as it's updated at year end also) to know that 12% of balls like this were caught over X-number of years. Which is pretty straightforward to do and is probably what most everybody would do if they wanted to analytically measure fielding value.
The issue with defensive stats, however, is this. If the team is really good/bad or even lucky/unlucky at positioning players, it may be that the 12% catch would actually be caught 70% of the time given the player's initial positioning. BIS doesn't track player initial locations (other than noting shifts) because they aren't available on TV and even if they did, which number should we go with (88% or 30%) as we don't really know how much of the positioning is due to the team or the player?
Now if we just look at location and hang time the fielder making the play saved 88% of a hit, but we could also make the case that the fielder saved 30% and the team saved 58%. Do the fielding coach/aGM/scouts need a WAR number? The fact is some portion of a hit was prevented and we try to recognize that value.
There are other strategic impacts all over the field (platooning is a similar case).
This is something I've been thinking about a lot lately in the hope that we'll see detailed positioning from MLB's new system. I'm not sure there is a satisfying answer here. Somebody earned that 58%. One approach describes what actually happened on the field and gives the value to people on the field and another describes the player's skill in fielding.
This is also why BIS throws out shift plays. These non-standard fielder locations lead to very large runs saved values since there are plays made that would be impossible to make without being in a shift.
My main point here is to caution people that even if we had perfect fielder location data, we would not then immediately have a completely satisfying answer as to which fielder was the best.
If you want to read more about WAR, we explain most all of it in the about section on baseball-reference.com.