My Answer to “I Don’t Like How Complicated WAR Is and How It Is Constantly Changing.” “WAR is Like GDP for Baseball”
Posted by sean on March 29, 2013
Some of the common critiques of the Wins Above Replacement framework include: 1) Why do FanGraphs and Baseball-Reference.com have such different numbers, 2) How can we trust it when the numbers change, and 3) How can we trust it when I can't calculate it?
For the first question, our announcement today of a consistent replacement level between FanGraphs and Baseball-Reference.com has done a considerable amount to bring our two methodologies into alignment at least on the question of how big of a basket of WAR to hand out to players each year. Previously, FanGraphs allotted nearly 300 additional WAR due to a much lower replacement level. Our meeting in the middle has erased this difference to zero.
For the next two questions, I would point to a very widely quoted and very widely used statistic from economics, Gross Domestic Product (GDP). Here is the Wikipedia article on Gross Domestic Product. I'm going to argue that WAR is essentially GDP for baseball.
- GDP is an estimate of the market value of a country's goods and services. WAR gives you an estimate of a player's overall value.
- Wikipedia lists three different approaches to computing GDP, all trying to come to the same value. There are at least that many different implementations of WAR trying to estimate the same value.
- The guidebook for computing a country's GDP runs over 600 pages and requires the collection of statistics that no one person could accumulate. The explanations of the various WAR frameworks probably run 200+ pages across all of the sites with a WAR framework and at least on Baseball-Reference.com accumulating and crafting the data to go into the calculations has take years of work.
- Calculations for historical GDP change as new information or new industries and types of work are performed in an economy and then added to existing calculations. Calculations for WAR change as new data becomes available from proprietary vendors, RetroSheet or sabermetric study (such as pitch framing, pitch blocking and pitch f/x data).
- GDP is computed after the fact and we often aren't officially notified of a recession until many months after a quarter ends and researchers may continue to change GDP estimates years after the fact. WAR can change retroactively as we get additional information on how a new park plays or has changed or how a player's quality of competition has changed.
- GDP is a single number showing a broad measure of economic output--for example, Great Britain and France's GDP numbers are close enough to be indistinguishable to anyone watching their economies on a day-by-day basis. Yadier Molina, David Wright, Buster Posey, Ryan Braun and Andrew McCutchen were all within a half a win of value last year--a difference so small that a case could be made that any of them were the most valuable player last year.
- GDP does not account for every aspect of economic performance such as whether the production is from rebuilding after a disaster, the economy's asset value or black market production. WAR does not account for every aspect of player value such as clubhouse persona, any value from clutch/non-clutch performances, use by the manager, salary/contract status, or likely growth or decay of skills.
- The people who compute and create GDP calculations are economic experts who are building on years and years of economic study and research. The people who compute and create WAR or WAR-like frameworks are building on and expanding the years of sabermetric research by experts now employed directly by teams (Sean Smith, Tom Tango, Keith Woolner, Bill James) as advisors or experts in the area of statistics and evaluation (Nate Silver, Pete Palmer).
- The people who use and rely on GDP, news media, politicians, business owners don't have a prayer of computing it, but rely on subject experts to provide well-reasoned and carefully calculated estimates of economic value. The people using WAR (GM's, news media, agents) to estimate player value don't have a prayer of calculating it, but rely on subject experts either publicly or privately to provide well-reasoned and carefully calculated estimates of player value.
I can certainly understand unease with using one-number estimates like WAR, but I would point out that it comes from a long line of research, thought, and process that is common throughout the social sciences.
Editor's Note: You have my permission to republish any or all of this provided you link to this original post and attribute it to me. Sean Forman