Sports Reference Blog
How Many Baseball Writers Have Called or E-mailed to Talk to Me About What Goes Into WAR? Zero.
Posted by sean on September 30, 2012
You may have heard that the AL MVP is between a player who may win the Triple Crown and a player who most (if not all) of the stathead-friendly sites say is the best player in the league this year. There have been a number of articles being written by veteran writers about how stupid WAR is--complaining it's incomprehensible, stupid, meaningless, dumb, formulas are different, etc. etc.
Here are a couple of recent examples:
Here is Bill Madden in the New York Daily News
Here is Jerry Green in the Detroit News
Now I'm painting the baseball media with a broad brush, but each of these types of articles gets my hackles up. I'm a fellow card-carrying-member of the BBWAA and one would think that I would be afforded some professional courtesy before having a stat we produce being berated in print.
Not a single member of the print media, the broadcast media or radio has reached out to me to learn more about WAR since this MVP controversy has erupted. Not one. First, I apologize to the curious and hard working media members who put in the time to study the game and its analysis in detail. You know who you are, and I appreciate your hard work. I'm sure many have taken the time to read our exhaustive introduction to WAR. But in the last two months not a single person has called or e-mailed asking for more information and that includes Bill Madden and Jerry Green.
So if you are a member of the media who is skeptical about WAR and want to get some questions answered. Or if you are a radio or tv host want to talk to me on the air or on the record to excoriate me for WAR's failings. Let me know. I'll appear on any radio show to discuss WAR and make time for any writer who wants to learn something about it or debate its merits.
Here is my contact information.
#97 Honestly it doesn't matter. The Angels are currently playing in an offensive context where fewer runs are required to create a net win than the Tigers are.
Now if there's a big change in the park (so that multi-year park factors can't be used) then there's a real issue. As noisy as single year park factors are (and there's a good study on rec.sport.baseball by Harold Brooks on this matter. Good luck finding it with the new Google Groups though), they're the best we can do.
Honestly, I like to re-run the numbers using single year and multi-year park factors just to get a sense of the range of uncertainty. It rarely makes a big difference.
Problems with WAR... (1) While it does take a look at some of the intangibles, it ignores how clutch a player is. Sure Cabrera had more chances with RISP, but he also batted 25 pts higher in those situation. WAR doesn't look at that. He knocks in runs not only because of his teamates, he makes the best of his chances. Year after year, so it's not a fluke.
In 2004 the Tigers scored 100 more runs than this years team. The number 3 spot batted .320, but only produced 95 RBI's. So if Cabrera was on the team, you are saying that he wouldn't have produced more (or just a few more because his BA is 10 pts higher)?
2) The Smell Test. In Sept/Oct Trout has the 2nd highest WAR in the AL. Trout has increased his lead during this month over Cabrera even though the slash lines are: Trout .279/.389/.486/.876 Cabrera .339/.402/.688/1.089
Trout's Defense and base-running was enough to make up the batting differential and then some? Another smell test. According to WAR Trout's 2012 season is the 10th best ever since 1950. For as impressive a season it is, there is no way you can honestly tell me that its the 10th best. The only players with better seasons since 1950?
2 by Mantle, 2 by Mays, 2 by HGH Bonds, Yaz, Morgan and Ripken.
You are telling me that ARod, Jeter, Pujols, McGwire, Chipper, Rose, Mathews, Griffey, Molitor, Yount, Murphy, Strawberry, Mattingly, Murray, Dawson, Gwynn, Brett, Rice, Bench, Canseco, Hamilton, Cabrera, Schmidt, Henderson and everyone other than the 6 guys above over the last 60 pluse years never matched or exceeded the value that Trout produced this year? Something smells really bad with WAR.
Brad
#99 There were changes made (capacity up a few hundred) and there was a new scoreboard installed on top of the CF fence (actually not sure when construction was finished). These kind of changes have been known to affect how a stadium plays.
There's an argument to be made that the numbers for the players on the 2010 team are wrong, but I can't see it for this year.
There's a different argument that there might be a different mix of pitchers being used home and road. Anybody who's interested in checking this is welcome to. I doubt it's happening.
Stengel's Yankees really did use a different mix of pitchers, but these days basically pitchers go when its their turn.
#102 RISP isn't intangible. It's a very tangible thing. Its importance is easy to overstate though.
An extra 30 points of BA with RISP is worth somewhere around 2 runs (just under).
RISP is simultaneously statistically significant in explaining team runs scored and not all that important.
30 points of BA with RISP sounds like about 5 hits? That certainly is worth more than 2 runs. In the AL this year, with RISP there have been 4736 hits and 7185 runs scored, or 1.5 runs/hit with RISP. So 30 points of BA with RISP should be worth about 7-8 runs.
Still very minor
Brad,
You are REALLY, REALLY underselling Trout's year. There are only six seasons since 1950 with a 160+ OPS+ and 40+ SB that played CF, 3B, 2B, SS or C. Morgan x2, Kemp, Cedeno, Trout and HoJo.
If you set SB>=0 then there are 89 such seasons.
It really comes down to what you think of Trout's defense.
If you think he's an average defender then credit him in 8.5 WAR, a good defender 9.5 WAR, a great defender 10.5 WAR.
To say that Cabrera was more valuable you have to IMO
1) Ignore the park factors completely
2) Assume they were both average defenders
3) Give Cabrera a big bonus for playing in a weak division and helping get his team into the playoffs
4) Give him mad props for playing well in Sept.
Regarding WAR for Sept/Oct. I'm not sure where you got that as we don't show it. And even if we did, you are looking only at batting. Players play half the game in the field you know.
Another way to consider Trout's season. His 169 OPS+ which excludes defense and baserunning is the 44th highest among 2B, SS, C, 3B and CF since 1950. Even if we give him no credit for going 48 of 52 in SB and playing great defense, that puts him in the top 50 among that group.
#105 I've checked it out. (but encourage you to do so as well). The way I got this value was to run multiple regressions. In this case using OBP, SLG and BA with RISP vs team runs per game. (And I ran it for the year I had data for 1992-2000 IIRC -- before BB-ref made it easy to get these numbers)
Remember, it's not 5 extra hits. It's 5 hits in high leverage situations (at least with respect to run scoring. It may or not be high leverage when we're talking game situation). That leverage only takes you so far.
Or if you like it's 5 hits in the high weight category and 5 fewer in the more frequent situation. Again, there's a limit to how much you can get from (timing, clutch, luck select according to what floats your boat)
JB, your reaction proves my point exactly. You misunderstood most of what I said, then flailed furiously through looney tangents to try to fight off what you thought my points were. It's OK, you can believe whatever you want, I really don't care. I was writing that more to the other folks.
Apart from the nuts-and-bolts reasons, I find it hard to take WAR seriously, because WAR adherents deride all opposition as dishonest or intellectually defective.
Why? Because I disagree with something that has numerous versions, which are constantly being revised, by as much as 30 to 40%. Irrelevant, say WAR-fans. Here's some more SABR-stats to prove why your objection is irrational. Take that!
You think Trout is MVP? Fine. Based on WAR? Your decision. Support Trout.
Two things to remember:
1. Three decades ago, TPR and TPI were going to define the complete player. Today: TPR? TPI? What? ...Exactly.
2. History's the real judge. Emotions are high right now. The real MVP won't necessarily be who won the trophy, but the one who is remembered as the most valuable, and it might not be the guy with the highest WAR. Unless, of course, revised calculations "discover" that he was, in fact, the WAR-champion.
Mark,
I think sabermetricians would applaud your skepticism. It was skepticism that led to all the improvements in those metrics you're talking about (and all the ones in between). Even today, there's multiple websites using similar WAR frameworks all competing against each other.
I don't know what you're expecting. If you expect perfection, you're not going to get it. If you thing the MVP should be based on who "feels" like the MVP (see post #20), well accurate is that going to be? What if you feel something different later? If you want people who disagree with you on the internet to be more polite, then good luck with that. :-)
#111
DavidRF,
I don't see any applause for my skepticism. In fact, I detect exactly the opposite. And no substantive responses (other than more SABR-stats) to skeptics like me who point out that multiple versions and constant revision don't exactly inspire confidence.
You say seems to me that WAR *IS* being portrayed as close to perfection as it will get.
So you'd cede your human reason to a machine? Baseball is a game, played by men. I'd rather trust my instincts than a computer.
If, later, I decide I'm wrong, I'll know I made the wrong decision based on the data. At least I won't be kicking myself for suppressing my reason and handing my judgment to an equation (one of two, three, or however many versions of it, anyway) cooked up by some statistician, who might (as has happened) revise the formula and changed something by 30 or 40%.
Life can't be broken down into numbers. Yesterday's definitive TPR/TPI is long forgotten, BA/HR/RBI are still around.
READ THIS POST
As I write this, let it be known that I have only read the opening post, and ZERO replies. So if what I have to write has already been written, I am very sorry for the duplication.
Perhaps the reason no one has contacted Sean is that relatively no one uses WAR. Don't just count the media, but count their listeners, viewers, and readers. Count everyone. And when you do, hardly anyone has hear of this stat. The lack of use does not make WAR a bad stat, but it's just not mainstream. And the very very very very few who have heard of it... well... many many many of THOSE people have rejected it. Sorry, but it's true.
The number of people who have picked up the phone offering free publicity is roughly the same number of people who have read about WAR, have a question or two, and is willing to give this new metric a chance once issues have been resolved or fuzziness has been clarified.
Sorry to be the bearer of bad news, but almost everyone who follows baseball can be described in one of three ways in this discussion about WAR:
Never heard of it,
Read about it and don't like it,
Read it, like it, but I wouldn't use it as my number one way of evaluating players. (It's just another tool)
I won't get into the flaws of WAR here in this thread, as that is not the banner at the top.
As a member of the media, my sole purpose in writing is an attempt to explain a lack of "buzz".
It's just a guess, but I am pretty sure I have this one right.
"Life can't be broken down into numbers."
Congratulations on nullifying the entire scientific enterprise with one assertion.
Mark
you sound like you don't understand. Go to the reference material here or on fangraphs and read up on the newer stats. It will help you understand that they aren't just made up nonsense. Most of them are better at describing performance than their baseball card counterparts.
do you really think batting avg is better than OPS?
do you really think a guy who drives in 130 RBI's for a first place team is better than a guy who drives in 98 runs for a last place team? don't you think the quality of the offense around him might affect that?
Is WAR perfect? of course not, is it improving all the time? yep is it a BETTER number to look at to view TOTAL PLAYER VALUE than batting avg, rbi's and HR's? absolutely.
Your argument is that the MVP is Most Valuable Hitter, but I seem to remember the word PLAYER in their somewhere. and that means defense and baserunning.
Trout had a better year than Cabrera, played in a tougher division, AND his team won more games. Cabrera put up good numbers sure, but he didn't even have an elite season. He won some goofy old timey award based on stats that stuck during the horse and buggy days.
That's like judging a science project by how pretty the poster is... or giving the NFL MVP to the best option QB
learn the new stuff, it's not hard to understand, and it will open up your appreciation of true value.
(I guess Beane taking another low payroll team to the playoffs isn't enough evidence? nor the FACT THAT EVERY TEAM IN BASEBALL HAS STATISTICIANS STUDYING THESE NUMBERS?)
I just gave you some applause! You don't think competing algorithms striving to achieve the same goal is healthy and constructive? How does anyone claim that's perfect if they don't get the exactly same answer.
Human reason is in algorithm. You've complained before that the algorithm changes, now you're complaining that it doesn't? If you don't understand it, then don't use it, but the contributions are clearly laid out on the contribution page (link above). You can look at the data and decide for yourself. If you don't want to understand it, then why are you complaining that you don't understand it? But your complaint is that the people who do understand it are overselling it so therefore it must be bad?
Do you think the numbers will be revised in the future making Cabrera a better fielder and baserunner than Trout? I'd cite the two other competing websites that independently came up with the same conclusion but you don't want any evidence that involves baseball knowledge learned since WWII and this blog-post doesn't accept highlight videos.
"Life can't be broken down into numbers. ... BA/HR/RBI are still around."
I'm watching a recording of last night's Colbert Report as I read this. You crack me up. :-)
MVP Voters have been using a WAR-like framework for decades even if they didn't know it. How many triple crown categories did Yogi Berra ever lead the league in? He never led the league in anything at all. Voters gave him 3 MVP's based on positional adjustment and defense. Maury Wills won an MVP with triple crown numbers of .299/6/48 because voters care about baserunning. Plenty of other examples, but none of this means anything to you today because... why?
You can vote your gut feelings if you want. We've been through enough of these to know how things work. Its just like the Academy Awards, there's usually a "Should Win, Will Win" nature to these predictions. Sometimes they don't vote for the new guy because they don't have it set in their minds that he's good yet and think it might be a fluke. Sometimes they don't vote for the perennial candidate because he's had better years and they're sick of voting for him. Sometimes they vote for the guy with better teammates becaues his team went to the playoffs. Sometimes they don't vote for the guy who's teammates are too good because they would have won without him. Sometimes they don't vote for the guy who is a jerk off the field. Sometimes they do vote for the guy who WAS a jerk off the field because his rehabilitation makes a great story. Some of us come to baseball-reference.com to get away from all that but maybe that's just me. :-)
I have offered good-faith reasons (as have others) about reasons for disagreeing with the validity of WAR.
Not a single WAR fan has addressed these. One more time:
1. Different versions permit significantly different outcomes for the same player during the same time period.
2. Constant changes to the WAR equation;
3. Opaqueness of WAR mathematics;
4. Constant revisions allowing variations of 30 to 40 percent from previous results.
In simple terms, the three items above define unreliability in a mathematical model: variations, deviations, unreliability.
#114: You are, as I understand it, accusing me of being anti-science? If a scientist submitted WAR in a paper, given the constant revisions, he'd lose credibility rapidly. Scientists like consistency. WAR has none.
#115: You're making a lot of assumptions about me personally and about what I'm thinking. You're as presumptuous as you are wrong.
#116
"Human reason is in algorithm."
So, that's why Artificial Intelligence is so easy to program that everyone has it!
I stopped reading there.
One great thing about baseball is comparing players from different eras via statistics. Now, I know players are more athletic, and all the other arguments for players appear to be more athletic. But, true fans across generations ignore that for the most part and discuss Hits, Batting average, Home Runs, and RBI's. So, why bring in a new stat to muddle conversations between age groups. Miguel Cabrera IS the MVP!
When Brandon Moss replaced Daric Barton the A's were 24-31. They were 70-37 after that. When Chris Carter replaced Kila Ka'aihue the A's were 37-40. They went 57-28 after that. But WAR tells us that Moss was worth 1.9 wins and Carter 0.9. Not even 3 wins. So WAR fails in the sense of giving us a real life picture of actual value. Does anybody really believe the A's win 91 games this year with Barton and Ka'aihue playing the whole season?
Algorithms are written by people. There's no AI involved. This is not a learning algorithm.
Your complaints:
1. When the versions disagree, then you have a point. And if you understand what's going on you can analyze the biases of each implementation and see which one you agree with. But they don't disagree here. bb-ref says Trout is ahead 10.7 to 6.9. Fangraphs says Trout is ahead 10.3 to 7.3. BaseballProspectus uses a different scale, they have Trout up 9.0 to 6.1. Each of these websites would like nothing better than to outdo the other two.
2. I'd cut you some slack on this one if the difference was in one of the more volatile numbers (fielding). But the fielding contribution only validates what people see with their own eyes. And even if you cut the difference in half, Trout has a big lead. Even if you remove it, Trout is still up. Post #106 already explained this to you (had you stopped reading then, too?). The batting part of WAR has changed very little since the early 70s. Run estimatation based on plate appearance outcomes is a solved problem. Very minor tweaks since then. Park factors are over 30 years old and their calculation hasn't changed much.
3. There are a lot of details. They're all spelled out in the documentation. If you don't like summing them up into one number (a lot of people don't), look at the contributions page instead and cross-reference with other sites.
4. See #1. A flukey fielding number is not the difference between these two. If you look at the contributions page, you can take outlier numbers into consideration.
Did you stop reading again? Are we all wasting our time? Batting average and RBI still the king?
"truth never triumphs, it's opponents just die off"- Max Planck one of the founders of quantum theory
Like most fans who both love baseball and have an interest in advanced stats, I think the 2012 AL MVP vote will be very exciting. On the one hand, Trout clearly had the better all-around season as borne out by his outstanding top-ranked WAR.
But ... the fan in me looks at Cabrera and how he did something historic - the first man in 45 years - FORTY-FIVE YEARS! - to win the Triple Crown, and there's something romantic about achieving a feat that puts him alongside Cobb, Gehrig, Williams, Mantle, Hornsby, etc.
Yes, it's true AVG, HR, RBI are laughably inadequate at telling the whole picture of a batter's individual productivity but the Triple Crown is still nothing to spit at.
Mark,
1. Different versions permit significantly different outcomes for the same player during the same time period.
WAR is a model that tries to spit out a 'value' based on different inputs and different weights on those inputs. The inputs and weights are determined by the philosophy and methodology behind that specific version of WAR. So it makes sense that different websites with different methodologies will have different WAR outcomes. FanGraphs and Baseball Reference both have clear, distinct philosophies behind their models and the validity of both are still debated in the statgeek community. If anyone you've talked to has claimed that any WAR stat is perfect or infallible, then they're wrong. I just don't see that many people doing that.
If you could point out a specific philosophy, or a specific input, or a specific weight value that you disagree with, I have no doubt that many people will be willing to have a civil discussion regarding your concern. But to dismiss the entire concept of WAR because different models produce different results is a little silly.
#119 is saying
"leave me alone with your pockets, zippered dungarees and fancy horseless carriages!! they are a fad like jazz music and playing baseball at night!"
Just putting this out there...
Do y'all think the reason it took 45 years to see another Tri-Crown winner, is because in the DH era, there evolved an emphasis on power at the expense of making contact or getting on base?
Intuitively, there is no more shame nowadays in taking a ferocious swing with a 2-strike count because the potential benefit of achieving an HR outweighs the risk of striking out. In the pre-DH era, most batters would do anything to not strike out, thus improving AVG or OBP.
Also, I had a look back at the MVP voting tables for the years that the AL Tri-Crown winners did NOT also win MVP... If you want to be outraged, look at 1942 and 1947 AL when Williams lost both times even though his OPS was nearly 200 ahead of the MVP winner.
It's sad that nothing much has changed in mainstream media attitudes some 60-70 years later: "If you don't talk to me or give me an interview, I'll screw you over at the end of the year during the MVP vote."
Sad that the media was always sucking DiMaggio off for years and blindly ignoring the media-prickly Williams' historic greatness.
2. Constant changes to the WAR equation
Even by scientific standards, modeling something this complicated requires a certain extent of trial-and-error. It's not always obvious what variables completely capture 'value' or what variables might conflate the end result. A revision means new information was discovered, and then included into the updated formula. Shouldn't that be a good thing?
I don't know how many scientists you know, but those guys are constantly looking for ways to revise their stuff. There's nothing 'consistent' about the evolution of a research paper after going through peer-review.
David Cone and Michael Kay had a nice and very intelligent discussion of the Cabrera vs. Trout for MVP debate during Yankee game 162 last night. Cone in particular expressed his enthusiastic appreciation of WAR, with its ability to value defense along with hitting, and Michael Kay said it was time to advance the discussion beyond merely counting HR and RBI. Seems to me that RBI's have become an obsession of MVP voters for too long, so anything that breaks that logjam would be a plus. Anyway, not everyone in the media is anti-WAR, and it's nice to see smart former players like Cone speaking up for it.
That said, I wouldn't be too mad at anyone who voted for Cabrera for MVP. I was a teenager when Yaz won his Triple Crown, and I'm not so young anymore. Winning one of those under current conditions is a rare achievement, and if somebody want's to reward Miggy for that rare achievement, the way they rewarded Maris for his HR record in 1961 when Mantle was clearly the better all-around player, I'm not going to say no.
Trout and Miggy have had two of the best seasons you'll ever see. Voters can't go wrong with either.
Sabermetrics is defined as "knowing something about baseball that was not previously known". That's great, we all can appreciate that.
My question is, does WAR fit that criteria?
The 7th post states that Pujols' 2003 WAR on BR has changed from 10.9 to 8.4 and on Fangraphs has revised the same season from 9.0 to 10.1. When combined, that's a 33% change to the value of Pujols' season (-23% on BR and + 10% on FG).
33% would make a .300 hitter a .200 hitter. Now, I understand that WAR is a work in progress, as defined by the constant changes, but shouldn't that eliminate it from serious consideration until the kinks are worked out?
Don't get me wrong, I enjoy sabr stats. I love looking at the game in new ways and I love the BR site in particular, but I think we are a long way from having an accurate one-stop shopping stat to quantify the true value of a baseball player.
WAR is a nice idea that's been botched so far.
Initially there was oWAR and dWAR and you could add the two together to get WAR. That made sense... after all you are either on offense or defense. Baserunning is part of offense. In the myriad of changes the WAR is calculated differently by different people. In addition even within this site it is a shifting number as changes and revisions are made to seasons long completed.
It's become so poor that many people are fine with throwing the baby out right along with the bathwater. And that may actually be a reason that Trout will not win the MVP. Because the argument put forth on WAR is full of needless complications.
Look. Anyone decrying WAR has some mixture of either 3 points: 1) WAR doesn't work; 2) WAR is implausible for the average fan to understand/calculate; 3) they like additive/traditional stats.
#1 is ignorant. #2 and #3 are totally rational positions - just ones that *shouldn't* result in an attack on the "new math." But we are people and this is the internet, so it does....
Bill James has been quoted as saying that he recognizes that the HOF is not the "Hall of WAR" and I believe on some level the MVP has a similar charge. Look no further than Felix's Cy Young in 2010 to see the incorporation of advanced metrics into baseball. That said, you cannot begrudge a writer/fan if they think the Triple Crown winner and #3 AL WAR for a playoff team is a more deserving MVP candidate than the #1 AL WAR for a massively under-performing 3rd-place-in-their-own-division team.
There is no debate the Mike Trout was the best player in baseball in 2012. But Bill James might say that the MVP is not necessarily the "Most WAR Player."
#117 (below):
1. WAR uses statistical models which capture FAR more data than other methods and therefore need to have subjective calculations. Changes in these calculations can be helpful / unhelpful and can result in changing WARs - but that certainly doesn't mean that WAR < BA. That is a logical leap that you have not proven....
2. Again, the vastness of the data allows for subjective judgment in what is included (park factors, how to measure defense, batted ball data, etc.) and what weights are given to each category of data.
3. It is not opaque. It is complex - two very different concepts.
4. Wait, this is 1 point disguised as 4 - right?
Oh... and science is constantly being revised. Notable inclusions are the theory of gravity which existed for 100s of years pre-Einstein, the billions of $ managed by quantitative investment funds (which adjust their models) and astronomy, writ large.
----------
I have offered good-faith reasons (as have others) about reasons for disagreeing with the validity of WAR.
Not a single WAR fan has addressed these. One more time:
1. Different versions permit significantly different outcomes for the same player during the same time period.
2. Constant changes to the WAR equation;
3. Opaqueness of WAR mathematics;
4. Constant revisions allowing variations of 30 to 40 percent from previous results.
In simple terms, the three items above define unreliability in a mathematical model: variations, deviations, unreliability.
#114: You are, as I understand it, accusing me of being anti-science? If a scientist submitted WAR in a paper, given the constant revisions, he'd lose credibility rapidly. Scientists like consistency. WAR has none.
#115: You're making a lot of assumptions about me personally and about what I'm thinking. You're as presumptuous as you are wrong.
"1. Different versions permit significantly different outcomes for the same player during the same time period."
Yup. We can't model offense more precisely than to within 14-15 runs when looking at the team level. That translates to something close to 5 runs for the individual (full-time) player,
For most players the good offensive metrics will cluster around an area.
But yeah, it's true that every now and then you'll get a fairly big difference.
The big issue is on the defensive side. Teams have changed what they're doing (in particular shifting more frequently and more radically than they ever have in the past) and frankly some defensive methods don't seem to have a real good handle on this right now.
UZR largely throws its hands up on the whole shifting issue (as I understand it, they simply don't count plays when the shift is on). I haven't seen a PMR sighting in a while. Plus/minus doesn't count radical overshifts (like when the Jays play Lawrie in shallow right field)
Right now I personally don't use the full WAR structure. The offensive side is pretty solid (but I do like to look at EQA, VORP and Steve Mann's old version of linear weights) as are the positional and playing time adjustments. For the defense I work backward. How much better does a player have to be relative to his peers to make up the offensive/positional difference.
"2. Constant changes to the WAR equation;"
They only seem constant. We went through a major set of changes when Sean took over from Sean. WAR 2.0 was the result -- came out in May this year.
"3. Opaqueness of WAR mathematics;"
I don't see them as particularly opaque. Conceptually not all that different from what Pete Palmer was doing back in the early 80s (park effects are derived the same way). Sean is simply taking advantage of more information (reached on error stats, adjusting DPs for opportunities, baserunning) and doing the "slope correction" (adjusting for the particular year's offensive context) in a different way.
They aren't something that you can really do without a computer, but that's really true of any method more complex than OPS.
OK, you can't really replicate Sean's work without a retrosheet parser (or equivalent) -- though you can fake it using the table of weights Sean provides in the WAR intro. But it's not *that* tricky to go through the raw data to get all of the context info.
"4. Constant revisions allowing variations of 30 to 40 percent from previous results."
Already covered.
"In simple terms, the three items above define unreliability in a mathematical model: variations, deviations, unreliability. "
Sure. Any mathematical model has its imperfection. The standard error for WAR is certainly not less than a win (but it's not a heck of a lot higher than that). I'd be a lot happier with some disclaimers about that, but that's reality.
I think I know Sean well enough that he's neither blind to this nor sensitive to this. It really is as good as we can do right now.
The point is that any part is solid enough that an objector should be able to point to a specific issue (as for instance Brett Lawrie's defensive numbers earlier in the season) rather than just "doesn't look right" or "may be flaws"
I'll say this about the American League MVP award.
The Angels were 6-14 before bringing up Mike Trout.
The Angels finished with a better record than the Tigers.
and most importantly:
A team of nine Mike Trouts against a team of nine Miguel Cabreras for a 162 game season, based only on this season's statistics, would most certainly end with the Trouts victorious. People forget that speed and fielding actually matter in baseball.
"Initially there was oWAR and dWAR and you could add the two together to get WAR."
Sean has explained why he changed the definition of dWAR when he took over from Sean. Basically he wanted a structure that allowed for comparing the defensive value of guys playing different positions. Hence the built in positional adjustment.
Makes a lot of sense from that point of view even if it's confusing to people who didn't read the fine print last May (raises hand) -- or since.
I think that the potential for WAR to go awry is not only in defense, but in Park Factors and positional adjustments.
What "doesn't look right"
For most of the year, Darwin Barney was in the NL top 10 in WAR. This is a guy with a .653 OPS playing every day on a 100 loss team. I know he's a great fielder, but c'mon.
Ben Zobrist is another guy who seems to benefit from WAR. 1st in the AL last year and 10th this year, 6th among position players. He plays all of his positions well, not spectacular. He's a very good ball player but I'll put it this way, he wasn't close to last year's MVP.
Denard Span is the 9th most valuable position player in the AL, better than Pujols. Put Span in the center of the Angels offense and he gets about 50 RBI. Try making that up with his glove and legs.
Speaking of the Angels, I see Weaver is rated 10th among pitchers in the AL! 1st in WHIP, Hits/IP and wins and win pct, 3rd in ERA.
I also see that 3 of the top 10 position players are Angels(Trout, Torri Hunter, Pujols). It seems that these guys are getting credit for a portion of Weaver's WAR.
Baseball is played on the field and the teams and players have to respond to different situations. Teams do not play against a team of replacements, batters to not hit against replacements and pitchers to face replacements. They play against MLB teams and players.
Team WAR shows the Angels (38 WAR) and Brewers (35.1) as the two best teams respectively. So why not just award them the AL and NL pennants and the Angels the World Series title if the formulas have been finalized and perfected? The Orioles finished 26th (15.3) so it doesn't appear that WAR actually equates to Wins on the diamond.
Again I am not saying advanced metrics shouldn't be used to help value players, you just need to temper the hate towards anyone that doesn't agree with your analysis. Mike Trout had a great year and I can see why he is one of the favorites to win the AL MVP. However, when you look at the 10+ Triple Crown Articles on ESPN and SI, you would think he is the only one that should be considered. All but one article attacks Cabrera and promotes Trout. People in baseball know that Cabrera lacks in certain aspects of the game, but his combination of average and power is a rare and his triple crown should be enjoyed and celebrated, not crapped on.
I saw an article on ESPN where they rated his the worst triple crown season ever because of OPS+. I am sorry, but when you have 11 triple crowns in 45 seasons (average about 1 every 4 years), but then go 45 years without another and you are supposed to tell me it's easier to do now than in the past?
Bottom line is Value can not always be defined by numbers. That includes BA, HR's and RBI's as well as WAR. Cabrerra shouldn't get the MVP because he got the triple crown, but Trout shouldn't get it because he had the best WAR. I am just glad we will be able to see these two play for years to come.
@ #135 - Ron
I did read that May fine print. I just didn't get it. And if it loses me it will lose 90% of baseball fans.
Pujols has a career dWAR of 2.6 and career oWAR of 75.4 with a career WAR of 88.5
Most people are still going to say "where did these other 10.5 wins come from".
Right Brad. ESPN (Keith Law?) trying to sell WAR with Trout, claiming this great all-around player as their WAR poster boy.
Worst Triple Crown? Ugliest super model? The reason there hasn't been a TC winner is because there are almost twice as many players today as opposed to 1967 and prior. To win one when there are 15 teams in the league is special.
14, lol. I already moved the Astros!
Raker... re:#136
You mention Weaver having the 10 highest pitcher WAR. Get ready for this explination. Keith Law last year was saying how if he had a Cy Young vote, he was still up in the air whether he would choose Verlander or Sabathia since Sabathia had a better WAR (by .1) than Verlander. He explained that Sabathia was unlucky with a .318 BABIP while Verlander lucked out in that statisitic. If you didn't know that stand for Batting Average of Balls (hit) In Play. So HR's and K's don't count. The experts say this BABIP is just a chance statistic so they remove it and put a league average in to normalize a pitcher. So Weaver, who obviously had nothing to do with it, is only allowing an unusually low .241 BABIP (Velanders was .236 last year) while other pitchers that are ahead of him in WAR Hernandez (.308) and Darvish (.295) are just victims of bad luck and shouldn't be punished.
I tried debating, but Per Keith Law, it is a given that pitchers have no control over BABIP. The ironic thing is Batters are allowed to keep BABIP because it is a given that they do control a batted ball. In short...
Weaver allowing .241 and Darvish allowing .295 BABIP is purely random, while Trout having a .383 and Cano only a .326 BABIP is all because Trout is that much better hitter than Cano. You see why people that spend the time learning more about WAR that we become skeptical?
#136 Barney's probably the single most contentious ranking this year. 12 runs below average with his bat. I'd have expected a bit worse. The 10 times he reached on error helps a bit. Likewise his DP avoidance and non-SB baserunning. All tiny positives that are missed by many metrics. As a result WAR sees Barney as a better offensive player than (No biggie though VORP has him at +8 for instance and I think there's a good case that it slightly underrates him)
OK. So you've got a durable second-baseman who does a lot of the little things a little better than average. Not a horrible place to start from. The issue is precisely how good Barney was defensively. BIS has him at Mazeroski +. No other defensive system has him anywhere near that level.
The primary issue as best I can tell is how much credit Barney gets for the fact that the Cubs were unusually good at turning line drives into outs. And I honestly don't have an answer to that. I'd bet against him being an actual +30 true talent level, but even if it was just luck, it's luck that matters. TotalZone has him at "only" +17, but it's heavily regressed.
As for Weaver. "Only" 30 start, extreme pitcher's park. I don't see any basis for complaints that Verlander, Price, Sale and Harrison rank ahead of him by WAR.
Rodney? Lights out closing is hard to value. Certainly doesn't look wrong to me.
So the specific complaint would be precisely Jake Peavy.
Peavy pitched a little more, slightly less effectively on a per-inning difference. They really are of indistinguishable value.
Would I rather have Weaver than Peavy going forward? Of course. That doesn't change the fact that for this year Peavy's slightly greater workload helped offset the fact that Weaver was somewhat more effective while in the game. If Peavy was lucky, it still counts.
#141 It's a simple fact that variation in BABIP by pitchers is mostly luck. While among hitters there are huge variations. Most of the Three True Outcome players (IE high walks, high Ks, high HR) have to be extremely effective when they do connect or they're gone.
For pitchers, well it's a well researched matter. It's not that there is no variation. Dominant closers for instance as a group have very low BABIP. Flyball pitchers also tend to have a lower BABIP than groundball pitchers but as a group they also give up more bases per hit (IE more doubles and triples) and this almost perfectly offsets the advantage in BABIP.
That said, it's extremely important that Law's position on pitching value is very much a minority one in the stathead community. Most of us work from runs and adjust for the quality of the defense. Fangraphs pitching WAR is FIP based and
A) ignores aspects of run prevention that the pitcher has a role in (DP support, the running game for instance. Neither are random.)
B) credits all difference between expected hits and actual hits to the defense. Even when this assumes a defensive ability level that makes no sense (BABIP is not constant within a team's starters in any given year)
C) Works with expected runs rather than actual runs. First, while there has been a lot of work done on clutch hitting, there hasn't been that much done on clutch pitching. We genuinely don't know if there is such a thing as a true clutch pitcher.
But in a real sense it doesn't matter whether it's luck or ability. It counts.
FIP (and xFIP) are good predictive metrics. You can expect a pitcher's BABIP to regress to average over time. They tell you zip about in-season value.
Reply to Ed Dykhuizen Says:
October 3rd, 2012 at 5:12 pm
JB, your reaction proves my point exactly. You misunderstood most of what I said, then flailed furiously through looney tangents to try to fight off what you thought my points were. It's OK, you can believe whatever you want, I really don't care. I was writing that more to the other folks.
Your reply proves MY points exactly! I misunderstood because I didnt agree with you. Then you go back to demonizing when you dont get your way. For someone claiming to be writing to other folks sure seemed the opposite was true!
Looney tangents because you are wrong & cant look in the mirror & admit it. Yours was a truly sad & pathetic reply.
Old man acts old......
Proof that WAR is a garbage stat: Graig Nettles' WAR is better than Joe Torre's in 1971.
Take a look at the numbers. It is not even close. Joe Torre's season was infinitely better than Nettles in every way (both 3rd baseman).
WAR = CRAP
Frank,
You appear to be a man who believes defense doesn't matter at all. I suggest you consider Runs Created or OPS+ as your stat of choice.
Frank could also save himself a lot of time by judging players by what they do in the batting cage.
Once the actual game starts and the players have to take the field... with a glove on... to actually try to do something with the batted balls... at that point Frank is not interested in watching.
Admin., what I don't get is how do you go back to 1971 and definitivly decide what a specific player's dWar is? Baseball fans know Nettles was superb with the glove, but how do you come up with a number in the absence of hit trajectory, speed and location of balls hit?
It seems like just knowing that Ozzie, Belanger, Nettles, Brooks Robinson, etc, were great, influences their dWar somehow. I'm sure it's more than that, but if it's just number of balls fielded as opposed to other players, that's a weak basis to say that Nettles had a better 1971 than Torre, in my opinion.
We all know that fielding and base running matter greatly, it's that we're skeptical that those components can be accurately depicted in terms of runs saved, allowed and produced by using only play by play accounts.
Post #113 is spot on.
Thirty-three years after Bill James self-published his first abstract sabermetrics are still on the outside looking in.
Turning the AL MVP into a pissing contest about WAR is a sign of desperation, it reeks as nothing more than an attempt to stay relevant.
Cabrera "WILL" win the MVP, and he will for one reason..the Triple Crown.
It's a rare achievement, and it shows "offensive" dominance and consistency over a full season.
If Cabrera didn't win the TC, regardless of whether the Tigers made the playoffs or not, then Mike Trout "likely" would win MVP.
Trout had the better year, I don't think you'll find much argument in that, but "best" and "valuable" aren't always the same thing.
I'm an "old" guy, and would vote for Trout if given the opportunity. It's clear he had the better overall year, and the funny thing is I didn't need WAR to tell me.
Cabrera's going to win MVP next month, so best start the acceptance process now.
Chuck,
I'm not the one turning this into a pissing contest about WAR. Look at the articles I linked to above. The writers the articles above were the ones upset about WAR because they didn't like that people were considering it and that it didn't show them what they wanted.
Also regarding the TC showing dominance. Five singles. Your MVP vote is dependent on five singles.
#147 So let's look at Torre and Nettle in 1971.
Torre has a large offensive edge. He starts out 45 runs ahead, though nettles has a slight edge with DP avoidance and on the bases.
These numbers are pretty easy to defend.
Speaking of defense, here's where things get ... well not problematic, but certainly less certain.
We only have traditional fielding stats (plus known staff composition) to work with, but ...
Torre played third exclusively and is graded as just awful. Don't know about you, but I followed the game then and this definitely jibes with his rep. He was fairly new to the position and wasn't a well regarded defensive player. 25 runs doesn't seem outlandish to me. It's easy to imagine that stuffing (say) Ryan Doumit at third could cost a bundle of runs compared to an average 3B. Some catchers can make the move to third with ease (Brandon Inge for instance), but I can think of a fair number of attempted conversions that didn't go all that well.
Which brings us to Nettles. Nettles gets graded as brilliant. And frankly I don't think that's particularly contentious either.
That's primarily due to his very high assist totals and the number of DPs he started. Now it is true that Cleveland pitchers put lots of runners on, but Sean adjusts for opportunity.
30 runs above average is an amazing total (particularly since TotalZone is heavily regressed) but what we're talking about is a career (defensive) year from one of the best defensive 3B of all time.
Now I wouldn't bet my house on 30 runs. It is after all based on range factor and there are problems inherent in this. Generally speaking at the major league level you get a range of ~40 (defensive) runs between the best and worst at a position. This is higher than normal but not to any troublesome degree.
"Admin., what I don't get is how do you go back to 1971 and definitivly decide what a specific player's dWar is? Baseball fans know Nettles was superb with the glove, but how do you come up with a number in the absence of hit trajectory, speed and location of balls hit?"
Any number of people (Pete Palmer, Charlie Saeger, Bill James, Clay Davenport just off the top of my head) wrestled with this before Sean (Smith that is).
All have started with the basic assumption that good defensive players make more plays. So all are using what amounts to a range factor system and then they attempt to adjust for (in particular) pitching staff composition (because left-handed finesse pitchers will tend to have a markedly different distribution of balls in play than a right-handed power pitcher).
Before 1988 that's all we really have. From 1988-2002 we have more information and from 2003 on we have BIS data.
Because there is a fair degree of uncertainty based on range factor, the results are heavily regressed.
It's obviously not perfect, but if you look at the result you'll find a high correlation between contemporary reputation and TotalZone. And you'll generally find a career arc that makes sense (defense generally peaks early -- though there are plenty of exceptions)
"Thirty-three years after Bill James self-published his first abstract sabermetrics are still on the outside looking in. "
Actually you'll find a lot are on the inside. Gary Huckabay, Keith Woolner and Sean Smith (the inventor of WAR) are working for MLB teams (as is James himself). Craig Wright, Steve Mann are important figures in the sabrmetric community who have worked for MLB teams in the past.
Tango and MGL (and others) are consulting. Others (Dan Szymborski, Keith Law, Christina Kahrl among others) are working for ESPN or other media outlets.
Still very much a minority to be sure (though I'm pretty sure most if not all teams have a numbers guy)
Can I ask a question about how park factors are determined? Many of the previous posts have mentioned that Mike Trout plays in a much more pitcher-friendly ballpark than Miguel Cabrera. It's was stated above that the Angels park has gone from being neutral to a definite "pitcher's park", whole the park itself has not changed. Does the recent influx of high-quality pitching in the AL West have any to do with the change in park factor. It seems to me that any park will be more pitcher-friendly if Jered Weaver is on the mound, as compared to, say, Joe Saunders. If the Angels have above average pitching over a period of years, does that "change" the park to a pitcher's park? If so, then that doesn't really have an impact on Trout, as he doesn't have to face his own pitching staff. Just wondering.
Park Factors are a comparison of runs scored in home and road games, so weaver pitches on the road as well, so that will over time balance out.
Over the past two years, the Angels and their Opp. have scored something like 240 runs less in Anaheim than in the oppon. park.
Sean,
I was not at all referring to you, my apologies if it came across that way.
This has been an interesting thread to follow and I was speaking in more general terms than what was presented here.
Again, sorry.
Chuck
#155 Sean has the formula at
http://www.baseball-reference.com/about/parkadjust.shtml
That said, it conceptually simple. Compare runs scored and allowed in home and road games. (Devil's in the details and there are plenty of details to work out)
To answer your specific question though, more elite pitchers in the West "shouldn't" affect park factor. Teams don't get a completely random mix of opposing pitchers and in theory that could affect either the park factor or team run scoring.
Thanks for the answers.
There really is a huge difference in the Angels' home/away scoring. I guess that is why they have 3 hitters in the top 10 in WAR, because they recieve "credit" for the 81 games in an extreme pitcher's park. What struck me about the numbers was the numbers were the 20-25% increases in HR and BB on the road. Most of the other numbers were similar, except runs and rbi of course.
Torre vs. Nettles in 1971? If Torre is only 45 runs ahead of Nettles on offense, it's conceivable that it would be close to that difference on defense. To me, Nettles was as good as Brooks and that carpet Torre played on was fast so I'm sure that didn't play well with his lack of speed.
OK guys, thanks for the detailed responses.
#147 I should chime in to say that runs created and OPS+ are both mediocre metrics. Runs created has problems with extreme players (all multiplicative measures do) and OPS+:
A) under-values OBP
B) flat ignores all other aspects of offense but OBP and SLG.
(For all that, OPS+ has a standard error that is within noise of runs created for offensive players. Runs created works a fair amount better at the team level)
If you want a good offensive metric that you can do yourself with relative ease, extrapolated runs isn't a bad choice (linear weights without a slope corrector) and you can get EQA from Clay Davenport's site if you want a complex metric where the heavy lifting has been done for you. (VORP too, if you want something with positional adjustments built in)
Amazing how so many of you think you can determine how many more runs Nettles prevented at third base than Torre, with no actual way of measuring this.
Kind of reminds me of the Obama stat of "job's saved" when there is no actual way to calculate a "job saved."
Bottom line: Torre > Nettles in 1971 by A TON....and WAR = CRAP.
Thank you Chuck (Post 150)
If WAR supporters were matadors, the bull they are trying to avoid is this:
Awarding "runs" or taking away runs just because a player played a certain position. They see value in a below average shortstop, just because he plays shortstop. And they take away value from an everyday first baseman just because he is playing first base. Even if he plays the position well... too bad. He ain't a shortstop, so lets dock him.
But you HAVE to have a first baseman over there, don't you? Why take away credit from a first baseman or a left fielder just because he plays the position?
Consider the team with two really good shortstops. Only one can play there, so perhaps the other guy is forced over to second, or maybe third. He gets docked for that, NO MATTER HIS SKILL LEVEL.
This only makes sense if all the good middle infielders are evenly spread over both leagues. That's just not the case. And just because a player is over at first base for his team, that does not mean he is a terrible fielder costing his team "runs" and wins.
These additions and deductions are based on the lineup card before the game and not the completed box score after the game. The positional adjustment is what kills WAR for me.
The idea of studying the various values of hits, walks, extra bases, outs, double plays and so on make the offensive side of WAR very easy to defend. OPS+ is far better than just looking at "triple crown" numbers but it is clumsy compared to WAR. WAR as a measure of offensive value is great!!!
BUT I still see so much variation between differing methods used to determine fielding prowess that it is hard to foresee that it can be resolved anytime soon. To get a true value on catchers is going to be next to impossible. For the fielders in front of the bat, a complete record of the trajectory and speed and type of ball hit in every game is needed and a baseline established for every similar ball hit for every position to establish a baseline before you can confidently say that Brandon Phillips is better at second base than Darwin Barney or that Torii Hunter is better than Jeff Francouer in right field...except you also have to chart their throws, check on their positioning on every play to be sure they have backed up the correct base...not to mention the skills intrinsic to some positions - making the turn and feeding the ball to the pivot are skills particular to the middle infielders. How well do the corner infielders charge and handle the bunt, especially the third sacker? How good is the first baseman at making the scoop or otherwise at least saving bad throws from finding the opposing dugout? How well does the outfielder turn his hips and prepare to throw on a fairly deep fly that could score a runner from third? I could go on and so could you! We need computer analysis and great programming to make future judgments. For the past? Let's agree people like Willie Mays and Ozzie Smith and Johnny Bench were amazing and look forward.
It is too bad that Mike Trout will not be universally acknowledged as the most dominant force in baseball this year, overshadowed by an archaic set of statistics. Might as well agree that a Triple Crown winner on a playoff team is going to be a slam dunk and not fight it. But it would be fun to see what a guy who appears to be half Mickey Mantle, half Eric Davis could do if he had maybe 19-20 healthy years in him! Stay healthy, Mike Trout, please?
"Amazing how so many of you think you can determine how many more runs Nettles prevented at third base than Torre, with no actual way of measuring this."
No way of measuring this with any kind of precision, sure. But anybody who was on the scene would tell you Torre was an awful defensive player, and WAR's numbers for Torre are certainly in line with this. As I said before, you expect the worst defensive regular to come in at around minus 20 runs.
Again, Nettles' numbers are the exceptions. They're pretty much off the charts. But then he did make a ton of plays and start a really unusual number of DPs. There's just no way anybody but an exceptional defensive player does this.
I don't blame anybody for wondering about 3.9 wins on a lousy team, but the raw number of plays made really is that high. And by rep he really was exceptional (though as long as Brooks was breathing he was the gold glover)
#164 You're demanding a level of precision not required of offensive stats. It's roughly the equivalent of crediting a hitter a hit any time he hits a line drive right at somebody (Al Oliver always "counted" these unlucky outs as hits, )
In terms of defensive value we don't need to care why the play was made, just that it was. Yeah, the fielder ends up with credit for what may have been a managerial decision. Personally I'm fine with that. Others aren't. We are going to get some form of field f/x (probably multiples). Some teams pretty clearly already have something better than is publicly available -- as you can see with more frequent radical defensive alignments.
The really tricky part of defense is how to handle discretionary plays. IE those plays that multiple fielders can make. "Always" an out, it's just that sometimes one fielder takes an unusual percentage of those plays. (I think the first of the gridded systems -- DA -- dealt with this well. Large zones that overlapped. Didn't care who made a play in a zone, just that the play was made.)
Very late on this, but in response to #70:
I feel like I didn't state my point clearly enough. Okay, so you find WAR contrived (I must admit, I don't really understand your analogy or what your trade has to do with it). That's fine. I see absolutely nothing wrong with that. But why can't it just be, "I've tried, but I don't understand it and I don't accept it" rather than acting like the people who DO accept it are part of some holier than thou secret club? That's what made me respond the way I did. People act like sabermetrics are ruining baseball when the reality is that you're free to watch the game in whatever way you want to, be that through the prism of WAR or thinking that the Triple Crown is a Godlike measure of greatness. All I was trying to say is that the explanations as to how things work are out there, and so many people reject them without even attempting to understand it because it shakes up their line of thinking (Bill Madden, Joe Morgan, etc).
For what it's worth, there's plenty of traditional stats that *I* don't think make any sense (differentiating between "plate appearances" and "at bats," errors and how scorekeepers decide to issue them) yet they're there, and I've never gotten an adequate explanation as to how they work (maybe there is). Yet I don't consider the people who don't think twice about those things to be part of some exclusive club.
And if anything, my larger point was this: I don't think that the "Trout for MVP" crowd is arguing that he should win solely based on WAR (I've literally never seen this anywhere; I welcome anyone to prove me wrong), nor is the idea of Trout being MVP all that ridiculous based on traditional stats if you account for the fact that he was batting leadoff - surely he would have batted in more runs if he were hitting third or cleanup. It's right in line with say, Rickey Henderson's 1990 MVP.