This piece originally ran in August of 2012
What happens when TSG reaches out to a bunch of nerds in high places who like numbers, triangle passes, Barcelona analogy misappropriations, Dax McCarty data, Roger Espinosa heat maps and deal in things called “machine learning” & “regression analyses?”
A massively data-erotic soccer column on game and player analysis that lies somewhere just left of Shelter Island and A Beautiful Mind, but well beyond Good Will Hunting.
Now our actors around the roundtable. NASA “Curiosity” seekers don’t have squat on these guys. They are:
» Steve Fenn: Steve is a mathematics lunatic (complimentary) and tweets at the poignant handle Optahunt. Steve also now writes for TSG & Big D Soccer … when Opta Chalkboards are not online. Welcome Steve, the John Nash of these proceedings. (Follow on twitter)
» Rui Xu: Rui is currently in his second season as the Performance Analyst with Sporting Kansas City. He graduated from the University of Southern California with a degree in Economics. Rui is like the Stu Ungar of this roundtable, pre-extracurriculars of course. (Follow on twitter)
» Alex Oshansky: Alex runs numbers for an investment fund in LA by day and makes soccer spreadsheets at night. This makes him the
Bernie Madoff Blue Horseshoe (about to become the) Warren Buffet of the soccer world. He already is here in this column. Alex also writes for TSG and also on his own blog, Tempo Free Soccer. (Follow on twitter)
» Devin Pleuler: Devin is a computer science graduate from Wentworth Institute of Technology in Boston, where he played on the men’s varsity team as a goalkeeper. He’s writes the Central Winger analytics column for MLSsoccer.com. He’s a TSG alum and now a member of the MIT men’s soccer coaching staff. So he’s obviously Will Hunting here. (Follow on twitter)
TSG: Okay, what is the biggest misconception in what appears to be the growing market of data processing and statistical evaluation in soccer?
Xu: I think the biggest one is that I don’t think we’re going to get the granularity that we get in baseball, and I don’t think it will surpass scouting in terms of importance when it comes to opponent scouting.
The BEST case scenario for soccer statistics is getting to where defensive statistics are in baseball., with Ultimate Zone Rating (UZR) and Defensive Runs Saved (DRS), which are known to take a long time to stabilize, and are very iffy to use in small sample sizes.
Olshansky: I don’t know if it it is a misconception exactly, but it seems like the analytical community is only just scratching the surface of what we can learn about soccer. The field is in a nascent stage, and I think some perspective regarding just how far there is to go is needed. Graham MacAree of SB Nation had a great treatise on the state of the field and what still must be accomplished.
For example, only recently have some analysts started to realize just how few crosses are actually completed (for some teams only 10%). What does this mean exactly? Is crossing now a bad strategy? It’s just one example of how little we really understand.
Fenn: Underestimating the value of context. Biggest culprit within that issue is pass completion without adjusting for pass difficulty or value. Devin’s recent Central Winger columns have been nice steps toward better understanding.
TSG: What is the one stat–in your expert opinion or through empirical review–that is the most mis-used stat in soccer?
Olshansky: I’m not the first one to say it, nor will I be the last, but possession % is still used far too often to denote who is outplaying whom. Devin has written about how it is often more a defensive statistic than an offensive one and is highly dependent on game situation.
Fenn: For me, the most misused is also the most used. Whether he actually said it or not, I agree with the Jonathan Wilson quote, “Goals are overrated.” Again, context is key, and Sam Green’s work on shot quality is a great way to gain better insight into scoring and goalkeeping.
Pleuler: Possession is undoubtably the most mis-used stat in soccer. Arguably, this is because it is significantly more nuanced and complicated than it is often made out to be. Since the number of possessions during a game is finite, it is in the best interest of the more efficient team to increase the rate of possessions. This is simply because there is less variation the larger the sample size. Conversely, after a team has scored, it’s in their best interest to decrease the number of remaining possessions in the game. The less possessions, the less chances there will be for your opponent to score.
Barcelona does this by gaining and holding possession for long periods of time. Stoke does this by sitting deep in their own zone and becoming comfortable with their opponents possessing the ball for long periods of time. Barcelona and Stoke are attempting to do accomplish exactly the same thing and yet we view one style as artistic and the other as anti-soccer. One of these styles artificially inflates possession percentages, and one artificially deflates them. To talk about possession statistics without acknowledging this kind of context is really really bad.
Xu: By a GIGANTIC margin, possession percentage. First of all, it doesn’t even describe what it claims to; it actually has nothing to do with ‘time;’ it’s calculated using team passes divided by total passes. The data providers are using pass volume as a proxy for time in possession because it’s easier to calculate. Graham Macaree has a great article on that here.
Secondly, nobody ever provides any context on possession percentage. Statements like “When the score is tied, the average team’s possession percentage is 50%. When they’re up by one, the average team’s possession is 40%. When they’re down by one, the average team’s possession is 60%” are actually pretty easy to find, and they tell WAY more of the story than just possession percentage by itself.
Finally, it’s unclear a) whether or not possession percentage is reflective of the scoreline at all, and b) whether possession percentage is reflective of the scoreline, or the scoreline is reflective of possession percentage. Did the team lose 1-0 because they had a lower possession percentage, or did they have a lower possession percentage because they lost? There are just too many unknowns and ambiguities for it to be as ubiquitous as it is.
Xu: Well, goal differential, but I’m guessing that’s not what you’re asking.
Essentially what you have to do is create an “expected goals” and “expected goals against” model, and compare that to your actual goal production to see if you’re overperforming or underperforming
You do that by finding inputs that create goals. That’s fairly simple: quantity of shots and quality of shots. Then you find inputs that create more shots and create better shots. For example, crosses increase the number shots that a team takes, but those shots tend to be of poor quality. 1-on-1 vs the goalie have a very high probability of scoring, but they don’t happen that often. You find all of these, input them into an “expected goal differential model,” and you ‘ll get a very, very good proxy for how good your team is actually playing.
Olshansky: I think if someone could create a concise and logical way to rate the quality of chances, that would be a very useful statistic. Soccer is a game that often comes down to maybe five chances per team per game, so the ideal metric would assess a team’s chances of winning as: (average quality of chance X chances) – (opponent average quality of chance X chances). Of course there are a lot of factors that go into rating the quality of a chance, but I don’t think this hypothetical statistic is a pipe dream.
Fenn: Chances created adjusted for shot quality for offense. Quality of shots allowed on defense. Any of our answers would have to stand up to rigorous testing before claiming “best proxy for winning,” though.
Pleuler: This is a difficult question to answer because so many metrics are built directly on top of one another. Expected goals (and goals against) is perhaps the most obvious choice. But, you can’t calculate expected goals without quantifying the quality of each goal-scoring opportunity. You can’t accurately quantify the quality of a goal-scoring opportunity without quantifying the defensive pressure applied during that opportunity. You can’t quantify defensive pressure unless you quantify defensive shape, game context, and a handful of other complicated metrics.
TSG: Ponder this:
Coaches tinker with line-ups all the time. If you employed a marketer or “business optimizer” in the role of coach, they might resort to a notion called A-B testing to improve the line-up. That is remove one player and replace him or her with another as a means of finding the best combination of players on the pitch. However there is a principle called multi-variate testing that allows for changing many variables at one time. Can this be applied to soccer in any way?
Fenn: Seems highly relevant to lineup selection and substitution patterns. Such analysis could shake up timing and nature of such decisions, which can be staid and predictable.
Xu: Yes, but with the caveat that you actually need to have a performance metric for the players in place. Right now, we don’t even have a publicly available equivalent to batting average in soccer, let alone a true, accurate proxy for talent like WAR is in baseball.
Once you have that, you can start to do some pretty interesting things with multivariate analysis. I’ve mentioned a couple of times that I think synergy is an underrated aspect of soccer, so even if Player A is an absolute better player than Player B, it is possible that Player B has qualities that synergizes with his teammates, creating a better team overall.
Olshansky: Theoretically, yes. But I think we are still a long way off from having a model that could accurately assess multiple variables. Honestly, I don’t even think we are at the point where someone could tell me how many fewer goals the LA Galaxy would have conceded if Omar Gonzalez had been healthy the entire year.
TSG: What new metrics are important or would you create? Do any of these (link) make sense?
Pleuler: Your metrics make a lot of intuitive sense, but they are mostly things that can be derived from base metrics that are already being collected. As a community, I think we are beyond the level of counting instances of particular on-the-ball events and should be looking at quantifying off-the-ball events. That’s where the future is.
Fenn: Using Smart Soccer technology I’d create a metric for off-the-ball positioning relative to opposition, teammates, and the ball. Very easy for observers to be unable to fully and acurately appreciate or criticize contributions in this area.
Olshansky: First of all, I like the SSF (Stupid Stupid Foul) idea. I still cringe thinking about Fabian Johnson bringing down Marco Pappa when he had no support and two other US defenders seemed to have him covered.
I have been tinkering with, and quite like, the idea of a Forced Turnover (FTO) statistic for defenders. It’s more than just winning tackles; it also includes clearances, recoveries, and interceptions. Would be interesting to see the MLS leaders in FTO.
Xu: Win/goal probability is the big one for me. Every single action on the pitch is trying to accomplish one (or both) of two actions: scoring a goal or preventing a goal. Therefore, every single action has a specific value to one of those two ends. I want to know those values. Unfortunately, that is an incredibly time-consuming endeavor because that would require watching every single MLS game and noting the position of the ball and every player on the pitch at every single moment of time, but it is certainly possible.
The benefit of a stat like WPA/GPA is that it automatically captures the entirety of a player’s work. A lot of the concepts in your statistics are sound, but for me, I want to know all of a player’s value so I can determine if he’s overpaid/underpaid, etc.
TSG: One could argue that the real revolution in statistics is somewhat over now. Everyone–if they choose to spend the money–can dig deeper into “data.” The real revolution will be in the coach or manager or general manager who can combine statistical data into metrics and optimize on a per game or per player basis. Is this valid?
Olshansky: Somewhat this is true. I think the biggest issue is figuring out what to make of all the data. The real competition going on is among people like Rui, who are tinkering on spreadsheets trying to find wins amidst the avalanche of OPTA and Prozone data.
There is a certain game theory aspect to teams starting to incorporate statistical analysis more: how have the A’s done since their “Moneyball” halcyon days of the early 2000’s? How are the Rockets doing? The problem is that as soon as any metric becomes useful other teams will replicate it, rendering that initial advantage useless. This will be interesting to see played out going forward.
Fenn: Generaly agree, but I think a big part is analysts making their work accessable to coaches and players. A brilliant theory doesn’t help if it can’t be put into practice.
Pleuler: The revolution in soccer statistics will undoubtably take just as much social engineering as real engineering – and we’re very quickly reaching the threshold where the social barriers are just as daunting as the technical ones. But, this doesn’t mean that very significant progress still needs to be made on the technical side. The “revolution” may be over – and that depends on how we define that word – but the hard work is undoubtedly still going on.
Xu: I’m not as sure about this one as I was when I was first starting this job. When I first started, I was confident that, like, baseball, you’d be able to develop conclusions using statistics quicker than using your eye, but that’s not true in soccer at all. It takes maybe a game to tell that Lionel Messi is the best player on the pitch, whereas statistically, it might take a few for all of the noise to cancel out. It’s actually pretty rare that I go to the coaches with something completely new about an upcoming team. Usually, I’m backing up an existing theory, or it’s something they already had an inkling about.
The real advantage of metrics is in player evaluation and roster decisions, in my eyes. With performance analytics, you can put together a more efficient team, or you can evaluate if you’re making the correct draft choices, or making the right transfer moves. Those are the macro-level decisions that really impact an organization.
TSG: Justifying it anyway you wish, who is your favorite player and/or what is your favorite stat about said player?
Olshansky: Vinnie Jones, he of Lock, Stock, and Two Smoking Barrels and Snatch, holds the EPL record for quickest booking: 3 seconds. Art imitating life?
Xu: Aurelien Collin. Because, c’mon. Dude’s crazy, and he’s also an incredibly kind and generous person.
I don’t know if it counts as a ‘stat,’ but I absolutely love looking at Roger Espinoza’s heat maps for games. He worked really hard in the offseason on conditioning, and it shows in his work rate.
Pleuler: I very much like Dax McCarty of the New York Red Bulls. I always seems to find him creeping toward the top of the different types of analysis that I do. As for a favorite statistic about a player – I don’t have one. It would probably be a bad idea for me to become attached to certain statistics because I want to be able to throw them away when I find something better. I do, however, think goalkeeper throw-ins are hilarious.
Fenn: Age is a stat, right? Good, then it’s @InfostradaLive after Messi set the Barcelona goal record: “At 27, Alexander The Great overlooked his empire and wept bitter tears because there were no more worlds left to conquer. Lionel Messi is only 24.” If this is hypocritical to my earlier context rant, I guess I’m like Private Joker’s peace sign next to “Born to Kill.” I think I was trying to say something about the duality of man. The Jungian thing.
TSG: Do you think there will ever be a single metric that accurately scores the full value of individual soccer players?
Pleuler: There already is a single metric that articulates the full value of a soccer player. It’s salary. The problem, of course, is that our method of calculating salary it is far too dependent on subjective opinion. What further complicates the measurement of player value is that players are worth differently to different teams.
Fenn: The higher priority is lots of smaller metrics evaluating individual skills. With those a coach could target specific abilities needed for his team’s style of play. Weighing and summing individual metrices would be my preferred approach to an overall player rating. Would such a metric definitively say if one player is better than another? Probably not, but it could be good enough to revolutionize transfer and salary pricing.
Olshansky: Accurately? I don’t know. Probably some day, but the arc of time is not on the contemporary soccer analyst’s side. I think it will take many more years of data, experimenting, etc. before a truly accurate individual metric system emerges. I do think a good approximation is not far away. At this point, the best statistics are only useful for comparative purposes by position and are most useful for attacking players, but I am hopeful that defenders may still yet get their full due.
Xu: I do. I think WPA/GPA gets very close, and we’re not super far away from that. It’ll be tough to break down in terms of a player’s individual characteristics (aerial ability, tackling ability, finishing ability, dribbling ability, etc.), but it captures a player’s totality, which is perfectly usable.
[end roundtable] Thanks guys.