Paging Will Hunting: ESPN’s Soccer Power Index

Soccer. Front Page,  ESPN.com. Great, right?

SPI

ESPN unveiled its shiny new Soccer Power Index (“SPI”) this morning and all I can say is…my head hurts. I think I am a fairly intelligent dude–college degree, former investment banker,  fairly well read–but I couldn’t figure out how the below formula helps me predict who is going to win the World Cup.

AGS = ((GS-OPP_DEF)/( MAX(0.25,OPP_DEF*0.424+0.548))*(AVG_BASE*0.424+0.548)+AVG_BASE

(Yes, this is an actual formula from the SPI “methodology” article.)

In what can only be called the continuing Americanization of the beautiful game, ESPN.com directed its stat geeks to concoct a complex formula that would serve as a predictor of international competition, player impact and the surface temperature on Mars at precisely 8:05am on June 12th.

First and foremost, I applaud ESPN.com for spending the resources on this endeavor, attempting to objectify the sport and elevating it to home page content.

But what’s the point of going to such lengths to come up with a less subjective who’s-better than who list? According to the SPI “purpose” article, “[the] SPI is designed to serve as a general guideline…it will start a few debates — but [isn't] expect[ed] it to settle them.”

Couldn’t that just have been done by compiling a “power poll” by a panel of “experts” or even readers of TSG? (The latter of which would certainly be more accurate.)

Considering it took two lengthy articles (which I did read) to explain the approach and methodology complete with competitiveness coefficients, recentness factors and composite ratings, I am not inclined to argue the methodology of the formula. One read-through, however, does raise a lot of questions.

In general, the complexity of the methodology in which so much is thrown into a black box that spits out a rating does lead to a lot of subjectivity around what is and what is not counted (and how) which inherently makes this less objective. Not to mention the fact that the SPI is a product of  a former baseball stat geek; though Nate Silver is a well-respected statistician in his own right.

spi rankGetting out of the weeds a bit, perhaps the better question is this: Can the quintessential team game filled with nuance be explained solely through statistics?

The answer is some shade of “no,” but in an American sports landscape filled with PER numbers (basketball), VORP stats (baseball) and QB Ratings (football) ESPN.com thinks that’s what soccer needs in order to sell it to the American public in advance of the World Cup.

The SPI as an arbiter of team strength will no doubt attract some American stat geeks to take a closer look at the sport, but 8,000 words of explanation about a single number doesn’t make the game more accessible to the curious sports fan wishing to kick the tires of a “new” sport.

At the end of the day, soccer fans have another list to debate in addition to the FIFA rankings. And the Will Huntings of the soccer world have some new luscious numerical details to delve into.

To the right is the top 20 of ESPN.com’s SPI (full list at the link). Brazil and Spain come out #1 and #2, respectively. England, after missing out on Euro 2008, but sailing through WC qualifying ranks 3rd and the US sits in 14th (4 spots ahead of Mexico).

Of note on the United States is that the SPI predicts the USMNT will score 2.30 goals “against average teams” yet in reality, the US has only eclipsed the two-goal mark 5 times in their last 24 contests.

Let the arguing begin.

About these ads

22 responses to this post.

  1. Posted by matthewsf on 2009/11/11 at 11:50 AM

    Haven’t read the article yet Mark beyond “intelligent dude” — you lost me right there.

    From,
    Your brother

    Reply

    • Posted by Mark T on 2009/11/11 at 11:53 AM

      I modified it with “fairly” which is a word open to wide interpretation.

      Reply

      • Posted by matthewsf on 2009/11/11 at 11:56 AM

        I’d go with fairly….case in point: who insists that their brother try In-and-Out burger for the first time a half hour before playing a competitive basketball game.

        Fairly, open to interpretation, works for me.

        Reply

  2. Posted by Mark T on 2009/11/11 at 11:54 AM

    Here is another take on the SPI from the WSJ that attempts to explain it simply and has some comments from Grant Wahl.

    http://blogs.wsj.com/dailyfix/2009/11/11/the-count-espns-new-soccer-rankings/

    Reply

  3. Posted by matthewsf on 2009/11/11 at 11:54 AM

    Good points…I surmise that ESPN went with a statistic power poll because their coverage is so inept that they couldn’t be considered a panel of experts.

    Mark, we need to come up with the TSG experts and columnist (yikes, does that include us?), rating system and let the chips fall where they may around say….Harkes, Wynalda, Wynalda’s bald friend on Fox Soccer Fone-In who seems afraid of Wynalda and the one-and-only Alexei Lalas.

    Reply

  4. Posted by Timmy on 2009/11/11 at 11:59 AM

    Sounds like the BCS ranking system. Oh, but at least we have a tournament (WC) to decide a champion… Any new/additional coverage by ESPN is great in my books. 4 teams from South America in the top 10 seem high. I think Italy deserves a higher SPI ranking.

    Reply

  5. Posted by kaya on 2009/11/11 at 12:07 PM

    OK, I’m intrigued enough to read the espn 8000 word explanation… later.
    That’s simply high-larious. Gotta love americans…
    When traveling abroad I’ve often tried to explain to people I meet along the way why americans can’t seem to get into soccer. I often fall back to explaining how much we love making boring sports interesting with stats. (Sorry, but I find this to be the only possible justification for baseball.) Since soccer is difficult to quantify with stats, ergo it’s not popular… and then they go and come up with this.
    Awesome.

    Reply

  6. Posted by Matt B on 2009/11/11 at 12:11 PM

    While I don’t necessarily agree with some of the assumptions this formula uses, it is definitely a huge improvement over the FIFA rankings. I can’t believe the FIFA rankings don’t include things like margin of victory and home-field advantage.

    Reply

  7. Posted by kaya on 2009/11/11 at 12:15 PM

    Well, the beginning of it already explains what a lot of people who don’t watch soccer probably don’t know. The question is: how many of them are going to read this?

    Reply

  8. Posted by matthewsf on 2009/11/11 at 12:15 PM

    Seems kind of odd that England is #3….
    Honestly, how do you put England above Germany at this point. Only because your editorial staff resides there?

    Two words:

    R-O-B-E-R-T G-R-E-E-N….c’mon.

    Reply

  9. Posted by kaya on 2009/11/11 at 12:16 PM

    I’ll take FC Porto. Does that mean we get the incredible hulk?

    Reply

  10. Posted by ChrisR on 2009/11/11 at 12:45 PM

    ‘Not to mention the fact that the SPI is a product of a baseball stat geek.’

    Seriously? Nate Silver knows a little more than the value of OPS vs. Batting Average (you guys surely know about http://www.fivethirtyeight.com). I have not read the ESPN break down on the methodology but I think it is great that ESPN got Nate Sliver to do this. Who knows what type of promotion will work but at least ESPN is giving it a go trying to promote SA2010.

    Reply

    • Posted by Mark T on 2009/11/11 at 12:51 PM

      Silver was baseball first, then politics. Either way, you’re right, my comment was a little more dismissive than it should have been. (see update)

      Reply

  11. While I don’t care to delve into how they got their results, I think it did an acceptable job. Though, I do take issue with England being ranked 3rd, they haven’t done squat on the world/european stage in quite some time (could just be everyone’s inflated expectations). The US as #14 is tenuous at best, I don’t know that we’d beat Serbia, Cameroon, or Paraguay in a one-off on our turf.

    Reply

    • You gotta read how Nate does his things before you can understand them. This test heavily weight current form at the expense of previous success. That is why england is so high at the moment.

      There is minimal, if any, “subjective” injection into the formula. Therefore, speaking of “inflated expectations” is inappropriate in this context.

      I really should write a “plain language” guide to this ranking system.

      Reply

      • And I should stop using “quotes”. Hey Matt/Mark can we get an edit button on our comments?

        Reply

      • Posted by Mark T on 2009/11/12 at 9:01 AM

        As I read it the formula it can go back as far as eight years depending on the team due to the dearth of information. Most teams data used is from the preceding four years. But overall it is waited towards “recentness”. (see, I can use quotes too!)

        My point in the original post about it being subjective was not that subjective data was being used. Rather what data is counted and what is left out is subjective. So, in such a complex formula there are many factors that were subjectively excluded or included (along with their weightings).

        I am guessing Silver had to tweak the formula multiple times to get Spain and Brazil up top and then let the rest fall where they may.

        Reply

      • Fair enough that recent form is rated higher than past successes (of which England don’t have many), but was England’s qualifying group that difficult? Sure Croatia were a decent side, and the Ukraine has a chance to make it to the finals again, but does beating Belarus and Kazakhstan twice in the last two years for a total of 4 victories really mean you’re a quality side.

        Sure the Qualifying schedules of most nations have a few cupcakes (I didn’t even mention Andorra was in their group, and sure we have Trinidad and Tobago), but I don’t see England being the third best team in the world currently just because their recent record shows 6 victories against sides that wouldn’t have made it into the Hexagonal in our region.

        Reply

        • Also, I must’ve glossed over this the first time, but how is Argentina 5th? Sure they’ve got shloads of talent, but they barely qualified.

          Reply

        • Posted by Pckilgore on 2009/11/12 at 8:19 PM

          Probably because the individual players talent as judged by their club performance is included in the ranking too.

          Look as a soccer fan, I’m not too optimistic about how sucessful this thing will be. As a stats/math nerd, Nate Silver is like a god – ive found out more than once he’s right a whole lot more than he is wrong. It will be interesting not only to see how this plays out over the next few months, but his thoughts on why it is/isn’t working could be pretty interesting. Soccer, as a whole, is pretty archaic when it comes down to determining the relative worth of players/teams compared to their payroll. Maybe this will teach us something new!

          This gives me the idea, I wonder what the “payroll” ranking of FIFA teams are. As in take a weighted average based on minutes of the salaries of the players and see how the rankings pan out. Could be interesting….

          Reply

  12. Put your money where your mouth is. Start the power poll!

    Reply

  13. [...] the Quantification of Soccer by ESPN… Given the piece TSG wrote about the Soccer Power Index (SPI), TSG asked the exec if ESPN was focused on quantifying the sport as a way to build the American [...]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 251 other followers

%d bloggers like this: