Winning matches, is it luck or skill?

One of the most attractive parts of football is obviously that it is so unpredictable.  Who would want to watch full matches when the outcome is known before the kick-off? In the low-scoring game that football is, predicting winners is difficult. Yet, at the same time, some teams are genuinely better than others, they win more matches and are rightly provided better pre-match odds of winning. The fact that they don’t win all of the time proves that there is an element of luck involved.

This post will try to separate the two entities that determine who wins a football match: luck and skill. And in order to do so, we must agree on the difference between these two qualities. The key concept that makes a lucky team different from a skilled team is sustainability. Any team would be able to pull off a miraculously good performance in a single match, but to string good results together requires more than luck, it requires a certain level of skill. The better the result and the longer the string of good results required, the higher the level of skill needed.

So far, so good. But which factors represent team performance and are sustainable? Goals scored? Goal difference? Points won? Shots taken?

 

Shots, conversion and saves

The brilliant James Grayson looked into this matter and used a large data set, containing 702 back-to-back seasons, to assess the season-to-season correlation between offensive and defensive parameters that indicate a level of performance (goals, shots, shot conversion, etc.). This research raised more than a few interesting points, most of which are inferred from ice hockey analysis, where the following statistical measures are much more common.

First, the variable that showed the best correlation between one season and the next proved to be the total shots ratio, or TSR. So the best predictor for future performance seems to be the fraction of shots within a match that a team takes. This shows even more correlation between seasons than the number of points obtained by a team. Please check James’ Blog, where he explains this very well, and in more detail.

Secondly, two variables that are very important in deciding which team wins a particular game, both the shot conversion (what fraction of shots ends up in goal) and the saves fraction (what fraction of shots conceded does not end up in goal), show little or no correlation going from season to season. And this deserves some explanation.

 

Regression to the mean

It seems that the fraction of shots converted, or shooting percentage (Sh%), and the fraction of shots saved, or saves percentage (Sv%), are more influenced by luck than by skill, and much more so. This is shown with the introduction of the concept of ‘regression to the mean’. What this important principle means is that any outlying performance over a short stretch of games will tend to move towards the average for that parameter. James explains this concept very well on his blog, and Wikipedia serves those wanting the most detailed of explanations.

So, if a team shows an excellent Sh% over a season of games, think Heerenveen’s 16.7% shot conversion, or an excellent Sv%, like Vitesse’s 9.0%, this shows an unsustainable performance. Next season, Heerenveen is more than likely to suffer from a severe drop in Sh% and Vitesse to suffer from a drop in Sv%, based on this principle.

 

PDO

In ice hockey analysis, both Sh% and Sv% have been combined into a single stat, called PDO. Ice hockey stats have the nasty habit of being named after their ‘inventor’, rather than after what they measure and the term ‘PDO’ has been launched by Brian King, whose internet alias happened to be PDO.

PDO = 1000 (Sh% + Sv%)

That’s all. Simple as that, a better shooting percentage and a better saves percentage gives you a higher PDO. Most commonly it’s multiplied by 1000 to get rid of the small numbers, but that’s just convenience. Since one team’s Sv% rises when another team’s Sh% drops, the average PDO over a match, or a league, will always be 1000.

The key concept, as explained above, is that a high PDO is simply not sustainable and a low PDO will rise with more matches played. It allows an easy assessment of how much of a team’s performance is due to skill and how much to luck.

 

Counter-intuitive

Of course, it seems counter-intuitive to assume that individual goalkeeping skills don’t vary from team to team, but in James’ dataset of 702 back-to-back seasons, the Sv% from one season and the next showed a correlation of just 0.098. Click on that link for a nice graph!

Regarding shot conversion (Sh%), more or less the same holds true. The R2 value is only 0.150, indicating that Sh% regresses to the mean by over 60%, or in other words, that luck is a factor 1.5 more important than skill when it comes to converting chances. This fits well with an article by the excellent Mark Taylor, who used Arsenal’s 2011/12 season to show that shot conversion is neatly correlated with the match situation. In other words, if Arsenal are chasing games, their Sh% is almost half the rate that is it when they lead comfortably. Match situation may be more important than the skill of the player pulling the trigger, with the obvious caveat being that better teams (higher TSR) take more shots from leading positions and achieve a higher Sh%.

For the total shots ratio, things are different than for Sh% and Sv%. The TSR from one season to the next shows less than 13% regression to the mean, indicating skill dominates luck a factor 6 here.

 

Conclusion

So in conclusion, the short term performance of teams that we are used to study, and a season of 30-something matches is definitely short term in a sport as low scoring as football, leads analyists to focus on luck, rather than skill. Using the simple concept of the PDO (thank you, ice hockey analysts!) allows to separate (un)lucky teams from (un)skilled teams, while the total shots ratio (TSR) is the best representation of a team’s skill.

To round off this post, here’s a table of the 2011/12 Eredivisie teams and their respective PDO’s, Sh%, Sv% and TSR.

PDO TSR Sh% Sv%
Heerenveen 1057 0.474 0.167 0.890
Twente 1038 0.587 0.152 0.887
Feyenoord 1037 0.578 0.131 0.906
Ajax 1026 0.678 0.144 0.882
AZ Alkmaar 1026 0.586 0.121 0.905
Roda 1019 0.414 0.154 0.865
Utrecht 1010 0.456 0.122 0.888
Vitesse 1005 0.528 0.095 0.910
NAC Breda 996 0.457 0.093 0.903
VVV Venlo 982 0.369 0.106 0.876
Nijmegen 980 0.528 0.087 0.893
Waalwijk 980 0.498 0.098 0.882
PSV Eindhoven 978 0.680 0.134 0.844
Heracles 977 0.506 0.112 0.865
Den Haag 976 0.410 0.101 0.875
Excelsior 968 0.344 0.078 0.890
Graafschap 963 0.380 0.093 0.870
Groningen 936 0.545 0.085 0.851

 

Data for this table has been provided by Infostrada Sports.

15 thoughts on “Winning matches, is it luck or skill?

  1. Bart

    Great stuff! 🙂

    Am I thus right in concluding that PSV are unlucky but skilled and that Heerenveen are lucky and medium-skilled?

    That Ajax are lucky and skilled? 🙂 hehe, we always knew Ajax was lucky, thanks for the proof. 😉

    Sorry for the silly Ajax comment.
    Anyway, are you going to take these numbers, compare them to previous years and add some possible conclusions?

    Reply
    1. admin Post author

      Yes, Bart.
      Your short summary is correct. Or, in other words, the chances of PSV having such a mediocre season (by their standards) are very slim. Ajax is not that far from the 1000 mark, so might be in line to more or less continue their level of performance and Heerenveen’s Sh% is unsustainable and will come down next year…

      I will indeed take the concept of PDO and use it for future analysis!

      Reply
      1. Tom

        OK, but what constitutes a large deviation from 1000?

        You say Ajax was ‘not far from the 1000 mark’ at 1026. You also say PSV’s ‘chances of having such a mediocre season are very slim’, but they are at a PDO of 978 (or, 1000 − 22). Aren’t they closer to 1000 than Ajax? So how do you work out which numbers are far enough away from 1000 to mean something?

        Reply
        1. 11tegen11

          Tom, you are correct that my statements on Ajax and PSV are not correct. The Ajax comment was posted before I updated some numbers, and showed an incorrect lower number at that time (1009).
          The is no reason to assume that deviations below 1000 should be treated differently from deviations above 1000.

          As to how far away from 1000 is enough to be meaningful, that a tricky area, and also subject to some personal preference.

          The most important aspect here is the number of games (in fact the number of shots) involved in the calculation. The more matches, the less deviation is to be considered important. In the three match Euro group stage that is presently being played, a large difference from 1000 is seen in many teams, even with teams below 900 and above 1100. Over a full season, we see that Groningen’s 936 and Heerenveen’s 1057 are quite extreme.

          Reply
          1. Tom

            Yeah. Some clever statistician somewhere (not me) should be able to work out what deviation from 1000 is significant. Maybe it would be helpful to make available the standard deviation of the datasets you’re looking at?

  2. Simon

    Interesting on Heerenveen. I have done à small analysis of Bas Dost in comparison to other seasons and his sh% was much higher this season and is likely to account for a lot of the high figure you have. PDO and TSR are easily the most interesting metrics I have discovered to date and I hope to have some time in the next few weeks to investigate these further, particularly when it comes to previous seasons.

    Reply
    1. Paul

      Where do you get the data, to see for example the sh% of Bas Dost? I’m really interested in statistics and football, but I don’t know where to get useful data. The best site I found is http://www.whoscored.com , but it isn’t possible to see sh%, sv% or TSR.

      Reply
  3. Mark

    Very interesting stuff.

    What i’m wondering is whether PDO is in fact largely luck, or whether it also represents something like ‘form’. I can imagine that over multiple seasons form also regresses to the mean, while if PDO represents form there might be a clear link between the PDO of the last and the next match. In case of luck PDO of the last match should be completely unrelated to the PDO of the next match. Is there any data on this?

    Reply
    1. 11tegen11

      Mark,

      There no data out on this, but I know people have looked at it and failed to find any correlation. The PDO in match 1 simply does not predict for the PDO in match 2.
      This may be a nice example of findings that reject a hypothesis not finding their way out to the open…

      Reply
  4. Alex

    Hey, just stumbled upon this article and found it quite interesting. I would like to compute the data for German Bundesliga, but I am a bit confused about the exact definition of Sh%/Sv%. Do you take the total number of shots or the number of shots on target (thus excluding blocked shots, post shots and badly aimed shots) for the denominator?

    Alex

    Reply
    1. 11tegen11

      Hi Alex,

      Thanks for your interest in this intriguing method.
      For PDO analysis I prefer to use total shots rather than shots on target, as the number of shots in football is generally low compared to other sports, like ice hockey where PDO was introduced…
      Results using total shots and shots on target seem not to be different at all, so total shots simply leads to less irrelevant random variation.
      I take my data from Infostrada and they don’t include blocked shots. Otherwise, I’d probably remove the blocked shots anyway as they may disturb the picture by being unevenly distributed among teams. You can imagine superior teams having a higher share of blocked shots than inferior teams have…

      Please let me know what your findings in the Bundesliga data are!

      Reply
      1. Alex

        Hey,
        thanks for your reply. I will try to look specifically into Home advantage and predictive possibilities of the PDO/TSR idea. However, I will collect the data for the 2012/13 season matchday by matchday, so there will most probably not be any results before at least midseason. (Looking at Home advantage unfortunately means having only “half” the data, thus cutting significance.)

        Alex

        Reply
  5. Mark

    Thanks your reply. Do you know whether this also holds if you correct for opponents strenght etc? I would be very interested in seeing the results of people who tried this.

    Reply
  6. Pingback: Introduction to Soccer Analytics – The Guys I Follow | Mixed kNuts

  7. Pingback: Waarom pechvogel Rutten onterecht geslachtofferd is

Leave a Reply