The ‘Lokhoff’ effect & The ‘McClaren’ non-effect

The previous post introduced the concept of PDO, a statistic that was originally introduced in the NHL a few years back, to determine which share of a team’s performance can be attributed to non-sustainable performances (‘luck’) and which share to sustainable performances (‘skill’). Since it’s an important concept in order to understand the following graphs, let me provide a short summary of PDO. A more detailed explanation can be found in the previous post, where lots of links are provided to suggest further reading on the topic.

In short, PDO combines a team’s shooting percentage (Sh% = the fraction of goal scoring attempts created that is scored)  and a their saves percentage (Sv% =  the fraction of goal scoring attempts conceded that is scored). It’s as simple as that.

PDO = 1000 * (Sh% + Sv%)

Does a high shooting or saves percentage gives your team a good chance of winning the match at hand? Certainly! But does it provide a solid base to build your future on? No, definitely not.

The single thing  to get your head around in order to get to grips with the concept is that what constitutes Sh% and Sv% in future matches is for almost 90% random and only just over 10% correlated to the previous performance. Again, see previous post

Now, what can we rely on then? Of all representatives of a good performance, parameters like points won, goals scored, shots taken , etc., the best correlation is found for the total shots ratio, or the fraction of shots in a particular match that is taken by your team. Future performances show over 60% correlation with historical data, making this the best predictor for future performance…

 

The ‘Lokhoff’ effect

Now, with the theoretic part out of the way, let’s get to the fun part. The graph below is a simple illustration of VVV’s points per game (PPG) over the course of the 2011/12 Eredivisie season.The graph shows the cumulative average points per game, restarting when the new manager was appointed.

Managed by Belgian Glen de Boeck, VVV found themselves struggling to avoid relegation and finished the first half of the season on the 17th place in the table, five points behind the safety of the 15th spot. De Boeck was fired, and experienced manager Ton Lokhoff was installed around Christmas time. With the season now over, VVV finished in 16th place and came one point short to avoid the relegation play-offs…

Points per game (PPG) under De Boeck (first half of season) and Lokhoff (second half of season)

In terms of PPG, the installment of Lokhoff certainly coincided with an improvement. During the first half of the season, managed by De Boeck, VVV obtained 0.59 PPG, never topping the 0.65 mark after more than a few games had been played. Under Lokhoff, however, VVV lived the dream and started out brilliantly, before settling down to finish at 1.24 PPG. With that rate, they would have finished a full season at 42 points, enough for a comfortable 12th place. Note that the graph, as well as the next ones, use a cumulative average over the course of the season, and that this cumulative average is reset upon the installment of the new manager. In other words, the number shown at match day five is the average PPG over the first five matches and the average shown at match day fifteen is the average over the first fifteen matches.

 

Luck or skill?

The key question now is, was De Boeck unlucky when managing VVV, or did Lokhoff really improve the performance. In order to answer that question, we can look at the PDO and the Total Shots Rate (TSR) for both parts of the season.

VVV's luck (PDO) and skill (TSR) in the first and second half of the season

In this graph it is well illustrated that the season started out with the PDO developing pretty close to the 1000 mark, which indicates a neutral amount of luck came VVV’s way. Later on VVV rode an unlucky series, with the PDO slowly tailing off, even as far as to finish the first half of the season at a dramatically low 970. The TSR, meanwhile, reflecting the sustainable level of performance, was ever increasing throughout the season and finished at 0.333, not high in absolute terms, but steadily rising as a sign that De Boeck got his team improving as he went along. The combination of an improving performance and the dropping PDO resulted in the rather flat PPG.

Under Lokhoff, VVV started brilliantly in terms of their TSR, but as more matches were played this extreme level of performance proved unsustainable, but they still showed an improvement over De Boeck’s 0.333 and finished at 0.400. However, following the trend in TSR under De Boeck, VVV might well have finished the season around the 0.400 level too!

A short bump in PDO helped VVV to their excellent start under Lokhoff. All in all, they finished the second part of the season with a PDO of 1010, mostly due to a rise in the final three matches, thereby compensating for the decreasing TSR, overall resulting in an upkick in PPG in the final three matches.

 

The ‘McClaren’ non-effect

VVV showed an remarkably different pattern of performance under both managers, as indicated by the rising TSR under De Boeck and the falling, though higher, TSR under Lokhoff. Twente’s story is quite a different one. Their season started out brilliantly, with the team winning their first four matches and eventually settling around the 2.0 PPG mark. During the winter break, manager Co Adriaanse was fired, not so much because of a lack of result, but because of communication issues with the players as the story ran. The return of manager Steve McClaren brought an excellent start, but soon the team’s performance faded and the season ended dramatically as McClaren finished way below expectations at 1.59 PPG.

Twente's points per game (PPG) curve under Adriaanse (left) and McClaren (right)

Let’s study Twente’s PDO and TSR to look for an explanation as to why McClaren started well, but faded so dramatically.

Under Adriaanse, Twente’s TSR was relatively stable, as was their PDO. However, their extremely high level of a PDO of 1068 over the first half of the season always seemed unsustainable. Twente’s high level of PDO held on almost during the entire season, even increased during McClaren’s first 8 games, but finally came down. Some awfully back luck saw McClaren’s team return to average in order to finish the second half of the season with an average PDO of 992.

Twente's luck (PDO) and skill (TSR) in the first and the second half of the season

 

In the end

Simple curves, yet interesting observations. VVV manager Glen de Boeck was fired after running into some bad luck, with the team’s PDO falling dramatically. Despite his steadily increasing TSR, which indicates an improving performance, this drop in PDO prevented his team from getting the points that their improving performance deserved. De Boeck’s successor Lokhoff did improve the team initially, but his TSR is dropping ever more, indicating that that initial bump in improvement will not hold for the long run. Assisted by a run of better luck he got off to a flying start.

At Twente, Adriaanse got the team performing at a rather constant level, around a TSR of 0.6. However, Twente’s PDO level over 1060 was never going to hold in the long run. Under McClaren, Twente’s performance initially dipped, but improved near the end to finish the season at the same level as under Adriaanse. However, a return to average PDO levels made the performance drop dramatically, a development that was just waiting to happen.

8 thoughts on “The ‘Lokhoff’ effect & The ‘McClaren’ non-effect

    1. admin Post author

      Yes, indeed!
      A major part of the higher TSR should carry over into the new season, while the low PDO will not…

      Reply
  1. Purplebeard

    It would be helpful to note what exactly is depicted in these graphs. Without any further specification, one would expect to find match-by-match statistics, but the points-per-game graphs contradict this view (as they contain data points that aren’t 0, 1 or 3). If this is some sort of rolling average, over how many matches do you average?

    Reply
    1. admin Post author

      A paragraph has been added to the main text now. The graphs depict rolling averages over the course of the season, with the new managers starting with new rolling averages.

      Reply
      1. Purplebeard

        Ah, thanks. The cumulative average produces somewhat misleading graphs, though, as it misrepresents variability. For example, a cursory look at the VVV graphs might lead one to think that the performance under Lokhoff starts out very erratic and becomes much more consistent towards the end of the season, whereas in reality it’s probably equally unpredictable throughout the second half of the season.

        For future analyses, I would suggest something like a running average over a sliding window of a couple of matches. This still produces smooth results but represents local variations more truthfully (like Twente’s poor form at the end of the season, which here is diluted by their earlier, better performances).

        Also, I hate to nitpick, but the points-per-game statistic seems to hold illegal values after two rounds for both VVV and Twente using the cumulative average. Unless I’m missing something, the only possible results after two matches are 0, 1/2, 1, 3/2, 2 and 3.

        Reply
        1. admin Post author

          Yes, I guess it’s a matter of what you’d like to show. In this case, the difference in overall results (PPG) is well shown. More shifts (as in four game running averages and the likes) will blur that part…
          As to the two game averages. Lokhoff’s VVV had three points after two matches, so both Lokhoff’s first game is off the chart due to scaling.
          Your remarks made me discover some errors in all four graphs though, so I ended up rewriting most of the second part of the article. Should be much better now!

          Reply
  2. Bert

    But how much of the TSR in the next season will remain when coach and players are different?

    Another point, as Ajax had a lot of injuries the last season, how much does the TSR change when De Boer was forced the play the likes of Lodeiro, Bulykin etc? That would say something about how much TSR is a reflection of a certain playingstyle or more about the quality of the players (obviously both would play a role I guess).

    Reply
    1. admin Post author

      Of course it will, you’re right about that.
      The TSR is simply a reflection of the fraction of shots that is taken by a certain team, reflective of the team’s strength.
      Data analysis of over 700 back-to-back season-wide performances showed the TSR to be the one parameter that is available for a single match with the highest correlation. Read James’ piece on it for further details: http://jameswgrayson.wordpress.com/2011/10/31/predicting-future-performance-revisited/

      Reply

Leave a Reply