Myth Busting: Does Playing the Second Leg at Home Actually Matter?

The Myth

Most football fan will tell you that when it comes to double-legged knockout tournaments, it’s better to play at home in the second leg than away. The idea is simple – the away team has the option (luxury) to play it safe (park the bus), aim for a draw, and then bank on winning the home game. Not the most elegant approach, but an effective one nonetheless.

Proponents of the strategy are not limited to fans and pundits – players and coaches can be just as guilty. Iker Casillas thinks it will give them an advantage over Schalke 04 in the Champions League this February. Dougie Freedman, former Crystal Palace striker, once attributed a victory against Charlton in the second leg of a promotion play-off semi-final to the incredible atmosphere. Patrick Kluivert, former Dutch international, believes it’s better to “make your mistakes in the first leg away from home because there is still time to put things right in the second match.”

"Maybe we have a slight advantage because we are playing the return match at home" - Carlo Ancelotti

“Maybe we have a slight advantage because we are playing the return match at home” – Carlo Ancelotti

But while the story is written by the pundits, the truth is written in the numbers. A 2007 paper by Lionel Page and Katie Page, of the University of Westminster and University of Queensland respectively, looked to find just that. The first academic research of its kind, the study made significant advancements but ultimately fell short of conducting a proper analysis of the phenomenom. It wasn’t until a follow-up study was conducted by three researchers from the Ludwig Maximilian University of Munich that the truth was uncovered: playing the second leg at home is, statistically speaking, irrelevant.

Evidence From the Champions League

It’s not hard to see why this myth has so much traction. On the surface, data from the Champions League fits the narrative: teams play conservatively in the first leg and open up in the second. This is reflected in the increase in goals scored in the return leg.

1st leg goals

2nd leg goals

1st leg goals per game

2nd leg goals per game

2012 (group)

136

148

2.83

3.08

2012 (knockout)

39

42

1.39

1.50

2011 (group)

120

134

2.50

2.79

2011 (knockout)

32

57

1.14

2.04

2010 (group)

131

145

2.73

3.02

2010 (knockout)

38

37

1.36

1.32

Total

496

563

1.99

2.29

Additional calculations by Michael Cox of Zonal Marking for the 2007-2009 Champions Leagues confirm the findings above. That is, with the exception of 2010, more goals are consistently scored in the second leg of CL knockout rounds than in the first.

This phenomenon can work for and against a team playing at home during the second leg. On one hand, the away team is less likely to sit back and park the bus. This lets the home team exploit their home advantage and take control of the deciding match. On the other hand, in a game that’s decidedly more ‘open’, a goal for the away team counts for double. Ideally these two effects would balance out and we would observe a 50-50 chance of advancing, even if more goals are scored in the second leg (e.g. if a team goes 1-0 at home and then loses 1-2 away, the away goal rule would outweigh the home field advantage).

Since we know more goals are being scored, the question becomes: for whom?

The First Study

Between 1994 and 2009, 56 percent of teams which played the second leg of a Champions League tie at home advanced to the next round. On the surface, this suggests home-field advantage frequently outweighs the benefit of the away goal rule.

But these findings are not sufficient. They do not take into consideration that the order of home-away matches for the CL round of 16 is not random.

In fact, group winners first play away, then at home. This was instituted as an added incentive for group winners, based on the very myth itself. This creates a sort of a self-fulfilling prophecy. Seeded teams play at home in the second leg, and it’s not clear if they win because they’re the better team, or because of their home-field advantage.

In their 2007 paper, Lionel Page and Katie Page look examine this specific aspect. Together, they use data from three different European Cup football competitions over 51 years in order to assess the accuracy of the second-leg advantage myth.

One solution they assess is to simply not look at data from the round of 16. Later rounds no longer seed home-away match order, randomizing the process and eliminating possible bias. But they find that this approach considerably reduces the sample size, and makes it harder to obtain meaningful results.

Page and Page instead use team-coefficients to adjust for round of 16 seeding. By controlling for team-quality based on UEFA rankings, they attempt to separate the effect of playing at home in the second leg and the effect of being a better team. Their results produce the following:

yo

This graph shows an overall decrease in second-leg home advantage, both adjusted and unadjusted for team coefficients, over time. Interestingly enough, the decrease in the late 60s in this advantage can be traced to the introduction of the away-goals rule. By increasing away-goal value, the tactic of parking the bus away and winning at home was no longer as effective. For a team playing away, losing wasn’t the end of the world, but scoring an away goal could be the difference that would send them through.

The temporary decrease in advantage between 1984 and 1990 coincides with a five-year ban on English club participation in European competitions as a result of the Heysel Stadium disaster. This suggests that, at the time, English clubs exhibited a more significant second-leg home advantage than their European counterparts. In fact, seven out of eight previous European Cup winners were English. It’s possible that part of this success was the ability to exploit the second-leg home advantage in either the form of favorable refereeing or by employing the ‘defend away, win at home’ strategy.

The Away Goal Rule

At this point, readers would be right to ask: what about the away goal rule? Why does everyone believe there’s an advantage to playing the second leg at home when the away team has the opportunity to play an extra 30 minutes where their goals count double.

Page and Page actually look into this. They analyzed 186 European cup ties which go into extra time and found that the probability of winning the second leg at home is 66.42%. These results are adjusted for team ability, regardless of the fact that ties going into extra time might reflect equally-skilled teams. Page and Page also find that matches that go to penalties on the home ground result in the home team winning 57.33% of the time.

Simply put, the advantage of playing at home for an extra 30 minutes considerably outweighs the advantage of having goals count for double for the same period of time. Note that, whether or not there is an advantage to playing at home first, home advantage is a very real phenomenon.

Arsene-Wenger1_2354548

“Statistically it is not proven. We like to think in our job that to play at home in the second leg is an advantage, but it is not proven at all in the statistics. It is 50:50” – Arsene Wenger

Busting the Myth

Page and Page make one crucial mistake that jeopardizes their conclusion that second leg at home advantage remains today. They only control for team UEFA coefficients, but this is not sufficient to isolate the advantage effect. Think about it this way: In Page and Page’s model, they attempt to separate the following effects:

chart

However, in order for their model to work, the two effects cannot be correlated outside of their impact on the dependent variable (in this case, the round of 16 performance).

If you’re a little confused right now, you should be. This is the same problem Page and Page thought they were solving. But their analysis was rudimentary: you can’t simply control for correlated effects in order to isolate them.

Suppose you have three variables A, B, and C. You hypothesize that A and B both positively affect C. The data shows as much: A, B, and C all rise together. You could jump to conclusions and assume you were right from the beginning. But really, B is positively affecting A and C, giving the illusion your initial hypothesis was correct when in fact A is completely unrelated to C.

For example, imagine that the US government introduces a new anti-smoking campaign. The campaign is a success, and raises the population’s life expectancy. The government also chooses to raise taxes to fund the campaign. One might observe taxes rising and mortality rates decreasing and falsely conclude that paying more taxes means you’ll live longer.

Likewise, Page and Page observe a lot of teams play the second leg at home because they’re good and win their group, and falsely assume that they win the next round because they played the second leg at home. In reality, a better team will have a higher coefficient going into the tournament, and will not only do better in the round of 16, but is also more likely to have the second-leg at home. This leads them to overestimate the effect match order has on the outcome.

Three researchers from the Ludwig Maximilian University of Munich took a closer look at the data, and noticed what Page and Page did. To solve for this issue, they controlled for group stage performance in order to completely isolate the home-away match order effect. Any effect team coefficients have on match order is now funneled into the group stage performance variable, giving them clear results.

The researchers found that, once they controlled for group stage performance, playing the second leg at home had no impact on a team’s chances of success. It turns out that the team that progresses is usually just better than their opponent, contrary to the superstitions of some of football’s greatest minds. Maybe this is the reason Wenger has been so successful – he knows when to listen to the numbers.

Advertisements

Research Roundup Part 1 – The Best Sub Strategy, Will Financial Fair Play Ruin Man City, and Why You Shouldn’t Always Fire Your Coach

Welcome to the first installment of a new Café Futebol series – The Research Roundup. In these posts, we take a long look at the newest and most interesting soccer literature and let you know what’s going on. We walk through the papers and then highlight key insights and concerns we have. 

In the first installment, we’ll take a closer look at research on three major questions:

  1. Is there an optimal substitution strategy?
  2. Will Financial Fair Play be good or bad for your Premier League team?
  3. Does sacking your manager really lead to a temporary increase in performance?

If you have a paper you’d like us to cover, send us an email through the Contact Us page or reach me at @Cafefutebol on twitter!

Is there an optimal substitution strategy?

Paper Information:

  • Title: A Proposed Decision Rule for the Timing of Soccer Substitutions
  • Author, Institution: Bret R. Myers, Villanova University
  • Journal: Journal of Quantitative Analysis in Sports
  • Date of Publication:  March, 2012
  • Link to Paper

The flow of soccer makes it difficult for managers to have a direct impact on the outcome of the match. Once the first whistle blows, the game is, for the most part, decided entirely by the players on the field. The manager is left with only a handful of options should the game go sour. A half-time motivational speech or a slight tactical adjustment may be of importance, but any major changes lie in the substitutions made. These three players remain the manager’s most critical in-game decision for affecting the outcome of a game.

It’s quite surprising then that very little progress has been made on optimal substitution strategy – until now. Bret Myers, an assistant professor of management and operations at the Villanova School of Business, seeks to fill this gap. In his paper, he sets out on inventing the first practical use of academic substitution research for managers’ use. Using data from the major European leagues, the MLS, and the 2010 World Cup, Myers describes his optimal substitution strategy (dubbed the “Decision Rules”):

Proposed Decision Rule:

  • If down at half time
    • Make 1st sub prior to 58th minute
    • Make 2nd sub prior to 73rd minute
    • Make 3rd sub prior to 79th minute
  • If tied or ahead
    • Sub at will

Myers writes:

As the game approaches the first critical point of the 58th minute, a coach should make at least the first substitute if behind. As the game approaches the next critical point of the 73rd minute, if still behind, a coach should make at least the 2nd substitute. If the team is able to equalize or go ahead once the critical point is reached, then it is allowable for the 2nd substitute to be withheld. However, if the team returns to a state of being behind prior to the last critical point of the 79th minute, then a coach should use both the 2nd and 3rd substitution prior to the 79th minute. If a team that was previously tied or ahead falls behind after the 80th minute, there is no specific recommendation on how a coach should use the remaining substitutes if still available.

He concludes that, if a team is in a position to follow the Decision Rule (i.e. if they are behind or tied by half-time), that they can maximize their chance of winning by doing so. He finds that teams that follow his guidelines improve (defined by scoring at least more goal) roughly 36% of the time. The results are less encouraging for teams that are tied or ahead by half-time. In these scenarios, the manager’s substitution timing has little impact on the result of the match.

Additional Charts and Graphs:

graph1

Italians were by far the most capable in using their substitutes at maximum capacity, evidenced by their success following Myers’s decision rule. La Liga had the lowest, indicating a league-wide lack of bench depth despite an overall willingness by managers to send in their substitutes before their German or English counterparts. Perhaps La Liga coaches aren’t as afraid to experiment with formations or lineups when behind, even if it doesn’t always work out.

Insights:

Traditionally, substitution literature has remained largely descriptive in nature, without offering much practical managerial use. Myers’s research is a refreshing initiative towards implementable tools, and regardless of his conclusions, represents an important step in gaining traction in the locker room.

Furthermore, Myers’s results suggest that coaches tend to underestimate the significance of a fresh set of legs on the field. Managers largely “overvalue starters and undervalue the role of substitutes” in a match. If this is one of the few metrics by which we can reasonably evaluate a coach, then further research and application is warranted.

Concerns:

Myers potentially muddles correlation and causation in a manner that might jeopardize his research. Myers believes that early substitutions reduce the effect of player fatigue and lead to an increase in team performance.  However, it is possible that managers are more willing to sub off starters early if they are confident in their replacement. A manager may only be willing to send in a higher quality substitute and a lower quality substitute for 30 and 20 minutes respectively.

For example, Myers notes that, in 2009, Bayern Munich followed his decision rule 5 out of 8 times while Dortmund only did so 2 out of 9 times. Bayern Munich has a much deeper bench than Dortmund. It’s possible that Bayern’s coach simply trusted his substitutes more, leading him to send them in earlier.

If early substitutes are endogenously correlated with higher quality players, then this may be responsible for the observed increase in results. If this is the case, then Myers’s conclusions become more fuddled. Timing is still important – a team with a deeper bench sends in their substitutes early precisely because they are aware of the physical toll on starters – but then the decision rule is no longer universal. It may only apply to teams with a deep bench.

The only substitute you always put in early

There is also a chance that a large score deficit indicates an opportunity to send in youth players for additional experience, knowing that the match has already been decided. Take the case of Barcelona – Bayern Munich 0:3 at the Camp Nou during the 2013 CL semi-finals. Barcelona was down 5-0 on aggregate before Tito sent in his first substitute in the 55th minute: Alexis for Xavi. Ten minutes later, Iniesta was subbed for Thiago Alcantara. No coach in their right mind subs off Xavi and Iniesta when there’s still a chance of winning. In the cases like that of Barcelona, who conceded two more goals following these substitutes, early substitution may be correlated with lower team performance.

Further analysis could isolate this effect by controlling for score deficit and team-season fixed effects based on the quality of the bench in relation to the starting lineup. In the meantime, Myers’s decision rule is insightful, but should be taken with a grain of salt.

What does Financial Fair Play mean for the Premier League?

Paper Information:

  • Title: Vertical restraints in soccer: Financial Fair Play and the English Premier League
  • Author, Institution: Thomas Peeters, University of Antwerp & Stefan Szymanski, University of Michigan
  • Journal: Working Paper from University of Antwerp, Faculty of Applied Economics
  • Date of Publication:  March, 2012
  • Link to Paper

In their paper, Peeters and Szymanski construct a profit model for club teams as a function of their wages, costs, and revenue generated from winning games. Once they have a model and verify it using empirical data, they simulate the effects of FFP. They find that the FFP’s break-even rule has a salary-cap effect similar to the one present in American sports. However, unlike in the US, the rule has not been negotiated as part of a collective bargaining agreement with unions and may not be exempt from competition law in the EU. If the FFP does have a salary-cap effect, it may not be compliant with EU regulations.

The basic premise of their model is that football clubs are not profit maximizing, but are instead constrained by a limited negative profit-line that their owner is willing to cover. In other words – football clubs consistently operate at a loss, only to be bailed out by their wealthy owner. Financial fair play is going to limit the loss any one football club can take. This effectively reduces a club’s budget.

The model of Peeters and Szymanski has three components: revenues, wages, and other costs (stadium maintenance, advertising, etc).  They assume that revenues are already being maximized and that other costs are already being minimized. The only variable that can be cut in order to fit into a smaller budget constraint is player wages.

The next step is to estimate the parameters of the model using data gathered from the top three tiers of English soccer from 1997 to 2008. This allows the researchers to verify that their model holds against real, empirical data, and that it can be explained intuitively. The results are positive – all of the model’s parameters fit their expected values at a statistically significant level. Essentially, the model is well-behaved, suggesting that it can be reasonably used to estimate the impact of a new budget constraint. Finally, Peeters and Szymanski simulate the effects of FFP under their model. They find that the new rules lead to an overall reduction in league player wages over time, although the winners and losers are mixed.

Several major powerhouses will likely remain unaffected. In particular, Manchester United, Arsenal, and Liverpool are all able to “consolidate their position in prediction point totals” due to their high revenue capacity and a statistically strong ability to convert wages to results. On the flip-side, Manchester City and West Ham look to lose considerably more than other teams. Although Chelsea follows a similar strategy to Manchester City in terms of exorbitant spending at the owner’s expense, they have established themselves as a preeminent club and do not “appear to face the same difficulty in sustaining its position under FFP.” The teams set to receive the highest degree of benefit are largely those located at the bottom of the table. They will gain a considerable advantage from the decreased cost of success in terms of player wages.

Additional Charts and Graphs:

graph2

You can see for yourself the results of the duo’s FFP simulation. The four scenarios presented correspond to an average accepted deviation of €15m, €10m and €5m per season, and the “final” scenario with a total acceptable deviation of €5m over three seasons. The deviations represent the amount by which clubs are allowed to overspend their budget during the first years of adjusting to FFP. They increase in leniency when read from left to right in the chart above.

Insights:

Financial Fair Play may not be such a bad rule if you support Liverpool or Arsenal. It will certainly take a hit at relatively wealthy newcomers such as Manchester City, PSG, and Monaco. It’s entirely possible that within a few years of FFP implementation we will see a resurgence of ‘historic’ teams. Tottenham, surprisingly, will still not qualify for the Champions League.

It would be interesting to see how further simulations play out over other European leagues. My guess is that La Liga and the Bundesliga would remain largely intact, while the French league may return to its classic free-for-all.

Concerns:

The profit model constructed leaves out an important factor from the equation: player transfer fees. As these fees increase over time, they become a more significant proportion of club costs and may not be actually be minimized, as Peeters and Szymanski assume. In this scenario, it is entirely possible that tightening a club’s budget will lead to a reduction of player transfer value instead of wages.  It is not surprising this effect is omitted – in Soccernomics Szymanski found that player wages, and not transfer fees, are essential in predicting league success. Nevertheless, when deciding a revenue equation, it is over-simplistic to assume clubs are ‘minimizing’ transfer fees.

Nevertheless, even if transfer fees are reduced instead of wages, it’s unlikely to change the results of the study. Successful, established clubs would remain largely unaffected, given that their prestige and winning-record means the club is attractive enough to entice players without paying inflated prices (i.e. their costs are already minimized). Meanwhile, clubs such as Manchester City would be forced to reduce the payments they can offer for player transfers, and teams at the bottom of the table would face a less inflated player transfer market.

Not Pictured: Wenger on the left, Moyes on the right

It should also be noted that this piece is a working paper and has thus neither been accepted to any academic journal nor officially peer-reviewed. I would normally avoid discussing working papers, but Stefan Szymanski is co-author of Soccernomics and a prominent author in the world of soccer economics and sports management. For this reason, I make an exception.

Does sacking your manager really lead to an increase in performance?

Paper Information:

  • Title: The Effects of Managerial Turnover: Evidence from Coach Dismissals in Italian Soccer Teams
  • Author, Institution: Maria De Paola and Vincenzo Scoppa, University of Calabria, Cosenza
  • Journal: Journal of Sports Economics
  • Date of Publication:  March, 2011
  • Link to Paper

If you follow soccer, you’ve probably heard of the ‘five game bump’. The bump is usually a streak of wins following the sacking of a manager. Players are so shaken by the departure of their coach that, the bump-advocate would argue, they become much more focused in upcoming matches. It is essentially a slap in the face, a well-meaning shake, meant to bring sleeping footballers back to the real world.

Scoppa and De Paola are more skeptical of the phenomenon. They know that positive results observed after a series of losses can be misleading. The new manager could even win more games and still be statistically worse than the last. The two use 1997-2008 Serie A match data to measure the effects of coach changes on team performance, while controlling for two major challenges: the tendency for random data to converge towards its mean and the variation of opponent quality.

The ’regression to the mean’ phenomenon states that if we observe an extreme initial measurement, then a subsequent measurement will tend towards the mean. In a footballing context, if we observe a string of ‘below-average’ club performances, then subsequent performances will tend towards the average, and we will observe an increase in performance. A team could do poorly, fire their coach, and do well, and none of this would be due change in management. Of course, the actual team average performance is unobservable, but if it exists at a higher level than what it takes to get a coach fired, then changing coaches may have no effect at all. Not accounting for this phenomenon could lead to a serious overestimation of managerial impact.

The second issue that Scoppa and De Paola must deal with is that coaches are not fired randomly. Since their dismissals are typically decided after a series of poor performances, weaker teams tend to replace the coach more often. In this case, coach changes are negatively correlated with team quality. This could lead to a serious underestimation of managerial impact.

Gareth “regression to the mean” Bale

In order to compensate for the two, Scoppa and De Paola devise a model based on team and season fixed effects. This model looks at all matches during a given season pre- and post-managerial change. Note that they are not looking at a ‘5-game-bump’, but an entire season. No inter-seasonal managerial changes are recorded since those may actually reflect a good run (good managers are poached by larger clubs). The fixed effect accounts for the fact that the old coach and the new coach do not play against the same opponents. This allows the model to correct for tough schedules and the impact they may have on managerial record.

The results of the analysis are mixed. The model estimates that changing coaches does not positively affect overall team performance, with the exception of number of goals scored. This suggests that firing the coach may not have anything to do with improving team performance, but may be the unfortunate side effect of team boards overestimating their ability to make optimal replacements, or may even serve to brand the coach as a scapegoat. In any case, despite observing an improvement in results post managerial switch, this paper suggests it has little to do with the switch itself.

Additional Charts and Graphs:

graph3

In case you were wondering, there is indeed an average increase in points per game following the sacking of a manager. You can see so here! The only thing that’s up for discussion is whether or not it has anything to do with the new manager, or if it’s simply a statistical phenomenon. As it turns out, it’s probably a statistical phenomenon. The data suggests that firing Andre Villas-Boas isn’t going to save Tottenham and that perhaps Manchester United’s board is wise to keep David Moyes on contract despite recent performances.

Insights:

Although firing the manager doesn’t seem to have any real impact on team performance, it does not mean it’s a completely useless action. Actually, they very fact that firing still occurs indicates that it serves a different purpose entirely. It can relieve pressure on players and on the board of directors. If the board is voted on democratically by club members, or chosen by stock holders in the event that the club is publicly traded (see: Manchester United), it may be in the interest of the board to find a scapegoat. Firing managers also relieves pressure on the managerial side. An incoming manager may find himself with a terrible situation and, often enough, nowhere to go but up. This may make it easier for the new manager to establish his team and methodology without worrying about immediate gratification.

Concerns:

The fixed-effects the authors choose to measure in their paper are confusing and often inconsistent. For example, the authors make a point to consider both rank difference and point differential, despite representing the same implicit factor: a difference in recent performance between teams. Nevertheless, they choose to gauge a team’s quality by their previous year’s rank as opposed to their points per game. It is also unclear as to why the authors choose to use previous-season rankings as an indicator for current form. These can be misleading given inter-season developments including management changes and transfers. A better indicator might be player wages and transfer fees, which previous studies have argued play a major role in determining league performance.

Lastly, there are two additional factors that ought to be addressed: tough scheduling and home field advantage. Although this typically affects higher-quality teams who engage in multiple tournaments, clubs faced with fixtures in a smaller time frame may experience a lag in performance due to stress and lack of adequate rest. It is worth investigating potential effects this has on ‘good’ and ‘bad’ streaks as a consequence of their manager. Additionally, soccer literature largely agrees that home-field advantage is a real phenomenon. If an incoming coach has more home games than away, he may appear to get better results when in fact he is useless. It is not clear whether or not this effect is accounted for in the study due to the vague wording, but I was unable to find any explicit mention of it, so I assume it was overlooked or is endogenous in another variable which was not clearly explained. In any case, it is a simple effect that deserves attention due to the general consensus of its reality.

Conclusion:

Thanks for reading!

If you have any questions or comments, send us an email through the Contact Us page or reach me at @Ncholst on twitter.