Research Roundup Part 1 – The Best Sub Strategy, Will Financial Fair Play Ruin Man City, and Why You Shouldn’t Always Fire Your Coach

Welcome to the first installment of a new Café Futebol series – The Research Roundup. In these posts, we take a long look at the newest and most interesting soccer literature and let you know what’s going on. We walk through the papers and then highlight key insights and concerns we have. 

In the first installment, we’ll take a closer look at research on three major questions:

  1. Is there an optimal substitution strategy?
  2. Will Financial Fair Play be good or bad for your Premier League team?
  3. Does sacking your manager really lead to a temporary increase in performance?

If you have a paper you’d like us to cover, send us an email through the Contact Us page or reach me at @Cafefutebol on twitter!

Is there an optimal substitution strategy?

Paper Information:

  • Title: A Proposed Decision Rule for the Timing of Soccer Substitutions
  • Author, Institution: Bret R. Myers, Villanova University
  • Journal: Journal of Quantitative Analysis in Sports
  • Date of Publication:  March, 2012
  • Link to Paper

The flow of soccer makes it difficult for managers to have a direct impact on the outcome of the match. Once the first whistle blows, the game is, for the most part, decided entirely by the players on the field. The manager is left with only a handful of options should the game go sour. A half-time motivational speech or a slight tactical adjustment may be of importance, but any major changes lie in the substitutions made. These three players remain the manager’s most critical in-game decision for affecting the outcome of a game.

It’s quite surprising then that very little progress has been made on optimal substitution strategy – until now. Bret Myers, an assistant professor of management and operations at the Villanova School of Business, seeks to fill this gap. In his paper, he sets out on inventing the first practical use of academic substitution research for managers’ use. Using data from the major European leagues, the MLS, and the 2010 World Cup, Myers describes his optimal substitution strategy (dubbed the “Decision Rules”):

Proposed Decision Rule:

  • If down at half time
    • Make 1st sub prior to 58th minute
    • Make 2nd sub prior to 73rd minute
    • Make 3rd sub prior to 79th minute
  • If tied or ahead
    • Sub at will

Myers writes:

As the game approaches the first critical point of the 58th minute, a coach should make at least the first substitute if behind. As the game approaches the next critical point of the 73rd minute, if still behind, a coach should make at least the 2nd substitute. If the team is able to equalize or go ahead once the critical point is reached, then it is allowable for the 2nd substitute to be withheld. However, if the team returns to a state of being behind prior to the last critical point of the 79th minute, then a coach should use both the 2nd and 3rd substitution prior to the 79th minute. If a team that was previously tied or ahead falls behind after the 80th minute, there is no specific recommendation on how a coach should use the remaining substitutes if still available.

He concludes that, if a team is in a position to follow the Decision Rule (i.e. if they are behind or tied by half-time), that they can maximize their chance of winning by doing so. He finds that teams that follow his guidelines improve (defined by scoring at least more goal) roughly 36% of the time. The results are less encouraging for teams that are tied or ahead by half-time. In these scenarios, the manager’s substitution timing has little impact on the result of the match.

Additional Charts and Graphs:

graph1

Italians were by far the most capable in using their substitutes at maximum capacity, evidenced by their success following Myers’s decision rule. La Liga had the lowest, indicating a league-wide lack of bench depth despite an overall willingness by managers to send in their substitutes before their German or English counterparts. Perhaps La Liga coaches aren’t as afraid to experiment with formations or lineups when behind, even if it doesn’t always work out.

Insights:

Traditionally, substitution literature has remained largely descriptive in nature, without offering much practical managerial use. Myers’s research is a refreshing initiative towards implementable tools, and regardless of his conclusions, represents an important step in gaining traction in the locker room.

Furthermore, Myers’s results suggest that coaches tend to underestimate the significance of a fresh set of legs on the field. Managers largely “overvalue starters and undervalue the role of substitutes” in a match. If this is one of the few metrics by which we can reasonably evaluate a coach, then further research and application is warranted.

Concerns:

Myers potentially muddles correlation and causation in a manner that might jeopardize his research. Myers believes that early substitutions reduce the effect of player fatigue and lead to an increase in team performance.  However, it is possible that managers are more willing to sub off starters early if they are confident in their replacement. A manager may only be willing to send in a higher quality substitute and a lower quality substitute for 30 and 20 minutes respectively.

For example, Myers notes that, in 2009, Bayern Munich followed his decision rule 5 out of 8 times while Dortmund only did so 2 out of 9 times. Bayern Munich has a much deeper bench than Dortmund. It’s possible that Bayern’s coach simply trusted his substitutes more, leading him to send them in earlier.

If early substitutes are endogenously correlated with higher quality players, then this may be responsible for the observed increase in results. If this is the case, then Myers’s conclusions become more fuddled. Timing is still important – a team with a deeper bench sends in their substitutes early precisely because they are aware of the physical toll on starters – but then the decision rule is no longer universal. It may only apply to teams with a deep bench.

The only substitute you always put in early

There is also a chance that a large score deficit indicates an opportunity to send in youth players for additional experience, knowing that the match has already been decided. Take the case of Barcelona – Bayern Munich 0:3 at the Camp Nou during the 2013 CL semi-finals. Barcelona was down 5-0 on aggregate before Tito sent in his first substitute in the 55th minute: Alexis for Xavi. Ten minutes later, Iniesta was subbed for Thiago Alcantara. No coach in their right mind subs off Xavi and Iniesta when there’s still a chance of winning. In the cases like that of Barcelona, who conceded two more goals following these substitutes, early substitution may be correlated with lower team performance.

Further analysis could isolate this effect by controlling for score deficit and team-season fixed effects based on the quality of the bench in relation to the starting lineup. In the meantime, Myers’s decision rule is insightful, but should be taken with a grain of salt.

What does Financial Fair Play mean for the Premier League?

Paper Information:

  • Title: Vertical restraints in soccer: Financial Fair Play and the English Premier League
  • Author, Institution: Thomas Peeters, University of Antwerp & Stefan Szymanski, University of Michigan
  • Journal: Working Paper from University of Antwerp, Faculty of Applied Economics
  • Date of Publication:  March, 2012
  • Link to Paper

In their paper, Peeters and Szymanski construct a profit model for club teams as a function of their wages, costs, and revenue generated from winning games. Once they have a model and verify it using empirical data, they simulate the effects of FFP. They find that the FFP’s break-even rule has a salary-cap effect similar to the one present in American sports. However, unlike in the US, the rule has not been negotiated as part of a collective bargaining agreement with unions and may not be exempt from competition law in the EU. If the FFP does have a salary-cap effect, it may not be compliant with EU regulations.

The basic premise of their model is that football clubs are not profit maximizing, but are instead constrained by a limited negative profit-line that their owner is willing to cover. In other words – football clubs consistently operate at a loss, only to be bailed out by their wealthy owner. Financial fair play is going to limit the loss any one football club can take. This effectively reduces a club’s budget.

The model of Peeters and Szymanski has three components: revenues, wages, and other costs (stadium maintenance, advertising, etc).  They assume that revenues are already being maximized and that other costs are already being minimized. The only variable that can be cut in order to fit into a smaller budget constraint is player wages.

The next step is to estimate the parameters of the model using data gathered from the top three tiers of English soccer from 1997 to 2008. This allows the researchers to verify that their model holds against real, empirical data, and that it can be explained intuitively. The results are positive – all of the model’s parameters fit their expected values at a statistically significant level. Essentially, the model is well-behaved, suggesting that it can be reasonably used to estimate the impact of a new budget constraint. Finally, Peeters and Szymanski simulate the effects of FFP under their model. They find that the new rules lead to an overall reduction in league player wages over time, although the winners and losers are mixed.

Several major powerhouses will likely remain unaffected. In particular, Manchester United, Arsenal, and Liverpool are all able to “consolidate their position in prediction point totals” due to their high revenue capacity and a statistically strong ability to convert wages to results. On the flip-side, Manchester City and West Ham look to lose considerably more than other teams. Although Chelsea follows a similar strategy to Manchester City in terms of exorbitant spending at the owner’s expense, they have established themselves as a preeminent club and do not “appear to face the same difficulty in sustaining its position under FFP.” The teams set to receive the highest degree of benefit are largely those located at the bottom of the table. They will gain a considerable advantage from the decreased cost of success in terms of player wages.

Additional Charts and Graphs:

graph2

You can see for yourself the results of the duo’s FFP simulation. The four scenarios presented correspond to an average accepted deviation of €15m, €10m and €5m per season, and the “final” scenario with a total acceptable deviation of €5m over three seasons. The deviations represent the amount by which clubs are allowed to overspend their budget during the first years of adjusting to FFP. They increase in leniency when read from left to right in the chart above.

Insights:

Financial Fair Play may not be such a bad rule if you support Liverpool or Arsenal. It will certainly take a hit at relatively wealthy newcomers such as Manchester City, PSG, and Monaco. It’s entirely possible that within a few years of FFP implementation we will see a resurgence of ‘historic’ teams. Tottenham, surprisingly, will still not qualify for the Champions League.

It would be interesting to see how further simulations play out over other European leagues. My guess is that La Liga and the Bundesliga would remain largely intact, while the French league may return to its classic free-for-all.

Concerns:

The profit model constructed leaves out an important factor from the equation: player transfer fees. As these fees increase over time, they become a more significant proportion of club costs and may not be actually be minimized, as Peeters and Szymanski assume. In this scenario, it is entirely possible that tightening a club’s budget will lead to a reduction of player transfer value instead of wages.  It is not surprising this effect is omitted – in Soccernomics Szymanski found that player wages, and not transfer fees, are essential in predicting league success. Nevertheless, when deciding a revenue equation, it is over-simplistic to assume clubs are ‘minimizing’ transfer fees.

Nevertheless, even if transfer fees are reduced instead of wages, it’s unlikely to change the results of the study. Successful, established clubs would remain largely unaffected, given that their prestige and winning-record means the club is attractive enough to entice players without paying inflated prices (i.e. their costs are already minimized). Meanwhile, clubs such as Manchester City would be forced to reduce the payments they can offer for player transfers, and teams at the bottom of the table would face a less inflated player transfer market.

Not Pictured: Wenger on the left, Moyes on the right

It should also be noted that this piece is a working paper and has thus neither been accepted to any academic journal nor officially peer-reviewed. I would normally avoid discussing working papers, but Stefan Szymanski is co-author of Soccernomics and a prominent author in the world of soccer economics and sports management. For this reason, I make an exception.

Does sacking your manager really lead to an increase in performance?

Paper Information:

  • Title: The Effects of Managerial Turnover: Evidence from Coach Dismissals in Italian Soccer Teams
  • Author, Institution: Maria De Paola and Vincenzo Scoppa, University of Calabria, Cosenza
  • Journal: Journal of Sports Economics
  • Date of Publication:  March, 2011
  • Link to Paper

If you follow soccer, you’ve probably heard of the ‘five game bump’. The bump is usually a streak of wins following the sacking of a manager. Players are so shaken by the departure of their coach that, the bump-advocate would argue, they become much more focused in upcoming matches. It is essentially a slap in the face, a well-meaning shake, meant to bring sleeping footballers back to the real world.

Scoppa and De Paola are more skeptical of the phenomenon. They know that positive results observed after a series of losses can be misleading. The new manager could even win more games and still be statistically worse than the last. The two use 1997-2008 Serie A match data to measure the effects of coach changes on team performance, while controlling for two major challenges: the tendency for random data to converge towards its mean and the variation of opponent quality.

The ’regression to the mean’ phenomenon states that if we observe an extreme initial measurement, then a subsequent measurement will tend towards the mean. In a footballing context, if we observe a string of ‘below-average’ club performances, then subsequent performances will tend towards the average, and we will observe an increase in performance. A team could do poorly, fire their coach, and do well, and none of this would be due change in management. Of course, the actual team average performance is unobservable, but if it exists at a higher level than what it takes to get a coach fired, then changing coaches may have no effect at all. Not accounting for this phenomenon could lead to a serious overestimation of managerial impact.

The second issue that Scoppa and De Paola must deal with is that coaches are not fired randomly. Since their dismissals are typically decided after a series of poor performances, weaker teams tend to replace the coach more often. In this case, coach changes are negatively correlated with team quality. This could lead to a serious underestimation of managerial impact.

Gareth “regression to the mean” Bale

In order to compensate for the two, Scoppa and De Paola devise a model based on team and season fixed effects. This model looks at all matches during a given season pre- and post-managerial change. Note that they are not looking at a ‘5-game-bump’, but an entire season. No inter-seasonal managerial changes are recorded since those may actually reflect a good run (good managers are poached by larger clubs). The fixed effect accounts for the fact that the old coach and the new coach do not play against the same opponents. This allows the model to correct for tough schedules and the impact they may have on managerial record.

The results of the analysis are mixed. The model estimates that changing coaches does not positively affect overall team performance, with the exception of number of goals scored. This suggests that firing the coach may not have anything to do with improving team performance, but may be the unfortunate side effect of team boards overestimating their ability to make optimal replacements, or may even serve to brand the coach as a scapegoat. In any case, despite observing an improvement in results post managerial switch, this paper suggests it has little to do with the switch itself.

Additional Charts and Graphs:

graph3

In case you were wondering, there is indeed an average increase in points per game following the sacking of a manager. You can see so here! The only thing that’s up for discussion is whether or not it has anything to do with the new manager, or if it’s simply a statistical phenomenon. As it turns out, it’s probably a statistical phenomenon. The data suggests that firing Andre Villas-Boas isn’t going to save Tottenham and that perhaps Manchester United’s board is wise to keep David Moyes on contract despite recent performances.

Insights:

Although firing the manager doesn’t seem to have any real impact on team performance, it does not mean it’s a completely useless action. Actually, they very fact that firing still occurs indicates that it serves a different purpose entirely. It can relieve pressure on players and on the board of directors. If the board is voted on democratically by club members, or chosen by stock holders in the event that the club is publicly traded (see: Manchester United), it may be in the interest of the board to find a scapegoat. Firing managers also relieves pressure on the managerial side. An incoming manager may find himself with a terrible situation and, often enough, nowhere to go but up. This may make it easier for the new manager to establish his team and methodology without worrying about immediate gratification.

Concerns:

The fixed-effects the authors choose to measure in their paper are confusing and often inconsistent. For example, the authors make a point to consider both rank difference and point differential, despite representing the same implicit factor: a difference in recent performance between teams. Nevertheless, they choose to gauge a team’s quality by their previous year’s rank as opposed to their points per game. It is also unclear as to why the authors choose to use previous-season rankings as an indicator for current form. These can be misleading given inter-season developments including management changes and transfers. A better indicator might be player wages and transfer fees, which previous studies have argued play a major role in determining league performance.

Lastly, there are two additional factors that ought to be addressed: tough scheduling and home field advantage. Although this typically affects higher-quality teams who engage in multiple tournaments, clubs faced with fixtures in a smaller time frame may experience a lag in performance due to stress and lack of adequate rest. It is worth investigating potential effects this has on ‘good’ and ‘bad’ streaks as a consequence of their manager. Additionally, soccer literature largely agrees that home-field advantage is a real phenomenon. If an incoming coach has more home games than away, he may appear to get better results when in fact he is useless. It is not clear whether or not this effect is accounted for in the study due to the vague wording, but I was unable to find any explicit mention of it, so I assume it was overlooked or is endogenous in another variable which was not clearly explained. In any case, it is a simple effect that deserves attention due to the general consensus of its reality.

Conclusion:

Thanks for reading!

If you have any questions or comments, send us an email through the Contact Us page or reach me at @Ncholst on twitter.

Advertisements

One thought on “Research Roundup Part 1 – The Best Sub Strategy, Will Financial Fair Play Ruin Man City, and Why You Shouldn’t Always Fire Your Coach

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s