dulcolax en espaol

Good or Lucky? Evaluating MLS’s Top Managers

There’s been discussion on some of the Sounders FC boards lately about whether or not Sigi Schmid should be fired. One of the common defenses that comes up is that he’s won 2 MLS Cups so therefore he is a good manager. The question is, in a league based on parity with a playoff system that almost everyone qualifies for, how hard is it to win an MLS Cup? Is Sigi Schmid, or other top MLS managers, statistically different from the rest of the league?

To answer these questions I looked at both playoff and regular season performance of Sigi Schmid, Bruce Arena, Frank Yallop, Dominic Kinnear, and Steve Nicol. All of these guys have won the MLS Coach of the Year award, coached in the league for more than 8 seasons and, with the exception of Steve Nicol, who has lost the cup final a remarkable 4 times, won multiple MLS Cups. They are the best that MLS has to offer.

What’s nice about MLS is that it is a salary capped league so issues of managers having better squads and resources to work with are minimized compared to other leagues in the world. That doesn’t mean that things are completely even though. Dominic Kinnear lost Stuart Holden and Ricardo Clark to European teams with no compensation. Not surprising, the seasons since those departures have been Kinnear’s two worst seasons. However, given that MLS strives for parity, I’d expect that these top managers consistently outperform the rest of the league. Over a long season, their teams should earn more points than others. Looking at points per game (to account for different length seasons) we see that indeed, the average PPG for the managers is higher than the league average with the exception of Frank Yallop.

Average PPG for top MLS Managers compared to leagueHowever, when we test to see if the results are significantly different from that of the league using a t-test, we get mixed results.

Manager p-value
Bruce Arena 0.10
Sigi Schmid 0.20
Dominic Kinnear 0.39
Steve Nicol 0.79
Frank Yallop 0.71

What this means is that there is a 10% chance that Bruce Arena’s results aren’t any different from a random manager, 20% for Sigi Schmid and so on. Put another way, at an 80% confidence level, we don’t think Schmid’s and Arena’s higher average PPG is due to luck or coincidence whereas we are unsure about the other managers (an interesting note about Schmid — if you ignore his stellar first few years in the league and focus on the last 8, his numbers are also not statistically different from the league.  Arena doesn’t have the same trend).  Does this mean that these guys aren’t really that good and instead are just lucky? Looking at the distributions of PPG reveals something interesting.

The top managers have never had a season where they earned less than 1 PPG.  Is this just a coincidence?  No — there is less than a 1% chance that this is a coincidence.  So while we can’t say for sure that these guys are better than average, what we can say is that they are consistently not terrible.  This makes sense since we are looking at managers with lengthy spells in MLS.  A terrible season would probably have resulted in their termination so what we’re left with is a self-selecting list of guys who are really good at not being really bad.  Not exactly a ringing endorsement for MLS’s managerial elite, but these guys are winners, right?  You don’t win multiple MLS Cups by just not being terrible, right?

The first step in winning the cup is to qualify for the play-offs.  MLS is ridiculous and 50-80% of teams have qualified for the play-offs (this season is a meager 55%).  These managers have shown that they don’t have terrible seasons so most years they qualify for the play-offs.  8 teams qualify, so all you need to do is win 2 home and away series and the final.  Since MLS is all about parity, let’s assume that for each stage in the play-offs, each team has an equal chance of winning.  This isn’t a perfect assumption, but it is close enough (the average winning seed is 3.5 whereas if it were perfectly even we’d expect 4.5, however n=15 so we’re pretty close given the small sample size). Given the even probability, how likely is it that the manager’s could equal their MLS Cup hauls?

Manager P(Same # of Cups)
Bruce Arena 22%
Sigi Schmid 26%
Dominic Kinnear 20%
Steve Nicol 30%
Frank Yallop 22%

What’s the moral of the story here? MLS’s best managers aren’t that different from the rest of the league aside from the fact that they’ve managed to preserve their jobs long enough that they get a lot of shots at the crap shoot called the MLS Playoffs. Should the Sounders fire Sigi Schmid? Probably not. They’ll probably have a mediocre enough season to qualify for the play-offs and from there it’s anybody’s game.

3 comments

  1. [...] Good or Lucky?: On Football’s Sarah Rudd evaluates MLS’s top managers. [...]

  2. Inv says:

    I agree with some of your articl, as it was obviously well thought out and there some good points. But I also dissagree in that they are lucky. When Columbus hired Sigi, he had two years to build a team to a specific system. In that time he won a Cup, a Supporters’ Shield, and made the team relevant again at a time when it was in shambles before he came.

    With that framework we won another Supporters Shield and watched a Jackass piss the groundwork and blueprints away, so I don’t believe he was lucky. Besides, he didn’t have time on his side.

    • Sarah Rudd says:

      Sigi is an interesting case because he is right on the cusp of being statistically different from the average manager, depending on what confidence level you want to use. The example you point out is a tricky one. Winning two supporters shields in a row is hard — harder than winning two MLS Cups in my opinion. No doubt he assembled a good squad, but the fact they also won it the year after he left shows that his impact from a managerial aspect is minimal. Any somewhat competent manager could have won with that squad. Maybe Sigi is a better GM than manager? He hasn’t been able to duplicate that success in Seattle and he has less power when it comes to player acquistions from my understanding of Seattle’s FO. He also doesn’t have Guillermo Barros Schelotto in Seattle. GBS, for me, is one of the best players to ever play in MLS. Columbus paid him a relatively low salary (most seasons) which freed up space to surround him with good players. How much of the success of the Crew came from Sigi and how much from GBS? It’s really hard to say.