Those of you who read EaglesRewind know I occasionally feature guest posts. Following is one such post, that I felt deserves a broader audience, so I'm reposting it here. I'll provide a comment to the model at the very end.
Time for another guest post from Jared Cohen. You may remember him from:
- One of the Coolest NFL Charts Ever
- The Fourth Down Decision Series and Cheat Sheet (Part 1 of 4)
- When does it make sense to return a kick?
Well today, he’s ready to unveil at least the first iteration of his Hot Seat Index, and attempt to predict which coaches will be fired at or by the end of the season. You can follow him @jaredscohen and at kebertxela.blogspot.com. Without further ado:
So the Eagles had another especially depressing loss to the Giants this week. And with such a depressing loss, it got me thinking of another depressing outcome, watching your team stink to the point where they fire their coach.
So, this was something I wanted to wait on until much later in the season, but with Greg Schiano working as hard as he can to get fired as soon as possible, I needed to put it out a bit early.
One of the things I hate hearing about as the NFL season progresses is all the discussion around the coaching ‘hot seat’. It’s not that I mind the speculation, but all of the discussion centers on conjecture and the occasional ‘anonymous source’ (the exception being Schiano where the sources have noticeably opted to forgo anonymity!)
As I was thinking about it, I really wanted to see if we could make this a bit more objective and data driven. So with that as the goal, I gathered some data, did some analysis, and am now ready to introduce the NFL Coaching Hot Seat Index!
The Coaching Hot Seat Index is a model that, at any point in the season, will give you the approximate odds of an NFL coach getting fired after the season!
It’s based on a collection of data on all NFL coaching seasons since 1980. I broke the data down and for each coaching season, identified whether a coach was fired or kept their job (with some adjustments for retirements etc.)
With over 900 observations (and ~200 firings!), I started looking for factors which were significant in predicting whether coaches got fired or not. I tried lots of things, number of wins, playoff appearances, having the last name ‘Kotite", all to see what was really significant in predicting when a coach will get fired.
I ended up with two primary factors, which were by far the most significant in predicting a coach getting kicked to the curb.
- Total point differential (points scored-points given up)
- Change in team wins from prior season (current year wins-prior year wins)
Those shouldn’t come as a huge surprise. (Note: I’m disappointed because I couldn’t examine another factor I wanted to see, the difference between Expected Wins at the Season Start and Actual Wins at season’s end…I wanted to use gambling lines to get at it but didn’t have nearly enough data to look at it. I still think it would be a more compelling variable)
But anyway, with those two factors, and using (or, abusing) a technique called logistic regression, I arrived at an equation to give us the odds a coach will get fired. Logistic regression is basically something you can use to help predict the likelihood of a binary outcome (a coach either gets fired, or he doesn’t) based on some variables (in this case, his team’s point differential and win change from prior season).
In the end, the result is the percentage chance (out of 100%) a coach will be fired after the season. The higher the odds, the more likely the coach is going to be filing some unemployment papers right around the Pro Bowl.
But I couldn’t just create a model and throw it out there without testing it a little. So I created a first draft of it, using only data from 1980-2011, and used that model to ‘predict’ the 2012 season based on its point differential and change in team wins (which, obviously, I already knew). When that passed the sanity check, I updated the model for 2012 and used it on teams from this year.
Those results are included below, with coaches sorted by their ‘odds to be fired’
Not a bad first result. Romeo Crennel, Mike Mularkey, Andy Reid all got canned, and those were the coaches with the highest likelihood. The model obviously isn’t perfect, as the plenty of coaches can still get fired, but at least there’s a structure and some logic here (we can also rank the coaches from safest to least safe!!!)
So this is interesting, and now we can apply it to the current season, and see what 2013 coaches are most likely to be fired!!!
Now, I had to make a couple of assumptions, because this model is based on a full season of performance, which of course we won’t have. But we do have a good sample and some reasonable projections of performance we can use as proxies.
For point differential, we can use a teams’ current point differential and pro rate it out for 16 games. This doesn’t account for changes in either a team’s performance or strength of schedule, but it’s not unreasonable.
For win difference from last year, we can take an estimate of the team’s full season (which I’ve borrowed from Football Outsiders Playoff Odds report, which runs simulations to calculate average team wins) and then just check the difference from 2012 wins.
So, at this point in the season (Week 7, because Week 8 is going on as I write this), which coaches have the highest odds of getting fired?
Well – it’s no surprise to see the Jaguars on the list, but we may have to make an exception for first year coaches as they typically get more than one season to right the ship. We’ll give Gus Bradley a pass (although no one can argue their performance is historically bad…it’s no wonder the odds are higher than anything seen in 2012)
Tom Coughlin checks in at number 2,
although we’ll unfortunately need to update that after the Eagles managed to lose to them this week. Their performance has been really bad as well, and any coach without Super Bowl rings would likely be on their way out. But Coughlin may overcome the odds with all the goodwill he’s built over the years (or maybe he changes his mind and "retires")
Next comes Schiano, where I think we can all agree the odds calculation actually UNDERRATES his odds of being let go. This guy might want to think about booking some tee times in late November if he continues at his current pace.
But the next coaches on the hot seat, Gary Kubiak, Leslie Frazier, and Mike Shanahan, should also probably get their acts together if they want to stay employed.
From an Eagles perspective, Chip Kelly doesn’t seem to be in too much danger (although as I said, we probably should eliminate all first year coaches as a general rule). Of course, this assumes Kelly behaves competently, unlike the absolute sh*t show we just saw against the Giants (which my brother is probably already dissecting)
Of course, from a FORMER Eagles perspective, it looks like Andy Reid has done quite alright for himself in the move to Kansas City.
I’m just saying.
Brent's Comment: Overall, I think this is an interesting exercise (obviously, or I wouldn't have run it). My biggest word of caution would be to not get too hung up on the percentages. Unfortunately, whenever you build a model like this, the results APPER to represent some precision. Due to all of the external variables we can't account for, that's not the case here. Instead, I think it's more useful to just view it as a discrete ranking of which coaches are most likely to get fired and how their relative ranking compares to others.
We could, of course, design it that way, and only show you the ranking, but that wouldn't be as fun. So we've chosen just to trust that you'll read this comment and use the data responsibly. Besides, transparency is important, if for no other reason than as another level of editing (the original version had a slight flaw, pointed out by a commenter, which is now fixed)
In any case, I think there are some additional steps we could take from here, and build things like Tenure, injuries, or maybe even Market Size into the model.
The value lies in the pre-season projections. We could potentially be able to project just how many games each coach has to win in order to keep his job. That would certainly make some teams more interesting to follow.