Data scientist, physicist, and fantasy football champion

QB reliability (a.k.a., A note to Marcus Mariota and Matt Stafford owners)

This morning I half-remembered an intersting tidbit about Marcus Mariota that my brother recently shared with me that he heard from somewhere else (I think a guest on The Fantasy Footballers podcast, but darned if I remember and darned if I’m going to ask him). Someone made a comment that Mariota did very poorly against top-half defenses and very well against bottom-half ones. This was followed by a joke about how that’s actually a desirable trait in a QB; you don’t have to worry about when to play him, he’ll tell you.

Armed with this third (fourth?) hand fact (joke?), it made me start thinking about fantasy QB reliability and trying to find when you know to play a QB. Were there other QBs out there like Mariota who can be relied on to do poorly against good teams and well against bad ones?

A lot of this analysis is just by eye. Deal with it. I just want to get a sense of things here and maybe tell an interesting story. If you want full on predictions I do that elsewhare. I’m trying to give something to those of you who like data-based narratives. If you’re like me and you end up streaming a QB often this may just give you the courage to finally play Jay Cutler*.

[* Author is not responsible for Jay Cutler-related damage to your league standing]

To get a sense of what a “good” or “bad” team is, here’s a graph of the average fantasy points per QB for each team:

.OppDiff-1.png

Some caveats: This is data from the 2015-2016 and 2016-2017 seasons, but ONLY for my guesses as to the starting QBs for this year. That includes a few guys (Glennon, Savage, Hoyer, Osweiler) who may or may not get the start this year and excludes guys who played last year but aren’t playing this year (Tannehill, Kaepernick, Kessler). This also includes games where a player was injured and sat out the rest of the game (artificaially low score). The analysis aggregates both years, so players or defenses who have improved or declined between years have all that data mashed together. My league also uses a slightly adjusted scoring system where sacks lose you 1 point, but really that should be the least of the concerns. Just keep all this in mind when you look at these numbers and see that they don’t exactly match what you’d find on Yahoo or ESPN.

Despite all of those warnings, defenses show up approximately where I think they should. First, I broke this up into 5 tiers (whenever I saw a small jump in the data, within reason). Next, I subtracted the player’s fantasy score each week by the average fantasy score by QBs against that team (i.e., if a team played DEN in a give week, the point differential is [QB score that week] - 10). Positive numbers are when a QB did better than the average QB does against that team, and negative numbers are when a QB did worse than the average. Below is a crazy plot of all that data:

.QBSkill-1.png

Players to the left consistently did better than the average player while players to the right consistently did worse. These names shouldn’t really surprise you. It’s just another way of saying that Tom Brady and Aaron Rodgers are good and always better than average and Jared Goff had a bit of a tough rookie year. If you’re looking for a Good QB, just grab someone from the left side of this figure, but that’s not the kind of reliability I’m looking for here. For those of us who stream QBs, you don’t get the option to just stream Aaron Rodgers. We need to look toward the center of this graph to find some value.

Sure enough, against easy teams (tiers 1, 2, and 3) Mariota outperformed their average while against the toughest tiers (4 and 5) he did worse than most other QBs. Derek Carr and Joe Flacco also do this to some degree, but Mariota really stands out.

I wanted to try and quantify the Mariota Effect, so I counted up the games where a QB overperformed against tier 1 or 2 teams and underperformed against tier 4 or 5 teams. The percentage of games in which a QB did well against a bad team and poorly against a good one is listed in the table below. Mariota, Carr, and Flacco are all up there along with Hoyer and Cousins whom I missed before when I was just looking at the figure. Against the best or worst teams these guys reliabily do better or worse than expected, respectively, more than 60% of the time. This isn’t a great metric though as you can see Prescott and Rodgers in the top 10; this analysis also catches guys who just always do well.

[Note: Seriously, if you have Aaron Rodgers, why are you even reading this? Go reflect on how good your life is and keep wondering why you only have 1 good RB on your team. Hint: it’s because you wasted a 3rd round pick on Rodgers.]

PlayerPctPosNeg
Joe Flacco0.764705882
Brian Hoyer0.666666667
Derek Carr0.64
Marcus Mariota0.619047619
Kirk Cousins0.611111111
Dak Prescott0.6
Jameis Winston0.590909091
Philip Rivers0.566666667
Eli Manning0.555555556
Aaron Rodgers0.545454545
Matthew Stafford0.545454545
Tom Brady0.529411765
Ben Roethlisberger0.526315789
Cam Newton0.523809524
Blake Bortles0.517241379
Tom Savage0.5
Andy Dalton0.476190476
Tyrod Taylor0.473684211
Matt Ryan0.434782609
Alex Smith0.423076923
Andrew Luck0.411764706
Drew Brees0.409090909
Jay Cutler0.4
Trevor Siemian0.384615385
Josh McCown0.375
Sam Bradford0.352941176
Carson Palmer0.35
Brock Osweiler0.333333333
Jared Goff0.333333333
Russell Wilson0.32
Carson Wentz0.222222222
Mike Glennon0

 


Finally, what about players who always do as expected? That’s another type of reliability that I’d be interested in. In the table below I took the absolute value of the point differential and found the median for each player. This is the list of the most “reliable” players, i.e., players who neither overperform nor underperform against defenses. Matt Stafford wins here with a median point differential of about 3.3 points. If you want someone who does as they’re expected, Stafford is your guy. Winston, Palmer, Bradford, and even Luck deserve mention here as well; they all do about as well as expected, though Luck managed to hammer some lower-performing defenses. Streaming Luck might be tough, though, so look for those other guys when they’re going against an easy team.

Way down on this list, you have guys who consistently overperform (Brees, Brady), underperform (Goff, Glennon), or are just generally unpredictable (Roethlisberger, though you get a lot of info about him knowing whether he’s home or away). Remember, I’m not saying they’re bad, I’m just saying that they don’t do exactly what you’d expect of them. Not like reliable old Matt Stafford. You play him against Denver and you deserve exactly what you get.

PlayerMedianAbsPtsDiff
Matthew Stafford3.328234483
Jameis Winston3.638634483
Carson Palmer3.6456
Sam Bradford3.768333333
Andrew Luck3.804038462
Philip Rivers4.049833333
Andy Dalton4.442307692
Matt Ryan4.567222222
Alex Smith4.591666667
Joe Flacco4.730328571
Derek Carr4.73037037
Jay Cutler4.7307
Dak Prescott4.732052174
Tyrod Taylor5.005
Josh McCown5.0928
Eli Manning5.128571429
Kirk Cousins5.250533333
Carson Wentz5.2849
Blake Bortles5.531680162
Brock Osweiler5.577692308
Russell Wilson5.768319838
Cam Newton5.808888889
Aaron Rodgers6.053878205
Brian Hoyer6.073571429
Tom Brady6.161570048
Marcus Mariota6.6025
Trevor Siemian6.745
Ben Roethlisberger6.791893939
Drew Brees7.123076923
Tom Savage8.955454545
Mike Glennon10.31769231
Jared Goff11.42818182

 


Conclusions

If you have Brady, Rodgers, or Brees, just shut up and play him. If you have Goff, Glennon, or Siemian, congratulations on your 32 team league. For the rest of us who stream QBs from week to week, you can do a lot worse than Mariota, Flacco, Carr, or Bortles when any of them are in plus-matchups. Additionally, Stafford, Winston, and Palmer are the most likely to give you exactly what you’re expecting in a given week. I have Stafford and I already have to stream in week 1 against Arizona. And remember, even if Winston is on fire for the first 3 weeks you had better think long and hard when he plays NE in week 5.

Or, you know, just look at my predictions. They’re really going to be great this year. Go check them out!

QB points by rank

Fantasy points per game by position