Data scientist, physicist, and fantasy football champion

Quarterback data exploration, part 2

Last year I explored the QB data set a little, but there were a few things that I missed that I wanted to follow up on. The big one is the effect of home vs. away on an individual QB. I wanted to answer the question of whether Ben Roethlisberger is actually better at home as conventional wisdom has it. There are a few times where I found that conventional wisdom was not statistically significant (defenses home vs. away, kickers in certain stadiums), so I wanted to follow up on this one. It turns out that in the case of Roethlisberger the answer is two-fold: 1) yes; and 2) moreso than any other QB I mean you knew the answer to this why won’t you just believe people when they tell you things?

I’m using my package (datastreamr) to read the data. I also wanted to write this post because it will let me test the package and see if there are any functions I could use for exploration. Let’s get to it:

Filtering out the top QBs

There are a lot of backup QBs in this data who played 1 or 2 games and maybe only parts of those games. I don’t really want to use them in the data processing because I don’t know what to expect from them and their limited play time will likely throw off my analysis.

Well shit. I was hoping for three groups: Starters who played almost every game, starters who took over for an injured guy (maybe 4-8 games) and injured guys or backup QBs who played only a few games. What I got instead is what looks like a pretty clear break at 8 games between starting QB1s and the rest of them. There are a couple of exceptions; four guys who played 2-6 games did fairly well in terms of average points per game (Hoyer, Barkley, Foles, and Matt Moore) and four starters with 8 or more games had fewer than 12 pts per game (Osweiler, Keenum, Fitzpatrick, and Kessler).

Still, for this analysis I think I’d rather have guys who have played more than a specific number of games rather than just the best QBs in terms of points per game. I want enough games for decent statistics, so I’m going to draw the line at 8 games since it feels like there’s a natural break in the data there. Just remember that whenever I say “QBs play better at home” what I mean is “Starting QBs who played more than 8 games last year played better at home”. Filtering out only guys who played more than 8 games still leaves 31 QBs from last year, and that seems like a good set to play with.

QB fantasy points by home vs away

I made this figure in the last QB data exploration article, but now that I have the data from all of 2016 I want to take another look. Let’s run a T-test to see if the means are significantly different:

##  Welch Two Sample t-test
## data:  filter(data_QB_starters, HomeAway == "Home")$fptsstd and filter(data_QB_starters, HomeAway == "Away")$fptsstd
## t = 1.9766, df = 453.48, p-value = 0.04869
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.00777651 2.69198950
## sample estimates:
## mean of x mean of y 
##  17.66411  16.31422

It looks like I should include Home/Away in my models since QBs do a little better at home (difference in means = 1.35, p-value = 0.049 < 0.05). But what about individual QBs (cough Roethlisberger cough)? Some QBs (cough) are known to be much better at home. Is it worth it for me to create a model in which I use each individual QB’s home/away record to determine their home/away correction? If it’s worthwhile it could lead to better predictions, but if it’s statistically insignificant it could easily lead to overfitting and poor predictions.

Each quarterback’s Home/Away record

Here I broke out each QBs home/away scores and plotted them as box plots. The thicker line in the middle is the mean while the box represents the 25% and 75% quantiles (i.e., 50% of the scores fall within that box). Anyone with boxes that don’t overlap much is probably better at home or away. Check out Ben’s Boxes! You could stack those damn things!

Overall, most QBs do a little better at home which isn’t surprising given what we saw in the previous section, but individually we can see that most do better at home but a few actually do a little better away. I’d be a little surprised if some of these were consistent from year-to-year and not just unique to 2016, so I’ll have to reevaluate this yet again when I download 2015 data (and then again next year when I have some 2017 data, but by then I’ll be busy crushing my idiot brother and his idiot friends in our league).

Let’s run a t-test on each of these to see if any of them are significant:

Player Difference between Home and Away T-test p-value
Ben Roethlisberger 11.70 0.010
Derek Carr 4.74 0.060
Eli Manning 5.10 0.077
Cody Kessler 5.90 0.092
Jameis Winston -5.25 0.101
Matthew Stafford 6.48 0.121
Alex Smith 4.52 0.162
Carson Wentz 2.88 0.175
Ryan Tannehill 4.55 0.209
Tom Brady -5.24 0.243
Dak Prescott 5.28 0.264
Carson Palmer 2.91 0.354
Andy Dalton -1.34 0.545
Tyrod Taylor -3.02 0.550
Aaron Rodgers -2.25 0.556
Trevor Siemian -2.60 0.568
Case Keenum -2.74 0.584
Ryan Fitzpatrick -1.70 0.681
Kirk Cousins 1.38 0.682
Marcus Mariota 1.04 0.744
Cam Newton -0.96 0.799
Sam Bradford 0.73 0.813
Colin Kaepernick 1.01 0.834
Andrew Luck 0.68 0.846
Blake Bortles -0.61 0.851
Drew Brees 0.54 0.860
Russell Wilson 0.57 0.884
Philip Rivers 0.61 0.885
Matt Ryan 0.23 0.922
Brock Osweiler -0.31 0.925
Joe Flacco 0.26 0.955

In the table above I show the difference between mean home and away fantasy scores and the p-value of the t-test for each player. If we put aside statistical arguments for just a second, it looks pretty clear that Roethlisberger is much better at home with an average of 11 points more at home this year. That’s huge! The next closest were Russel Wilson and Dak Prescott with barely half that difference. With a p-value of 0.010 it defenitely looks like it might be significant. Plus, he only played in 14 games this year. If you can get a p-value of 0.01 on only 14 points you’re doing okay.

Unfortunately, breaking data out like this screws with your family-wise error rate. When you run multiple tests you have to control for the fact that you’re more likely to get p-values that look significant based on random data. You can control for this a number of ways, but the most conservative is to divide the usual p-value you’d accept (often it’s 0.05) by the number of groups (31) to get the new p-value that we want to beat (0.05/31 ~ 0.0016). Not even Big Ben reaches that.

I’m not certain what other techniques I should use to correct for this. I can try something like resampling, but I’m a little concerned about the results with so few data points already. But maybe that’s the point of it? This will require a lot more research. For now though I’ll try models both with and without the Player*HomeAway interaction term, but as far as statistical significance is concerned I’m going to need a lot more data to say whether an individual player beats the 1.5 point difference between Home and Away games that QBs get overall.


Short version: we can say with some confidence that there is an overall trend for QBs to do better at home (by about 1.5 points), but we can’t say definitively whether any individual QB has a number higher or lower than that. Roethlisberger has the biggest Home/Away gap but I still can’t say that he has a statistically significant difference from that 1.5 point difference. I’ll try both models next year and see what happens.

Conclusions: datastreamr and R in general

I’m writing this at the 2017 Rstudio Conference after a few days of learning new techniques and my life has been dramatically improved already. I used the map() and map2() functions to break out the data and run the t-tests, respectively, and tried using a tibble. No luck with the forcats package though. The QBs were initially in non-alphabetic order and I tried to use fct_reorder() to reorder them, but I couldn’t get the syntax right and ended up back in base-R with good old levels() and sort().

I didn’t come across anything new that I need to add to the package because I’m still exploring the tidyverse packages that I’ve been ignoring. For these data exploration posts I’m either doing something that only requires a few lines (like making the figures above) or something that I don’t really need to do every week (running a series of t-tests). No changes to the package this week.

Exploring the weather

Quarterback data exploration