Okay, by now I’ve hopefully posted my analysis of last year’s kicker models. Those of you who read it will remember that my predictions were about the same as a random guess. I’ve been working on models to improve my analysis, and I’m going to try a Generalized Linear Model, or GLM, first. I’m also working on learning Bayesian analysis, but I’m a long way from having something functional coming from that.
The idea is that my kicker data pretty clearly is not Gaussian. For one, the data is quasi-bounded at 0 (a few kickers got -1 points in a game, but it’s very uncommon). For another, there are some huge outliers when players have good games (see Cairo Santos’ 25 point game). Additionally, it’s not just the outliers, but the tails themselves have some weight to them; kickers get 12 or more points aout 17% of the time and 15 or more points about 5% of the time.
I’m going to just tell you about my analysis in JMP (statistical software) rather than show you because I feel silly saving and posting screenshots. If anyone wants the JMP data table with the models saved, just let me know and I can send it over.
The first thing I did was to check what function the distribution fits, and in JMP, the only distribution that works in the Gamma Poisson distribution, which is useful when your data is actually a combination of multiple Poisson distributions where each has a different parameter. Then I tried creating a few GLMs in JMP, and it worked really well. The first model I tried included:
- Team score
- Opponent score
- Home team (which stadium)
- Home or away
- The player (kicker)
- The team
Unsurprisingly, the last two terms were pretty heavily confounded and JMP couldn’t do a proper fit. I tried two new models, removing “player” in one model and removing “team” in the other. The paremeters looked a little different still, but both models said that home/away was not as significant as home team, so I removed the term for whether the teams were home or away. Finally, the opponent term is borderline significant (p-chisq ~ 0.13 in one model and 0.2 in another), so I get rid of that, too. That leaves me with two models. Both have Team score, opponent score, and home team terms and one has the player and the other the team. The models fit the data equally well, but I think the “team” model will be easier to code, so let’s evaluate that one based on last year.
Above I have the accuracy scores for 5 models for each week of 2016. The models are:
- The model I used last year (Model D)
- Random guesses for who will perform best each week
- Yahoo expert’s guesses
- The GLM I proposed this week including terms for estimated score, estimated opponent score, the home team, and which team I’m predicting for
- A GLM with score, opponent score, and team (excluding home team term).
It looks like the reduced GLM does the best, maybe by avoiding overfitting, but probably by dumb luck. In fact, I can tell you that the “model” term is not statistically significant (p = 0.25), nor is there a statistically significant difference between the random guess and the reduced GLM (p = 0.25 again). I’ll probably use it next year, but I still won’t be happy about it.
Well… this sucks. A bit of work for almost no improvement. I’ll bet my prediction intervals are a lot more accurate though since my function matches the shape of the data a little better.
Tune in next time to watch me continue to bash my head against a wall. I’m spending a little time learning Bayesian analysis techniques, so when I find out more about that I’ll likely give it a shot, but I wouldn’t expect anything fascinating.
At this point I think my best bet is to focus on new factors that might influence kicker scores. Maybe weather? I imagine wind and rain might have some effect, but it can’t be that large. Maybe one missed kick a game in the nastiest weather imaginable, but anything short of that is unlikely.
I’ll happily take suggestions via Twitter (@FFdatastream) as soon as I figure out how to use Twitter.