As good as I am at all of this I can sometimes get a little excited to click “publish” and I don’t always double check my code. For example, I missed taking the absolute value in my accuracy scoring for weeks. That could have been disastrous, but fortunately it had little effect overall.

I say this because I just noticed that in my week 10 kicker prediciton post I used the wrong models for C and D because I just wasn’t looking closely enough. Here I changed it to the correct models, and it didn’t have large effect on the models, but I’m still a little frustrated. I’ll be more careful in the future (about this one particular thing).

In Week 10 I tested four of my own models. Every model used the same 5 factors (once I did it right):

- The team’s predicted score
- The opponent’s predicted score
- Which team is the home team (i.e., which stadium)
- Who the opponent is
- Whether the player’s team is home or away

There were slight differences in the models:

- Model A was a survival model (Weibull regression) using 2016 data only
- Model B was a survival model (Weibull regression) using 2015 and 2016 data
- Model C was a linear model (normal regression) using 2016 data only
- Model D was a linear model (normal regression) using 2015 and 2016 data

In the table below I also included the actual rank and scores for the kickers this week and the average rankings of the top 20 most accurate (kicker accuracy) from FantasyPros. It changes a little through the week but I collected it Sunday morning.

Rank |
Model.A |
Model.B |
Model.C |
Model.D |
FP.20 |
Actual.Rank |
Actual.Score |

1 |
Nick Novak |
Cairo Santos |
Nick Novak |
Dustin Hopkins |
Justin Tucker |
Dustin Hopkins |
17 |

2 |
Phil Dawson |
Dustin Hopkins |
Mike Nugent |
Stephen Gostkowski |
Stephen Gostkowski |
Ryan Succop |
14 |

3 |
Mike Nugent |
Nick Folk |
Dustin Hopkins |
Nick Folk |
Mason Crosby |
Brandon McManus |
14 |

4 |
Chandler Catanzaro |
Dan Bailey |
Jason Myers |
Nick Novak |
Chandler Catanzaro |
Steven Hauschka |
14 |

5 |
Dustin Hopkins |
Greg Zuerlein |
Chandler Catanzaro |
Josh Brown |
Matt Bryant |
Dan Bailey |
14 |

6 |
Jason Myers |
Nick Novak |
Josh Lambo |
Justin Tucker |
Dan Bailey |
Cairo Santos |
13 |

7 |
Josh Lambo |
Josh Brown |
Phil Dawson |
Chris Boswell |
Josh Lambo |
Chandler Catanzaro |
12 |

8 |
Caleb Sturgis |
Stephen Gostkowski |
Caleb Sturgis |
Connor Barth |
Brandon McManus |
Caleb Sturgis |
11 |

9 |
Steven Hauschka |
Chris Boswell |
Andrew Franks |
Mike Nugent |
Cairo Santos |
Phil Dawson |
11 |

10 |
Dan Bailey |
Graham Gano |
Josh Brown |
Josh Lambo |
Caleb Sturgis |
Greg Zuerlein |
10 |

11 |
Josh Brown |
Steven Hauschka |
Nick Folk |
Cairo Santos |
Stephen Hauschka |
Jason Myers |
10 |

12 |
Chris Boswell |
Connor Barth |
Brandon McManus |
Chandler Catanzaro |
Will Lutz |
Matt Bryant |
10 |

13 |
Brandon McManus |
Mike Nugent |
Chris Boswell |
Dan Bailey |
Graham Gano |
Justin Tucker |
9 |

14 |
Nick Folk |
Justin Tucker |
Dan Bailey |
Blair Walsh |
Robbie Gould |
Mike Nugent |
9 |

15 |
Mason Crosby |
Josh Lambo |
Matt Bryant |
Greg Zuerlein |
Chris Boswell |
Nick Novak |
9 |

It doesn’t really look like any of these models really nailed it, but let’s see what happens when we use my version of FantasyPros’ accuracy algorithm. Remember how the accuracy algorithm works? In Model A I said that Nick Novak would be the top player, which means that I predicted that he would get 17 points which is what the real #1 player (Dustin Hopkins) got this week. That means that my error is the difference between his actual score (9 points) and my predicted score. Do this for every player, take the sum and you have the model accuracy. You can also divide it by the number of players evaluated (15) to get the mean difference between my prediction and reality, but we’ll see in a second that those numbers are depressing

There are also a few other points of caution. You should check out the Week 10 DEF evaluation post to see all of these. It feels a little silly to repeat them all here.

Model.A |
Model.B |
Model.C |
Model.D |
FP.20 |

46 |
40 |
46 |
50 |
44 |

They all did about the same, but Model B was a little better than the rest. If I look at the distribution of differences for each player and run a t-test then none of these are statistically significantly different from each other. In other weeks Models C and D were the best and I guess in this week it was Model B. Still though, I’m proud that I’m somewhat close the FantasyPros experts predictions.

I can’t shake the feeling that I should be leaning toward models A and C which both use this year’s data. Things change between years: quarterbacks, coaches, offensive and defensive lines. Each of these things should affect a player from year to year, but this week the model that did best was the one that was slightly closer the the historical values (meaning from both 2015 and 2016). OK fine, so the means are close, but what if there are just more big outliers in Models A and C that pull their accuracy down? Maybe the distributions of differences change between the models, and we’d see that A and C actually skew lower:

Nope. Hmmm, weeelllllll, maybe a little.

The thick line in the middle of the box is the median while the top and bottom edges are the 25th and 75th percentile. What I wanted to see to justify my love of Models A and C is those boxes shifted a little lower with one or two big outliers up high (or the whiskers extending very high). The whiskers are indeed a little higher for those models and the outlier in Model A probably pulls its accuracy down a bit. Still, the median lines are all the same for each model and I can’t justify strongly preferring one model over the other unless that line was a bit lower.

The median player prediction was off by 3 points in my models and 2 points in FantasyPros. That’s a single extra (or one fewer) field goal, so it’s probably understandable. I’ll keep working on it.