Data scientist, physicist, and fantasy football champion

Week 10 DEF - Evaluation

Another week on the books, and another week in which my predictions were good overall but a prediction for any particular team probably didn’t quite land. I had picked up LA last week based on my predictions and they ended up with 10 points. Good, but nothing amazing. Also toward the top of my list was SD which didn’t quite make it to the top with 3 points. I won this week by less than 6 points, so in a way I won because I followed my own advice. In another (more accurate) way I won because LeGarrette Blount scored 3 TDs.

I certainly hope nobody was expecting perfection.

In Week 10 I tested three of my own models. Model A included:

  • The team that’s playing
  • Who their opponent is
  • Their opponent’s predicted score
  • Their predicted score
  • Whether the team is home or away

Model A also used only 2016 data (i.e., weeks 1 through 9). Model B used just the first 3 terms above and only 2016 data. Finally, Model C used all 5 terms like Model A but used data from 2015 (weeks 1 through 17) and 2016. The top 15 teams from each of these models is:

 

Rank

Model.A

Model.B

Model.C

FP.20

DWC

True.Order

True.Score

1

LA

SD

LA

ARI

ARI

TB

24

2

KC

LA

GB

LA

BAL

MIA

17

3

SD

KC

KC

MIN

WAS

BAL

14

4

MIN

PHI

ARI

HOU

NYJ

KC

13

5

GB

MIN

HOU

NYJ

CAR

HOU

12

6

PHI

GB

BAL

KC

KC

TEN

11

7

ARI

ARI

CAR

BAL

SD

SF

10

8

CAR

BAL

PHI

CAR

MIN

NO

10

9

BAL

NYJ

MIN

SD

NYG

LA

10

10

CHI

CAR

WAS

NE

HOU

DEN

9

11

NYJ

MIA

NO

DEN

LA

NYJ

7

12

MIA

CHI

NYG

WAS

PHI

SEA

6

13

WAS

WAS

SD

GB

CHI

NYG

6

14

DEN

DEN

DEN

PHI

TB

ARI

6

15

ATL

ATL

NE

CIN

NE

WAS

6

If anyone knows how to easily put tables into a site using SquareSpace, please let me know!

In this table I also included 2 other available models that I look at every week and the final “true” score. “FP” is the average of the top 20 DEF experts on FantasyPros. It changes a little throughout the week, but this week I picked it up Saturday morning. “DWC” is Defense Wins Championships written by Dylan Lerch. I’m choosing his in particular becuase I read it every week, it’s a very well-written article, the analysis seems well thought out, and before I started modeling this myself I always used his weekly recommendations.

[I also may go back to that if my modeling doesn’t pan out. My desire to win is far greater than my pride.]

Note also that the score is what I calculated based on the data from The Huddle and the scoring in my league (Yahoo standard). This is looking like a bigger caveat than I thought, actually. For the most part my scores line up closely with the scoring for my league in Yahoo, as it well should since I based my scoring algorithm on these numbers to help me win in my league. There are times, however, when the numbers from The Huddle don’t match what Yahoo (my league) or FantasyPros use as the weekly score. Usually the difference is just one point, but one bad example: in week 9 Yahoo records BAL as getting 1 TD but The Huddle didn’t. FantasyPros record BAL as 8 points (with no TD), I get 5 from The Huddle, and my league in Yahoo claims 13.

Remember how the accuracy algorithm works? In Model A I said that LA would be the top team, which means that I predicted that they would get 24 points which is what the real #1 team (TB) got this week. That means that my error is the difference between their actual score (10 pts) and my predicted score. Do this for every team, take the sum, and boom, you have the model accuracy.

Yet another note: what happens when a team that isn’t in your top 15 ends up in the top 15 for the week (like TB this week)? If they were in the overall list that you submitted to FantasyPros it’s no big deal and it scores them the same way as above (your the 23rd ranked team is predicted to get whatever the true 23rd ranked team got, etc.). If, however, you didn’t rank them at all FantasyPros says you ranked them at 1 spot less than the lowest Pro ranked them. That’s all a little much for me to code and would probably lead to weird effects if I’m only comparing my own models and the average FantasyPros models. Therefore, I just say that the prediction was for whatever the #15 team got this week minus 1 (that’s 5 points this week). I think the discrepancy in the points between scoring systems is a bigger issue than this, so that’s where I’m going to devote my debugging efforts for now. Any complaints will be formally accepted and summarily ignored.

On to the scoring!

Model.A

Model.B

Model.C

FP.20

DWC

80

78

63

65

85

TB is definitely the shocker for the week, but two of my models also thought SD would be a lot better and BAL a little worse. Model C seems to be the winner this week, although the FantasyPros experts lived up to their name.

When you consider that the accuracy score for a model is the sum of the differences between predicted and actual scores for 15 teams, that means that a score of 63 is equivalent to an average difference of about 4 points. If you removed TB and MIA (which drastically overperformed their yearly average and my prediction) it wouldn’t look quite so bad. Then again, if I removed all the outliers I’d have perfect accuracy. That kind of thinking is dangerous. As much as I want to write off weeks like this I just can’t as a data scientst. This is real data, and removing outliers just because they’re outliers is not allowed and leads to anecdotes clouding my model. If I were reporting error bars I’d just have to broaden them a little and remind myself that even with an enormous 95% prediction interval there would always be 1 or 2 teams outside of it every week.

I’ll keep refining my models. For now though, Model C seems to be the model to beat. If I get time this evening (Tuesday) I’ll post this as well as week 11 predictions before the waiver deadline. If not… then I won’t, I guess. I’m a busy man and the only people currently reading this are exactly the people I don’t want to help.

Week 10 K - Evaluation

Week 10 K predictions