This is a quick Week 1 post-mortem for the QB model in the Predictive Playbook series. The pipeline ran end-to-end, published projections before kickoff, and now we’ve matched them to actuals to see how we did—and what to fix before Week 2.
🔗 GitHub Repo: https://github.com/ndbryant21-eng/fantasy-football-ml
📓 Week 1 Comparison Notebook: https://github.com/ndbryant21-eng/fantasy-football-ml/tree/main/notebooks
📊 Headline Results (2025 Week 1)
- Sample evaluated: 30 QBs (projections matched to box scores)
- RMSE: 9.14
- MAE: 6.73
- R²: −0.14 (Week-1 volatility in full effect)
- Bias (proj − actual): −3.38 (under-projection overall)
- Correlation (Pearson r): 0.32
- Coverage: within ±3 FP = 33%, ±5 FP = 50%, ±7 FP = 60%
Read: Mechanically sound, but Week 1 chaos + prior-season assumptions = wider errors and under-shooting the top end.
🧩 What Went Right (and Wrong)
What went right
- The Stream tier (common start/sit decisions) had a median error ≈ −0.73 → nicely centered.
- Feature signals behaved as expected: recent volume/efficiency, opponent pass-defense EMAs, QB rushing, Vegas totals.
What tripped us up
- Under-projection at the top end (spike weeks from stars).
- Team/role changes (new systems, new teams, rookies) aren’t fully captured in Week 1 when rolling features lean on last year.
- A few high-profile outliers stretched the tails.
🧱 Tier Takeaways
- Stream: Center is good (median near zero) but variance is wide—as expected for streamers.
- Start/Strong: Tended to be under-projected in Week 1. Expect normalization as 2025 data flows into the rolling windows.
🔧 What I’m Changing for Week 2
- Let 2025 data take the wheel: Rolling EMAs for defenses and player form will transition from 2024 priors to current-season quickly.
- Small tier-aware calibration: Light bias correction by tier to trim systematic under/over-shoots without overfitting.
- Uncertainty bands in the UI: Tier-specific “± FP” bands so readers can gauge start/sit risk.
(I’m deliberately skipping charts this week; we’ll resume visuals once the model settles in Week 2–3.)
🧠 Model Recap
- Model: XGBoost + post-hoc linear calibration.
- Features: L3 recent form (passing/rushing volume & efficiency, rates, ball security), opponent pass-D EMAs (YPA & TD rate allowed), opponent QB-rush defense EMAs, Vegas implied totals, home/away.
- Why EMAs? More weight on recency to reflect true shifts faster.
🛣️ What’s Next
- QB: Weekly publishes + short recaps; incremental calibration as 2025 data accumulates.
- New models: Rolling out RB, WR, and TE using the same framework (position-specific usage + opponent EMAs + context).
- Dashboard: A lightweight page for start/sit comparisons and matchup context.
If you want to check the code, artifacts, and weekly CSVs:
🔗 GitHub Repo: https://github.com/ndbryant21-eng/fantasy-football-ml
📓 Week 1 Comparison Notebook: https://github.com/ndbryant21-eng/fantasy-football-ml/tree/main/notebooks
Week 1 is noisy by nature, but the pipeline is in place and learning fast. Expect improvements in Week 2 and a stronger signal by Week 3.