Primer on Predictive Statistics in College Football

You can’t have an offseason in college football without passionately arguing with strangers about how good your team will be before they’ve even played a down. Oftentimes these conversations involve hyping up a pre-season SP+ rating or arguing why the FPI is clearly missing your team’s hidden potential.

If you’ve ever found yourself in one of these conversations and wondered why you were talking to nerds in the first place what the differences were between FEI, SP+, and FPI, this post is for you! Let it serve as your primer to the common predictive measures that keyboard analysts like to toss around.

Each of the following ratings are intended to predict a team’s overall performance in future games. Most of them are not meant to rate a team’s resume, though Sagarin comes pretty close. All the ratings are adjusted for the team’s strength of schedule in some highly-suspect-but-better-than-anything-else-we-can-think-up way.

It’s important to remember what these ratings cannot do. They are meant to quickly evaluate over 100 teams each week without requiring the statistician to watch a second of game film, but they can only incorporate what’s in their data, and they have blind-spots for things that fans and Vegas may know (e.g., sudden injuries, individual player matchups).

With that out the way, let’s talk some stats!

FEI

The Fremeau Efficiency Index, or FEI, was created by Brian Fremeau in 2006. FEI is a measure of “opponent-adjusted possession efficiency” that captures “the per-possession scoring advantage a team would be expected to have on a neutral field against an average opponent.” [BCFToys]

Each possession is rated based on it’s starting field position and outcome. Scoring a touchdown when recovering a fumble one yard from the end zone is not as valuable as scoring a touchdown when pinned back behind a team’s own twenty. Possession-level ratings are aggregated into an overall efficiency number, and the efficiency number can be converted into an expected scoring margin.

For example, the 2021 Baylor team ended the season with an FEI of 0.69 (10th in the country). Against an average opponent at a neutral site, FEI projects Baylor would win by 17 points (0.69 scoring advantage per possession x 24 possessions per game).

Pre-season FEI projections are based on five-year historical ratings, returning production, and recruitment ratings. This will be a common theme among most of the different ratings.

https://www.bcftoys.com/baylor

SP+

SP+, originally called S&P+ for Success rate & equivalent Points per play, was created by Bill Connelly in 2005. SP+ uses play-by-play data to evaluate teams on five factors that Connelly identified as being highly correlated with winning: efficiency, explosiveness, field position, finishing drives, and turnovers. The overall SP+ rating is a weighted average of the five factors.

Efficiency is based on a team’s success rate, which measures how well a team gains the yards they need for a first down or touchdown. On first downs, a “successful” play is one that earns at least 50% of the needed yards. On second downs, a successful play earns 70% of the needed yards. On third and fourth downs, a successful play earns 100% of the needed yards.

Explosiveness is based on a team’s equivalent points per play (PPP). PPP is the yards earned per play converted to expected points from those yards. Field position is just the average field position margin between teams, finishing drives are the percent of trips inside the 40 yard line that yield points, and turnovers are...turnovers.

SP+ uses historical performance, recruiting, and returning production to generate pre-season ratings. These factors are gradually phased out of the SP+ formula as the season progresses.

SP+ is reported as expected points per game against an average opponents on a neutral field. Baylor’s 2021 team ended the season with an SP+ rating of 15.6 (15th in the country), so SP+ predicts they would beat an average opponent at a neutral site by 15.6 points.

F+

F+ is just an average of FEI and SP+, reported by Football Outsider. As far as I know, F+ does not have a simple conversion to expected margin of victory against an average opponent, and it tells you nothing that you can’t glean from looking at FEI and SP+ separately.

FPI

The Football Power Index, or FPI, is ESPN’s in-house predictive model for overall performance. FPI uses play-by-play data to calculate the expected points added (EPA) of each play throughout the course of the game.

EPA is adjusted for strength of opponent and converted into an expected margin of victory against an average opponent on a neutral field using relatively more sophisticated statistical modeling. A team’s FPI rating is simply this expected margin of victory, but individual game predictions also account for factors like number of days of rest between games and distance required to travel to the game.

Pre-season FPI ratings are based on four years of historical performance, returning starters, historical recruiting success, and coaching tenure. The weight placed on these factors is reduced throughout the season, but they never fully leave the model.

Baylor’s 2021 team ended the season with an FPI of 12.4 (15th in the country), so FPI predicts that Baylor would beat an average opponent on a neutral site by 12.4 points.

Sagarin

Jeff Sagarin is a sports statistician who has published ratings for different sports on USA Today for decades. His college football ratings were one of the six components of the BCS computer poll, and his ratings are still frequently cited despite their diminished importance in the Playoff age.

Sagarin does not reveal his exact methodology, but it seems to be based purely on wins/losses, margin of victory, and the strength of opponent. Defeating a higher rated opponent is worth more points than defeating a lower rated opponent, and winning by a wider margin is worth more points than winning by a smaller margin. If you know ELO rating systems, this should sound familiar.

The difference in ratings between two teams is the expected margin difference at a neutral site. Baylor’s 2021 team ended the year with a rating of 82.26 (10th in the country), so they would beat an average opponent (with a rating of 56) by 26 points.

Which rating is best?

Each rating system has its own advantages and disadvantages, and looking for consensus across the different ratings is probably smarter than trying to derive insight from where they differ.

Objectively, one of them is going to be the most accurate, but without real-time recording of game predictions, it’s difficult to make those comparisons yourself. ThePredictionTracker has done so for over 50 different ratings systems, but it doesn’t include SP+ or F+. Worth noting, the most accurate ratings in 2021 (according to absolute error) were the Vegas lines, averages of all the ratings, ESPN’s FPI, and FEI. The difference between FPI and FEI was miniscule in 2021 (0.01 average point error per game), but there doesn’t seem to be a lot of consistency in the top performers across years.

College football is no place to be reasonable, though, so here are my hot takes. Sagarin’s ratings use the highest-level data with a methodology that can easily be adopted across sports, so it is most likely to attribute flukey outcomes to a team’s skill and miss the nuance of the sport. Consistently moving the ball down the field but turning it over frequently and escaping with a 7 point win is counted the same as blocking a punt on the final play of the game and running it back for a TD.

ESPN’s FPI is probably well designed by people who take pride in doing their job very, very well…but as a product of a corporation with a vested financial interest in portraying certain teams and conferences as better than others…I will never look to it for knowing something the other’s don’t.

SP+ is intuitively very appealing. If a team does well in things that typically lead to winning, they are rewarded with a higher SP+ rating. The biggest drawback of SP+ is it’s behind ESPN’s paywall while all of the other ratings I’ve listed are freely available. To his credit, Bill is very active on Twitter (@ESPN_BillC), and you’ll often see him share some of SP+’s more interesting predictions.

I also like looking at FEI. Teams have different styles of play that may be unequally rewarded or penalized in SP+ and other play-by-play based systems (e.g., a screen heavy team that takes occasional shots downfield versus a more efficient but less explosive team), but at the end of the day, every team wants to maximize their points per possession and minimize their opponent’s. For this reason, F+ is also a nice go-to statistic, as it balances the benefits of SP+ and FEI.