I've seen the FiveThirtyEight NBA Projections get a lot of hate, especially for their Warriors prediction, but it really doesn't make sense to pick and choose teams when evaluating a model's performance. Looking at things holistically gives a much better sense of how it does.

Compared to consensus Vegas Odds - which is I think a fair benchmark - FiveThirtyEight's projections performs basically identically. On average, Vegas's predicted win total was off by 6.93 wins for a given team, while FiveThirtyEight was off by 7.27 wins.

Both approaches had some of their biggest whiffs for the same teams - way underestimating Cleveland, and overestimating Portland and Indiana. FiveThirtyEight did much better with the Lakers, Houston, and Brooklyn while Vegas did much better with the Warriors and Pelicans.

A lot of the error we saw was the result of random events that neither model could predict, namely injuries and trades, or of things very hard to predict, such as drastic changes in the performance of particular players or the performance of rookies.

Some of you might be saying, "Hey that's still worse than Vegas though - so it does suck!" To that, I say there's still a ton of value in a model that performs roughly as well as Vegas, which is that its assumptions are explicit and transparent. As an example, it had a very low projection for the Warriors, but it was easy to figure out that this was because of pretty low performance projections for Jordan Poole (2022 MIP candidate), Klay Thompson (coming off two major injuries... can you blame it?), and James Wiseman (negative value last year, did not end up playing this year). As a fan, I could look at the projections and still be optimistic by expecting those particular projections to be wrong. Vegas odds, on the other hand, are completely opaque and don't really tell you anything.

Here is the full breakdown if you are interested:

​