NCAA Tournament Prediction Accuracy Update: Revenge Of The Nerds

Looking at the most accurate pickers through the men's Elite Eight.

To call this NCAA men’s tournament “unpredictable” would be an understatement. For the first time ever, zero No. 1 seeds made the Elite 8, with no team seeded better than No. 4 landing in the Final Four. The average seed number for the last remaining quartet of teams (5.8) is higher than any tournament since 2002 except for 2011’s wacky Butler/VCU-laden Final Four (6.5). Looking at the pre-tournament predictions from a few weeks ago, it’s clear that nobody knew too much about what was going to unfold.

But some predictors still did better than others. After the first round, I used this handy tracker from the New York Times to look at the most accurate pickers (among a mix of humans and computers) through the Round of 64. I figured we should do that again now, updating the rankings through the Elite 8 — with a special twist.

Just like before, I’ll be judging each forecast by its Brier Score — the squared error between its probabilistic prediction and the actual outcome for each game of the tournament. But the special change now is that each round will carry different weights for its predictions, based on the escalating value of games by round in ESPN’s Tournament Challenge game. (The multipliers scale from 10 in the Round of 64, to 20 in the Round of 32, then to 40 in the Sweet 16 and, finally, 80 in the Elite 8.)

Multiplying the Brier Score for each game by the weight for each round and adding it all up over the whole tournament, here’s an updated ranking of NCAA pickers on the men’s side:

Early in the tournament, ESPN’s users had the best Brier Score of any picker, based on calling some first-round upsets more accurately. But the computer systems have gotten their revenge in the time since, with every statistical projection rising over both sets of user picks. In particular, Ken Pomeroy’s prediction has been the best of the bunch, as it gave higher probabilities to San Diego State and Florida Atlantic’s Final Four runs than the other data-based systems (much less the users, who were very down on both teams), while maintaining fairly typical odds for UConn and Miami.

That’s not to say it’s been a great tournament for any predictions: Pomeroy’s numbers called for a 1-in-65,871 chance that each of UConn, FAU, Miami and SDSU would make the Final Four. But in a tournament that put the Madness in March, it was still better to trust the numbers than your gut.