Do you miss the old FiveThirtyEight RAPTOR ratings? I sure do. But while those are no longer updating via Disney (I don’t know their long-term future fate), I have taken it upon myself to create the next best thing — a more basic version of RAPTOR that uses pace-adjusted box-score and on-vs-off data to estimate what the more full-fledged model (with player tracking/etc) would have said about a player.
You absolute legend - I was devastated to learn Disney had pulled the plug on RAPTOR, and a desperate, borderline hopeless Google search brought me here. This is exactly what I was looking for.
I've done enough social science to regard a correlation of .89 as extraordinary, and if you care to update your spreadsheet regularly I anticipate I will be using it a LOT this season - at least until the real thing gets back up and running. Thank you!!
Glad you found it! I'll be aiming to update daily, along with my NBA model that (again) isn't quite a 538 replacement -- it actually works very differently under the hood -- but should help fill something of a void in the market right now: https://themessenger.com/sports/nba-prediction-model-2023-24
Right now the only way is to make a copy of the spreadsheet for yourself and use Google Sheets' filter functions (or download it into Excel). I've been talking to Substack folks about how to make embedded charts friendlier, so I might add an interactive table at some point, but that's TBD.
Your accuracy on guard defense will improve massively if you can include just one more variable: offensive fouls drawn. Without it, you are missing one of the primary defensive statistics that correlates with RAPM. The direct value is ending the opponent possession is substantial, and the value in the RAPTOR regression exceeds the direct value of drawing the offensive foul.
Do some of the ratings systems tend to overrate players on good teams and underrate players on so-so teams? For instance, the Mavs' Dereck Lively has an EPM of 1.7 while the Bulls' DeMar DeRozan has an EPM of 0.9 and there is no way Lively is having a better season than DeRozan.
I haven't actually looked at that -- it's worth an offseason experiment to see if players who move between bad and good teams (or vice-versa) have RAPTOR changes in accord with that theory. One factor driving the difference here is that Lively's pure plus/minus impact is way better, which helps him (but that's all relative to/adjusted for team).
I would also say that comparing Lively and DeRozan is like apples and oranges. In any rating system, you should put fewer grains of salt into comparisons between players who are doing similar things or have similar roles. DeRozan is a high-usage wing who plays 38 mpg; Lively is a low-usage (if highly efficient) big who plays 24 mpg. They have practically nothing in common.
Final note is that this is where WAR comes in handy. DeRozan is demolishing Lively in WAR, and it's not just because he has played more games. His WAR/game is much higher.
(Also, I thought as I was first typing this that you meant RAPTOR had those respective numbers. But I am realizing DeRozan is rightly a lot higher in RAPTOR! But my points still stand above.)
Not closely-guarded at all! I can write more about that at some point when I have the time. But it's just based on BPM-style regression using per-100-possession metrics, blended with a plus-minus component, regressed slightly toward positional means (that's the "mild position adjustment") and adjusted for team offense/defense.
Dec 30, 2023·edited Dec 30, 2023Liked by Neil Paine
Hi Neil, love what you've done here. One question: why doesn't the TOTal RAPTOR column correspond more closely with WAR/82? For example Jamal Murray and Anthony Davis are both +4.1 in TOT, but AD's WAR/82 is twice as high as Murray's. I would have expected these two metrics to correspond much more closely.
Thanks! It looks like Murray is actually slightly lower than AD in eRAPTOR (+3.3 vs +4.1), but also Davis has played 30 of the Lakers' 32 games (69% of all possible minutes, which is actually what fuels WAR/82) while Murray has played only 20 of Denver's 34 games (37% of possible minutes). That big difference in % of minutes played explains most of the gap -- especially when we consider that WAR/82 is per 82 *team* games, not per 82 player games.
Totally fair assumption! I like per team games (in all sports) because it gives me a sense of full-season production pace, which I find easier to mentally contextualize as the season progresses (and equalizes between players whose teams have different numbers of games played)
Thanks Neil! I'm curious whether you think taking out the player tracking has a big negative effect on Jokic, or if it's just reflective of his (relatively) worse play this season. Haven't seen him not leading the Raptor board by a mile in about 3 years
I think it has some effect, with outlier performances being compressed some because we are less confident about true contribution without the extra tracking data. Last regular season, Jokic had 20.3 RAPTOR WAR vs Luka at No. 2 with 13.0. In eRAPTOR WAR last season, Jokic had 15.5 WAR vs Jimmy Butler at No. 2 with 12.7. So the gap is smaller between Jokic and the field in eRAPTOR.
In terms of Jokic's actual play this season, I don't think he's really lost anything from last year. His eRAPTOR has gone from +9.2 to +9.0. But it's worth noting that SGA is having an AMAZING season so far.
I'd love any feedback or ideas for how to make it better.
[This is my first time making a streamlit app (was a lot of fun!) and I haven't coded anything in 5 years... so if you're looking for something crazy... I'm probably not your guy!]
Let me know if you have any ideas / thoughts. I've had a lot of fun with the player comparison at the bottom (e.g. MJ v Lebron). It's also interesting to see the longevity of some players (check out John Stockton).
Hi Neil - Love the new new table! I was curious if it's possible to somehow track eRaptor overtime? I think it could be interesting to show trends throughout the season to show how players' play improves/declines.
The Messenger NBA model is based on rolling schedule-adjusted ratings from a team's past X games (50 for the regular season, 110 for the playoffs).
These are just estimated RAPTOR player ratings, but if you were to compile them into a team model, you would do what we used to do at 538 (which I don't have the bandwidth for right now): Keep updated depth charts for every team's current roster, project each player's RAPTOR and MPG, and compile those into team ratings that adjust for injuries, differences between regular season and the playoffs (shorter rotations, some players elevate their performance vs not, etc).
That's a lot of work and I wasn't completely convinced it was worth it in the end, but I could see that aspect being part of an ensemble that also includes Messenger-style rolling ratings and Inpredictable-style betting ratings. Maybe that is the next step.
Nov 21, 2023·edited Nov 21, 2023Liked by Neil Paine
Awesome, thanks. I noted in the spreadsheet tab of teams by team there was a net rating, but now looking again it looks like that isn't adjust, just point differential derived. Appreciate the work you are doing to keep the dream of 538 alive!
You absolute legend - I was devastated to learn Disney had pulled the plug on RAPTOR, and a desperate, borderline hopeless Google search brought me here. This is exactly what I was looking for.
I've done enough social science to regard a correlation of .89 as extraordinary, and if you care to update your spreadsheet regularly I anticipate I will be using it a LOT this season - at least until the real thing gets back up and running. Thank you!!
Glad you found it! I'll be aiming to update daily, along with my NBA model that (again) isn't quite a 538 replacement -- it actually works very differently under the hood -- but should help fill something of a void in the market right now: https://themessenger.com/sports/nba-prediction-model-2023-24
Thank you, this is awesome! Is there any way to sort by highest and lowest raptor by player for only the current season?
Right now the only way is to make a copy of the spreadsheet for yourself and use Google Sheets' filter functions (or download it into Excel). I've been talking to Substack folks about how to make embedded charts friendlier, so I might add an interactive table at some point, but that's TBD.
thank you so much for turning this into a 23-24 season leaderboard! you're the best!
Your accuracy on guard defense will improve massively if you can include just one more variable: offensive fouls drawn. Without it, you are missing one of the primary defensive statistics that correlates with RAPM. The direct value is ending the opponent possession is substantial, and the value in the RAPTOR regression exceeds the direct value of drawing the offensive foul.
That's a great point -- probably something to add next offseason. Seems like BBR has those consistently going back to 2005-06.
Neil--thanks for this! But do you have team totals, too? If so, I'll be glad to subscribe! Let me know at dreysamuelson3@gmail.com
So excited to see this. Thanks for putting this together.
Great stuff, Neil. Thank you
Do some of the ratings systems tend to overrate players on good teams and underrate players on so-so teams? For instance, the Mavs' Dereck Lively has an EPM of 1.7 while the Bulls' DeMar DeRozan has an EPM of 0.9 and there is no way Lively is having a better season than DeRozan.
I haven't actually looked at that -- it's worth an offseason experiment to see if players who move between bad and good teams (or vice-versa) have RAPTOR changes in accord with that theory. One factor driving the difference here is that Lively's pure plus/minus impact is way better, which helps him (but that's all relative to/adjusted for team).
I would also say that comparing Lively and DeRozan is like apples and oranges. In any rating system, you should put fewer grains of salt into comparisons between players who are doing similar things or have similar roles. DeRozan is a high-usage wing who plays 38 mpg; Lively is a low-usage (if highly efficient) big who plays 24 mpg. They have practically nothing in common.
Final note is that this is where WAR comes in handy. DeRozan is demolishing Lively in WAR, and it's not just because he has played more games. His WAR/game is much higher.
(Also, I thought as I was first typing this that you meant RAPTOR had those respective numbers. But I am realizing DeRozan is rightly a lot higher in RAPTOR! But my points still stand above.)
Would you be able to elaborate more on your process re: recreating 97-2013 RAPTOR via regression, or is that going to remain a closely-guarded secret?
Not closely-guarded at all! I can write more about that at some point when I have the time. But it's just based on BPM-style regression using per-100-possession metrics, blended with a plus-minus component, regressed slightly toward positional means (that's the "mild position adjustment") and adjusted for team offense/defense.
Hi Neil, love what you've done here. One question: why doesn't the TOTal RAPTOR column correspond more closely with WAR/82? For example Jamal Murray and Anthony Davis are both +4.1 in TOT, but AD's WAR/82 is twice as high as Murray's. I would have expected these two metrics to correspond much more closely.
Thanks! It looks like Murray is actually slightly lower than AD in eRAPTOR (+3.3 vs +4.1), but also Davis has played 30 of the Lakers' 32 games (69% of all possible minutes, which is actually what fuels WAR/82) while Murray has played only 20 of Denver's 34 games (37% of possible minutes). That big difference in % of minutes played explains most of the gap -- especially when we consider that WAR/82 is per 82 *team* games, not per 82 player games.
Oh ok, got it! I had assumed it meant player games ☺️
Totally fair assumption! I like per team games (in all sports) because it gives me a sense of full-season production pace, which I find easier to mentally contextualize as the season progresses (and equalizes between players whose teams have different numbers of games played)
Thank you! Where can I find the odds for each game as it was implemented on 538?
I don't have those listed anywhere, but I probably should at some point (if not just in the same way I have the "upcoming games" for NFL Elo: https://neilpaine.substack.com/p/2023-nfl-elo-ratings-and-win-projections)
That now exists: https://neilpaine.substack.com/p/2023-24-nba-elo-ratings
Thanks Neil! I'm curious whether you think taking out the player tracking has a big negative effect on Jokic, or if it's just reflective of his (relatively) worse play this season. Haven't seen him not leading the Raptor board by a mile in about 3 years
I think it has some effect, with outlier performances being compressed some because we are less confident about true contribution without the extra tracking data. Last regular season, Jokic had 20.3 RAPTOR WAR vs Luka at No. 2 with 13.0. In eRAPTOR WAR last season, Jokic had 15.5 WAR vs Jimmy Butler at No. 2 with 12.7. So the gap is smaller between Jokic and the field in eRAPTOR.
In terms of Jokic's actual play this season, I don't think he's really lost anything from last year. His eRAPTOR has gone from +9.2 to +9.0. But it's worth noting that SGA is having an AMAZING season so far.
Hi all - I put together a streamlit app to help visualize the data: https://eraptorviz.streamlit.app/
I'd love any feedback or ideas for how to make it better.
[This is my first time making a streamlit app (was a lot of fun!) and I haven't coded anything in 5 years... so if you're looking for something crazy... I'm probably not your guy!]
Whoa!! Awesome. Gonna check this out.
Let me know if you have any ideas / thoughts. I've had a lot of fun with the player comparison at the bottom (e.g. MJ v Lebron). It's also interesting to see the longevity of some players (check out John Stockton).
Hi Neil - Love the new new table! I was curious if it's possible to somehow track eRaptor overtime? I think it could be interesting to show trends throughout the season to show how players' play improves/declines.
Thanks! As part of adding the new leaderboard, I've been putting a CSV of the numbers up on GitHub:
https://github.com/Neil-Paine-1/NBA-elo
So I could, at some point, create a way to explore/visualize the archive over time. (It just doesn't go back to opening night, sadly.)
How would you summarize the difference between this RAPTOR model and your model over at the Messenger?
The Messenger NBA model is based on rolling schedule-adjusted ratings from a team's past X games (50 for the regular season, 110 for the playoffs).
These are just estimated RAPTOR player ratings, but if you were to compile them into a team model, you would do what we used to do at 538 (which I don't have the bandwidth for right now): Keep updated depth charts for every team's current roster, project each player's RAPTOR and MPG, and compile those into team ratings that adjust for injuries, differences between regular season and the playoffs (shorter rotations, some players elevate their performance vs not, etc).
That's a lot of work and I wasn't completely convinced it was worth it in the end, but I could see that aspect being part of an ensemble that also includes Messenger-style rolling ratings and Inpredictable-style betting ratings. Maybe that is the next step.
Awesome, thanks. I noted in the spreadsheet tab of teams by team there was a net rating, but now looking again it looks like that isn't adjust, just point differential derived. Appreciate the work you are doing to keep the dream of 538 alive!
Thanks! Yeah, that's just regular BB-Ref net rating. And the Elo ratings in there are just these basic ones: https://fivethirtyeight.com/features/how-we-calculate-nba-elo-ratings/
(I should say for anyone who wants them, I do keep the Elo ratings here: https://github.com/Neil-Paine-1/NBA-elo)
Interesting, thank you for that.
Hey, I'd love to help out with this. I do software and math and web apps.
Not impressed, Luka and Jokic defensive ratings hilarious. Horford was also roasted in the playoffs.
Why people HATE anal-ytics.