Measuring How NBA Players Help Their Teams Win

Dan T. Rosenbaum is an economics professor at the University of North Carolina at Greensboro. Besides this statistical work, Rosenbaum has been cited in numerous publications for his expertise on issues related to the NBA collective bargaining agreement and especially the luxury tax. He is thankful to the many remarkable individuals who have helped him tremendously in better understanding the NBA.

Interested in one of the top master’s in Economics programs in the country
or a new, innovative microeconomics-focused  Ph.D. program in Economics?

I. Introduction

“Good players lead their teams to wins.”  “Lots of players can fill up a stat sheet, but only the great ones are difference-makers.”  “The objective of a basketball game is not to accumulate points or rebounds or assists, but to win.  What statistic do you have for that?”  When I talk with knowledgeable basketball people who are skeptical of statistical analysis, I hear variants on these statements over and over again.  The argument is that winning is important and game statistics are only an imperfect measure of many of the contributions that players make to winning.

I very much take this argument to heart.  Basketball is not like baseball, a game structured around repeated one-on-one contests between pitchers and batters, where the contributions to winning of any given player can be measured well by individual game statistics.  Basketball is much more of a team game, and as noted by Dean Oliver, one of the leaders of statistical analysis is basketball, “teamwork is the element of basketball most difficult to capture in any quantitative sense” (p. 77).  While this argument often is overstated (as much of this teamwork can be measured using game statistics), the point still stands.  These limitations have led to new approaches to measuring the value of basketball players, approaches that make little use of game statistics like points, rebounds, and assists.

The most common approach is to compute plus/minus ratings that measure how point differentials change when a particular player is in the game versus when he is not.  Hockey has used such a plus/minus system for years, and now 82games.com is the first to make these data available for the NBA.  The logic of this approach is straightforward; teams should perform better when their good players are playing versus when they are not.  The intuitive appeal of this approach has not escaped teams’ attention, and my understanding is that most teams use plus/minus ratings to some extent.  However, these “unadjusted” plus/minus ratings do not measure the value of a player per se; they measure the value of the player relative to the players that substitute in for him.  In addition, there are differences in the quality of players that players play with and against.  A weak starter on a team with exceptionally good starters (relative to bench players) will generally get a very good unadjusted plus/minus rating – regardless of their actual contribution to the team.

Thus, a better measure of player value would “adjust” these plus/minus ratings to account for the quality of players that a given player plays with and against.  In addition, it would account for home court advantage and for clutch time/garbage time play.  Thus, unlike in unadjusted plus/minus ratings, these “adjusted” plus/minus ratings do not reward players simply for being fortunate to being playing with teammates better than their opponents.  Contributions for individual players are isolated statistically. In this article, I develop adjusted plus/minus ratings similar to the WINVAL ratings designed by Jeff Sagarin and Wayne Winston.[1]  I improve on past efforts by combining estimates of player value using both pure adjusted plus/minus ratings and a statistical index derived from these pure adjusted plus/minus ratings. This hybrid approach leads to player ratings that unlike press accounts of WINVAL ratings, pass the “laugh test” (p. 181).  In addition, the results from this approach are even less noisy than ratings based on traditional statistical indices alone.

Using data from the 2002-03 and 2003-04 seasons (with the latter season being weighted twice as heavily), I find that Kevin Garnett, Tracy McGrady, Andrei Kirilenko, Tim Duncan, and Shaquille O’Neal are the five most effective players in the NBA.  Replacing an average player with one of these five players would result in a team improving by about 14 points per 100 possessions or a little over 10 points per game.  In other words, in 2003-04 replacing one of the average players on the Orlando Magic with one of these five players likely would have made them a bit better than the New Jersey Nets and Memphis Grizzlies.

Perhaps more importantly, with these adjusted plus/minus ratings I am able to estimate what game statistics predict better performance on the court; these results help explain why certain players have such high adjusted plus/minus ratings.  It appears that rebounds are less valuable than typically assumed and steals, blocks, and avoiding turnovers are more valuable.  It also appears that having three point shooters on the floor helps teams and that players that can do it all – score, rebound, and assist – are more valuable than simply the sum of those game statistics.  In addition, even after accounting for all of those game statistics, players who play more minutes tend to be more valuable for their teams.  This finding suggests that coaches recognize those contributions to the team that are not measured by game statistics, and they play those players more minutes.

In this document I lay out a lot of the details of what I am doing and many of you may want to skip over those details.  If you just want to see my bottom-line ratings, go to Table 4 or Table 5.

II. A Discussion of the Set-up and Results

Here is the set-up that I use. Every observation is a unit of time in a game where no substitutions are made. There are more than 60,000 such observations per year in 2002-03 and 2003-04. With these data I run the following regression.

(1)        MARGIN = b₀ + b₁X₁ + b₂X₂ + . . . + b_KX_K + e, where

MARGIN = 100 * (home team points per possession – away team points per possession)[2]

X₁ = 1 if player 1 is playing at home, = -1 if player 1 is playing away, = 0 if player 1 is not playing

X_K = 1 if player K is playing at home, = -1 if player K is playing away, = 0 if player K is not playing
e = i.i.d. error term

b₀ measures the average home court advantage across all teams
b₁ measures the difference between player 1 and the reference players, holding the other players constant
b_K measures the difference between player K and the reference players, holding the other players constant

The reference players are all players playing less than 250 minutes in both seasons combined.  Observations are weighted by the number of possessions with (1) observations in 2003-04 weighted twice as heavily as those in 2002-03 and (2) higher weights during crunch time and lesser (or zero) weights during garbage time.[3] 

It is in this regression where the effects of the other players on the floor are accounted for.  The bs in equation (1) measure the point differential difference (measured per 100 possessions) of the given player relative to the reference players, holding constant all of the players that shared the floor with that player (and with the reference players), i.e. holding the other players constant.  What does this “holding other players constant” mean?  Strictly speaking, it means that we can take a player and surround him with four teammates and five opponents and compare how that player’s team would do versus how it would do if he was replaced by a replacement player keeping all of the other players the same.  This is what is meant by “holding the other players constant,” since we can repeat this exercise with any other combination of other players.   

Another way to think of these bs is that they are plus/minus statistics adjusted for the other players on the floor.  This takes out the effect of a player who is fortunate to always play with Kevin Garnett or unfortunate enough to always being matched with rookies or NBDL players. 

Table 1 presents the results from equation (1) for the top twenty players among those playing 250 minutes or more in the 2002-03 and 2003-04 seasons combined.  (I normalize these ratings so that the average player is given a value of zero.)  Kevin Garnett has by far and away the highest “pure” adjusted plus/minus statistic, being a full 19.3 points per 100 possessions better than the average player.  In addition, this estimate for Garnett is quite precise with it being statistically significantly different from most the rest of the players in the top ten.  The rest of the top ten (with the exception of Nenê and players with high standard errors and less than 1,000 minutes) are among the top players in the game – Vince Carter, Andrei Kirilenko, Dirk Nowitski, Tim Duncan, and Shaquille O’Neal.  The next ten contains another five players among the top players in the NBA – Rasheed Wallace, Ray Allen, Tracy McGrady, Baron Davis, and John Stockton.

That said, there are a number of outliers in the top 20.  Six of those nine outliers (Richie Frahm, Jason Hart, Mike Sweetney, Mickael Pietrus, Earl Watson, and Carlos Arroyo) have standard errors ranging from 4.3 to 6.3, so these players’ high ratings could mostly reflect sampling variation – although Frahm’s rating is so high that despite the high standard error and low number of minutes, it probably is something more than sampling variation.  Three players (Nenê, Jeff Foster, and Eric Williams) seem to have genuinely quite good ratings that cannot be explained away by sampling variation.  Foster replaced an All-Star in Brad Miller and his team did not miss a beat, ending up with the best record in the League.  Nenê played major minutes for a team that improved dramatically in 2003-04, and Eric Williams played on two teams (Boston and Cleveland) that played their best basketball of the season while he was with them.

However, even taking all of that into account, these ratings are quite noisy.  Another approach is probably necessary for these rating to be that useful.  Below I outline that approach which combines these pure adjusted plus/minus ratings with ratings derived from the relationship between game statistics and these pure adjusted plus/minus ratings.

(I also present offensive and defensive ratings that are based on the pure adjusted plus/minus rating plus an “efficiency” rating that measures how many points per possession are scored by both teams when a given player is one the floor.  By combining these two measures, I create offensive and defensive ratings.  However, given that I am using two imprecisely estimated ratings to arrive at these offensive ratings, I suspect these rating are measured with quite a bit of error.)

Table 1: Pure Adjusted Plus/Minus Ratings for the Top 20 Players in 2002-03 and 2003-04

(Full Table for All Players)