Thursday, January 14, 2010

Shooting and statistical significance

Sparked by a conversation I had on UMHoops yesterday, I sought out to figure out if the poor three-point shooting from Michigan this year is a regular statistical fluctuation or an accurate reflection of a team that's objectively worse than they were last year. My point of view is that the difference in the 33.4% shooting last year and this year's 29.7% is not really that out of the blue, and in fact, a fairly reasonable output given the data we have. And a random statistical significance calculator I found online seems to agree with me.

(DISCLAIMER: I did particularly poorly in Stats at Michigan and my logic herein might be flawed or flat-out wrong. I don't think it is and had someone much more knowledgeable on the topic look it over. A few times. I am also trusting that this calculator is doing the math correctly. My numbers come from the ESPN Michigan team page, so if they don't quite match the numbers you've seen elsewhere, this is where I'm getting them. But if my logic is correct here and this is a reputable calculator, then, well, read...)

I went into this experiment assuming that the difference in shooting percentage between the 2008-2009 Michigan hoops team and the 2009-2010 team--33.4% to 29.7%--was not statistically significant. Simplistically, this means that given the data, the 29.7% performance this year is a reasonable and unexceptional fluctuation in the team's output from last year. Furthermore, this would go to show that Michigan may just be the recipient of bad luck and experiencing a regular deviation from their performance last year, rather than being outwardly worse, as some suggest.

Now, someone who is really good with these sorts of things would be able to show you the formulas and explain what everything means. I can give you this:

So this nifty little program tells me I was right and the difference between these two shooting percentages is not statistically significant. But what does that mean exactly?

Well, like I said above, it means that we can likely say--albeit not definitively--that the 29.7% three-point shooting performance the team has shown this year is within the team's reasonable capabilities. That is, we can say that the performance this year is quite likely representative of the team's skill and performance given their outcomes last year. Again, this is not an exact process and is something to keep an eye on going forward, but we can't say definitively we're worse yet.

So my post At what point do we say: They just can't shoot, is mostly right. Though the performance this year may not indicative of the team's talent, it's difficult to say yet whether or not they're objectively worse this year than they were last year, at least for the time being.


Anonymous said...

What if you translated the shooting percentage difference between this year and last into a more intuitive points per game? If you take the number of 3PA this season (381) and multiply it by the differential percentage (33.4 - 29.7 = 3.7%) and then normalize it (15 games), the decline in shooting percentage is costing us nearly 3 points a game. In only 2 of our 7 losses (Bama by 2, BC by 4) would this extra 3 points have made a considerable difference. Still, I'd take 2 more wins...

You could take this analysis to another level by looking at each game independently and calculating the number of points lost due to the worse shooting.

Chris Gaerig said...

You'd also have to take into account the fact that the team is shooting a different percentage of their shots from three-point range this year than last (a significantly lower amount, to boot). Looking at the shooting percentage on a per-game basis basically reverts back to, "If we make more baskets, we win more games," which, of course. But what this analysis does is indicates that, while we may be underachieving, this deviation from last year's success is not unheralded or anything other than a reasonable fluctuation on last year's performance.

Furthermore, the sample size you get from a single game is not large enough to determine any sort of statistical relevance and make any determination other than if Michigan makes two more three pointers in game X they win.

Post a Comment