Thoughts (entirely not mine) on the Crapshootout

by Quisp on Jan 28, 2011 11:22 AM PST in Weird Science

The comments section has been alive with debate on shoot-out stats. I thought I would write a post about it. But since the posts have already been written, I will cut/paste. Follow the links though; the articles are great; the blogs should already be in your RSS feed.

Crapshoot - Behind the Net
[...T]here is not enough evidence to suggest that individual skill levels make a difference in player shootout performance. [...] From 2005-6 through the 2009-10 regular seasons there have been 5711 shootout attempts. There have been 1878 goals which is a 32.88% success rate.

[...] There were 571 shooters and 112 goalies involved in this analysis. The players that are do well and those that do poorly are often those that we would not associate with being the highest skilled. [...]

[...W]e looked at year to year correlation among players with more than 10 shots per year and none of those correlation were larger than 0.1. Thus, player performance does not seem to be related (except about the mean) from year to year.

[...] the variability in goalie save percentages is roughly what we would expect for a binomial process. Again this does not suggest that there is not skill differences between players (in this case goalies) but it does mean that the randomness (or noise) of the shootout is completely dominating the skill of the players.

[...[ the effect of each player is the difference from the mean effect [...]. Among these intervals we found one goalie whose credible interval showed evidence of performing better than the league average. That goalie was [...] Marc Denis. We found that no shooter outperformed the league average but several underperformed. Among those with more than 10 shots, the shooters that underperformed were Michael Frolik 1/11 (9.1%), Marian Gaborik 2/18 (11.1%), Martin Havlat 3/18 (16.7%), DanyHeatley 4/25 (16.0%), Tomas Plekanec 2/16 (12.5%), Alexei Ponikarovsky 1/12 (8.3%), Taylor Pyatt 1/13 (7.7%), Bobby Ryan 1/11 (9.1%), Michael Ryder 4/22 (18.2%), Stephen Weiss 4/24 (16.7%). All of this evidence supports that notion that we can treat the NHL shootout as a coin flip. Now this does not mean that a coach should send out any random player. What it means is that for the shooters that are selected --- the best on each team --- and the goalies in the NHL there is not enough evidence after five years of performance data that among these players any one is better than another at the shootout. This is not to say that there is not skill involved in the shootout. Rather the skill is vastly overwhelmed by the noise and, so, the shootout is a crapshoot.

KuklasKorner : The Puck Stops Here : Top 20 Scorers By Shootout Goals Versus Threshold
[...] It is alarming that 10 staged breakaways by Sidney Crosby should have so much value. Sidney Crosby played 1778 minutes of hockey in the 2009/10 regular season. He is credited with 29.4 goals versus threshold on the entire season. In over 29 hours of regular season ice time, almost 13% of Crosby’s value comes from ten staged breakaways. That is far too much emphasis given to the shootout. This is a fundamental criticism of shootouts in general. [...]

One question that needs to be addressed is the repeatability. Crosby went 8/10 in 2009/10, but went 3/10 in 2008/09 in the shootout. There are not enough shootouts to establish reliable statistics. The player with the best numbers in any given season is lucky. His few chances that year are above his more established success rate that years of shootout data would suggest. Basically, the idea is that in ten flips of a fair coin, it is not that unlikely that you will see 8 heads of 8 tails a reasonable amount of time, even though 5/10 is the most probable number. The player who has luck on his side will put up the best number in any given season, when his longterm average level is well below it. In short, shootout success can be measured, but it has limited predictive value.

KuklasKorner : The Puck Stops Here : Top 20 Goalies By Shootout Goals Versus Threshold
The shootout can be extremely important in the valuation of [goalies]. Antti Niemi was worth 11 goals versus threshold last year in the regular season and over 30% of them came on the shootout. This shows that the shootout is far too important in the NHL results. It is a skills competition that is often essentially a coin toss that largely serves to keep the standings random and help ensure a false parity in the NHL. A couple goalies have enough shots faced to show that they are strong in the shootout. Evgeni Nabokov and Pekka Rinne are the two goalies who have the best case to show they are strong shootout goalies. It turns out that wasn’t enough keep Nabokov in the NHL.

The League of Extraordinary Statisticians: The Inaugural - Behind The Net
Greg Ballentine, The Puck Stops Here: Shootouts might be spectacle, but they don't do a good job of determining which team wins a game any more than a hardest shot competition would. [...]Teams have not been able to sustain shootout success from one year to the next. Few players show significant success in shootouts for a sustained period of years. It is not clear that it is much of a consistent skill and it is overvalued. A player who scores three goals on three shootout attempts is worth almost two wins to his team that season and that is far too significant a value placed upon essentially a fluke.

The dangers of Overtime Losses - Behind The Net
While the shootout and the overtime loss have probably made the NHL more exciting for fans, the way they're presented in the standings has made it very difficult to determine how good a team is. Right now, game outcomes are grouped into Wins, Losses and Overtime Losses so that we can see how many points a team has, but nobody tracks whether a team won its game in regulation or in overtime or the shootout. Some teams [...] can get incredibly lucky in the extra frames, leading to a record that's way out of line with their actual performance. This is a very key piece of information - there's no correlation between OT/SO performance and overall performance, a fact that's acknowledged by playing overtime indefinitely in the playoffs to decide a winner.

Shootouts: Goaltender True Talent - Behind The Net
It doesn't matter how we slice the data, we get roughly the same result. At the absolute most, a top ten goaltender is five points better in the shootout than an average goalie (28% vs 33%). Over an average season - 10 shootouts or 33 shots - the 28% goalie makes a .500 team into a .570 team. That is worth 0.7 points in the standings, or 0.35 wins. That's perhaps 10% of a top goaltender's total value above average, so it's not insignificant, but given that the best performer so far is Johan Hedberg, who's well below-average in the thousands of minutes he plays during regulation play, shootout ability is unlikely to drive a goaltending decision.

Shootouts: Does past performance mean anything? - Behind The Net
The problem with assuming past shootout performance is predictive of future performance is that the sample size is too small. When coaches keep calling a certain player in the shootout it's presumably because he's shown breakaway abilities over the course of far more practice and game repetitions than the few shootouts we get to see. That's why the "below-average" groups both get to keep shooting even though they've done poorly in the past and also ultimately end up with a league-average shooting percentage. Their coaches are good enough to assess who has league-average talent. The [...] absolute best few shootout takers in the league appear to be slightly above average. [...] A handful of very good shooters have 45% talent; the next ten have 37% talent; and the vast majority of the rest of the players in the league who get to continue in the shootout are right around the 33% league-wide average. [...] With an average of 10 shootouts per team over the least four seasons, that's a 0.4 point boost, or 0.2 wins. When you get that for free, it's nothing to sneeze at - it's double what I estimate the value of a good fighter is - but it's still on the order of what you get from having the equipment guys make sure nobody's playing with an illegal stick.

Tweet Comment 2 comments | 0 recs |

Do you like this story?

Comments

Agreed

It’s a coin flip. My biggest problem now is so are the actual games. With the gap control always perfect, the game has become so tight checking that most every game comes down to who’s shot hits the post and bounces out, and who’s bounces in. The first Kings game vs the sharks, the Kings didn’t play well, and were beat, plain and simple. Not a coin flip. But the three games since then, even though the Kings outplayed the sharks in game 2, it was still 0-0 going into the third period. Had Brad Richardson not been smoked, gotten up, had the puck magically be beneath him wit a 2 on 1 down low going to the net, and Lewis not gotten the first goal (and then the puck luckily bouncing through the goalie’s legs for goal #2), then it’s still 0-0 in the third, with both teams playing tight-checking hockey, and it’s a coin flip. One won faceoff in the offensive zone, one lucky bounce off a skate, it’s a coin flip. In game #3, it was basically the same game. Except when Setoguchi got his chance at the end of the 2nd period from the high slot, a shot he sometimes misses the net with, sometimes hits the post with, and sometimes puts top corner, he just happened to nail it that game, whereas the kings didn’t nail their shot. The kings outplayed the sharks that game, but lost. This last game, over half the game has gone by and no one can score because of how tight checking the game is, then smith gets a bit of a rush opportunity based on a bounce in the neutral zone (coin flip on that bounce), shoots it at the net, and it hits a skate and goes through Niemi’s legs. It could have missed the skate and not gone in. It could have hit the skate but bounced into Niemi’s pads, instead of the little 2 inch opening where the puck had to bounce in order to go in. My point is, when the league was more wide open, and there was room for people to skate, it was skill, ability, that determined the games. But when you have such a tight checking league, it’s the bounces off skates that determines games. That’s why there’s all this false parity. That’s why Nashville has a better record than the Washington Capitals. It’s like, if the league is so tight-checking that it essentially renders every Alex Ovechkin into an average player, that means every player is basically average, so it’s two equally averaeg teams facing off against one-another every night, so it’s a coin-flipped. It’s like the skill, the ability, the things that used to determine games the right way, when that is neutralized, it becomes a coin-flip. It used to be that, and the same as can be said with most great skills, if you were really fast, and great at stickhandling, you would give your team a warranted edge in winning the game. Like Ovechkin, he helped his team win games with all those amazing individual effort goals he scored. Yet, if the league becomes so fantastic at gap control and backchecking that no one can stickhandle or use their speed anymore, it’s neutralized. neutral means, you know, it’s 50/50. Sorry I’m having a hard time explaining this but it’s like, how many highlight reel goals has Ovechkin scored this year? How often do you even see him rush up the ice anymore? You see it about as often as you see it with Justin Williams. And that’s the point, if because of how the game is, Ovechkin is neutralized in Justin Williams, and can only score as much as Justin Williams… then having Ovechkin on your team, having that skill on your team, something that used tob e an advantage, and helped you win your share of games, deservingly, then having that skill doesn’t help you win your share of games anymore, and you’re only winning the same amount as everyone else, i.e coin flip. Obviously some games, say Vancouver vs Edmonton, the gap in skill is so great that it sifts through. But if it’s the Kings vs Sharks, the two biggest teams in the league, who both backcheck hard, where there’s no room to create anything, and no one makes a mistake all game, where there are no human errors, or no human amazingness, like an end-to-end rush goal (which we barely see anymore becaues of that tight checking), well, if the game isn’t going to be decided by a mistake, because no one is making any mistakes, and if the game isn’t going to be decided by an amazing individual play, which it usually isn’t because those amazing plays are very hard to do without someone making a mistake, then what decides the game? It’s just randomness. One team dumps it in, tries to hit people and keep the puck in, then is able to exit the zone, then they dump it in, then the other team exits the zone… every few times, one team isnt able to exit the zone cleanly, and the other team gets the puck, kicks it to their defense, and throws it on net, hoping for a bounce. Then the other team has that chance. it goes back and forth, territorially. Eventually a taem does get that bounce, and the scoring starts there. Whoever gets the most wins. It’s just randomness. When everyone is playing like a tight-checking machine, and the teams are relatively even, it’s just a coin flip. It used to be that the skill came through so well that even a subtle advantage in skil would come through in the games. But now it has to be a huge difference for it to sift through. Anything even close and its neturalized by the tightness of the game, in my opinion.

by Kingofthefans...air fans on Jan 30, 2011 11:44 AM PST reply actions

wow can’t believe I spelled Smyth, Smith. But my apologies in general beacuse I was in a rush and didn’t have time to preview this,so I didn’t realize how ridiculously long it was, and that I didn’t break up a single paragraph. I think Niesy wants these typse of things being fanposts, even if it’s about the coin flip in this topic. I mean should it be a fanpost or a comment since, other than its length, it is a “comment.” I mean it’s about this thread.

Anyway sorry it’s long and if it’s too long and there’s a way to move it to a fanpost please feel free.

by Kingofthefans...air fans on Jan 30, 2011 4:18 PM PST reply actions