Black Box: Goalies

Robert Vollman
October 03 2011 10:14AM

 

 

Every week this season we'll be providing a statistical update on the Flames, broken up into four sections.  First, an OZQoC chart to clarify the roles each player has been assigned, then the team's success with them on the ice at even-strength, followed by special teams, and finally a look at the goaltending.

The decision of which statistics to include isn't an easy one, because a lot of the traditional goaltending statistics, like goals against average, shut-outs and wins, aren't really worth looking at.  Goals against average is as much a function of how many shots the Flames are allowing as the goalie's ability to stop them, and we already have save percentage to measure that.  Shut-outs are great, but your chance of winning isn't much different when you allow just one – consistency is far more important. 

Speaking of which, wins are more a measurement of how much the Flames are scoring than how many pucks the goalies are stopping.  Instead we'll look at two statistics: Quality Starts and Even-Strength Save Percentage.

Quality starts replace wins as a way of measuring how consistently the goalie is giving the team a chance to win.  They're only awarded when they stop an above-average number of shots (91.2%), or when the team allows very few shots and they play well enough to win by allowing fewer than 3 goals, and stopping at least 88.5%. Here's how this last section will look, using last year's data as an example.


Goalie           GS QS  QS%  ESSV%
Miikka Kiprusoff 71 41 57.7% .916
Henrik Karlsson  11  8 72.7% .905

Save percentage is often touted as the ultimate goaltending statistic, but it's not without its flaws. Even with Quality Start percentage helping us measure consistency, save percentage alone can easily be skewed, especially over such small sample sizes.

We'll look only at even-strength save percentage to avoid the effect special teams play can have, but there's little we can do to counter the way shots are recorded differently from one city to another.

Of save percentage, just remember that a lot of transient factors are involved, and it can take years to determine a goalie's true talent level.  Even Kiprusoff's even-strength save percentage has varied wildly from .907 to .941 since the lock-out, and that's with huge sample sizes.

Enough chit chat and preamble! Next week we'll begin - although with just a single game's worth of data.

This is hardly your last chance to offer suggestions, as we'll consider this a living effort, and tune it throughout the season.

781445e1560a4e1d0d641bc270a7eedc
Rob Vollman of www.HockeyAbstract.com is a regular feature writer on ESPN Insider, co-author of Hockey Prospectus 2010-11 and 2011-12, and regular contributor to NHL Numbers, Flames Nation and Arctic Ice Hockey. Innovator of Player Usage Charts, Quality Starts, GVS (Goals Versus Salary), the Snepsts Projection System, and known for work in League Equivalencies (NHLE). Twitter: @robvollmanNHL
Avatar
#1 Kent Wilson
October 03 2011, 10:30AM
Trash it!
0
trashes
Props
0
props

Interesting to see Karlsson's high QC percentage. Of course, the sample size is so small, not a lot can be taken from it.

Is there some way to place Kipper's QS performance in context of other starters in the league last year?

Avatar
#2 MC Hockey
October 03 2011, 10:53AM
Trash it!
0
trashes
Props
0
props

Nice job Mr Vollman! I really like alternate ways to measure players and it works especially well with goalies. Look forward to seeing it regularly...can you do it for the Jets too???

Avatar
#4 Domebeers.com
October 03 2011, 12:27PM
Trash it!
0
trashes
Props
0
props

I am an idiot. Explain the value of a stat that fluctuates so wildly?

Avatar
#5 Reidja
October 03 2011, 12:31PM
Trash it!
0
trashes
Props
0
props

Interesting analysis. Should we read into Karlsson’s quality start percentage? How many were because the team allowed very few shots? Could it be an effect of the team playing differently in front of him (i.e. more defensively?)

Some questions about these stats: 1) Where do the 91.2% and 88.5% thresholds come from? 2) What about the fewer than 3 goals threshold and how many shots is “very few”? 3) Finally, you often hear “the goalie has to be your best penalty killer”. Would there be value in displaying PKSV%? I wonder how well it correlates with QS% as this would include PKSV% from my read.

Avatar
#6 Clay
October 03 2011, 01:05PM
Trash it!
0
trashes
Props
0
props

FINALLY!

I am a goaltender myself, and will be the first to tell you that Kiprusoff's and Luongo's outright SV% are more comparable than the difference between the two suggests.

It is easy to inflate a SV% on a good team (I have done this through minor hockey and sr) and even easier to deflate it on a bad team (I have done this as well, playing very well in junior but underlying numbers dont appear to support the fact)

The truth of the matter is, the disparity in goaltending throughout the league has become very small. This is supported by the fact Detroit competes with "thin" goaltending and the total number of quality goaltenders on the free agent market.

I would also support the creation of a shot quality stat based on quality of competition (adjusting save pct. based on the opponents shooting pct. and win pct.)

Avatar
#7 rain dogs
October 03 2011, 01:30PM
Trash it!
0
trashes
Props
0
props

@Clay

Yup.

It's uncanny how much influence their is on goalies by the team in front of them. And most of that influence is completely unintuitive.

Quality starts is an interesting metric, but it is so heavily swayed by "team" as well.

You could cherry-pick last years... Oct. 28th? game vs Washington. Something like 4 or 5 PPGA... even if you look at only the even strength aspects, the team in front of the goalie with those kind of score affects, should consider it an outlier.

I was tracking quality starts, but considering "blow-outs" as outliers if at the hands of a poor PK. I think it gives a more accurate result.

But hey, I'm a goalie... so I'm a goaltender apologist.

Avatar
#8 Kent Wilson
October 03 2011, 02:09PM
Trash it!
0
trashes
Props
0
props

@rain dogs

Actually, the work being done now suggests that shot quality doesn't have much influence at all on SV%. Not in the NHL at least and not above natural variance (randomness).

Tom Awad's recent inquiry says the split for luck/other factors (skill, quality of team, etc) is about 70-30. Shot quality was the least relevant factor involved:

http://www.puckprospectus.com/article.php?articleid=558

One thing he notes is shot counter bias in some rinks (FLA, NJ and BOS are notorious). the NYR counter is also terrible at noting distance from the net as well, so those factors can sully investigations somewhat.

The guy finds a way to tease apart variance, skill and team effects for goalies will be a rich man I think. He could charge GM's a ton for that info, because clearly they have no idea as well.

Avatar
#9 Section205
October 03 2011, 02:16PM
Trash it!
0
trashes
Props
0
props

http://hockeyprospectus.com/article.php?articleid=921

I hope it's ok to refer to Robert's article at Hockey Prospectus.

Kipper's 57.7% is 16th out of 28 goalies with minimum 41 starts. Ahead of Roloson, Lehtonen, Halak, Lundqvist, Vokoun, Howard and others.

Goaltender GS QS QS% Tim Thomas 55 40 72.7% Roberto Luongo 60 41 68.3% Pekka Rinne 64 43 67.2% Sergei Bobrovsky 52 34 65.4% Antti Niemi 60 39 65.0% Jonathan Quick 60 39 65.0% Jonas Hiller 46 29 63.0% Marc-Andre Fleury 62 39 62.9% Corey Crawford 55 34 61.8% Ilya Bryzgalov 67 41 61.2% Carey Price 70 42 60.0% Ryan Miller 65 39 60.0% Cam Ward 74 44 59.5% Craig Anderson 49 29 59.2% Niklas Backstrom 50 29 58.0% Miikka Kiprusoff 71 41 57.7% Dwayne Roloson 54 31 57.4% Kari Lehtonen 68 39 57.4% Jaroslav Halak 57 32 56.1% Martin Brodeur 54 30 55.6% Michael Neuvirth 45 25 55.6% Henrik Lundqvist 67 37 55.2% Tomas Vokoun 57 31 54.4% Steve Mason 53 27 50.9% Jimmy Howard 63 31 49.2% Ondrej Pavelec 54 26 48.1% Brian Elliott 51 17 33.3% Nikolai Khabibulin 46 15 32.6%

(Sorry, this table did not paste as well as I expected)

Kipper's 41 QS is tied for 4th overall behind Ward, Rinne and Price. Ward had 74 starts and 59.5%.

Robert also identifies the stat Really Bad Start (RBS) and Kipper had 14 of those. I'll bet those 14 bombs really hurt Kipper's save% overall. Perhaps that could be seen as an argument that he was fatigued/overplayed?

I think there is room for optimism, because Karlsson's strong numbers over a small sample size is certainly much better than McElhinney and other backups Kipper has had.

Kipper should be able to rest more often and reduce the probability of fatigue and reduce his number of RBS.

Meanwwhile,if Karlsson can make about 20 starts with 65% QS (assuming he plays a few weaker opponents) then our goaltending should improve overall to get us 5 or 6 more points in the standings this year.

Avatar
#10 Section205
October 03 2011, 02:36PM
Trash it!
0
trashes
Props
0
props

http://www.hockeyprospectus.com/article.php?articleid=568

again, here's an article of Robert's which looks at 2008-09 and 2009-10.

Avatar
#12 Monaertchi
October 03 2011, 03:11PM
Trash it!
0
trashes
Props
0
props

So, let's see if I've got this right.

If a team is very offensive, meaning they have the puck most of the time and they have it in the offensive zone and therefore regularily allow few shots against, or/p>If a team is very defensive, keeping the puck to the perimeter or out of the defensive zone therefor allowing few shots against, then a goalie only needs to maintain a .885 SV% in order to rack up the QS%. If this is indeed the case, then it seems that this new metric isn't really telling us anything that EVSV% doesn't. I'd like to see Section205's table include EVSV% to see if this is correct.

Avatar
#15 Domebeers.com
October 03 2011, 03:56PM
Trash it!
0
trashes
Props
0
props

IDK, to be honest, I don't know how much of this stat is the goalie and how much of it is the team. IE: Where the team lets the opposition shoot on the goalie has a lot to do with how well a goalie plays. I think Clay and Rain Dogs already talked about it.

I would pair the stat with something else, something that tracked where the opp was allowed to shoot from.

Avatar
#16 wawful
October 03 2011, 04:15PM
Trash it!
0
trashes
Props
0
props

I think Karlsson's ice time last season was probably not a good sample of his ability for several reasons.

1) Too small to give good statistics.
2) He started against below average teams. (Average finishing league rank of opponents he started against was 19.3.)
3) He was most frequently deployed during back-to-back games, often ones in which the opposing team was also going back-to-back.

I would agree with others that it would nice to see a goalie stat that takes quality of opposition into account. There may be a difference between 40 shots from a first-place team when your defence is performing well and 40 shots from a last-place team when your defense has melted down. I'd also like to see something based on scoring chances and blocked shots in addition to just plain old shots.

It might also be worth looking for stats that correlate with changes in play-style. For example, players block a lot more shots now than they did in 2005. The Flames used to clear the shot-lanes when the opposing team was shooting from the blue line because they thought Kipper would have a better chance of making the save with a clear view of the puck. Now they block up the shooting lanes as much as possible. This probably means that a save percentage calculated purely from shots in 2005/2006 is not equivalent to one in 2010! e.g. In 2005, Kipper faced a lot more shots from the blue line that were easier to stop than the average shot he sees now. This may have caused his SV% to be somewhat inflated before blocking became the preferred team tactic.

Avatar
#17 Rain Dogs
October 03 2011, 04:27PM
Trash it!
0
trashes
Props
0
props

@Kent Wilson

I didn't actually say anything about shot quality did I? I talked about team...

I talked about even strength vs special teams.

Although, if we think special teams shot quality doesn't influence sv%/sh% at all....

why did we just sign Anton Babchuk to 2mil/season?

Avatar
#18 Section205
October 03 2011, 04:53PM
Trash it!
0
trashes
Props
0
props

Wow, lot's of good comments and discussion that gives me some new perspectives.

There is a lot of fluctuation for some good goaltenders. Is it really so hit and miss?

Also interesting that 2008-09 Thomas leads with 70.4%, then his teammate Rask leads 2009-10 with 69.2%, then Thomas leads again in 2010-11 with 72.7%. Surely Boston must be doing something right, which puts their goalies in the best position to succeed. I also believe that Boston has been one of the better 5 on 5 teams the last few years, which probably means strong EVSV% too.

Avatar
#19 Section205
October 03 2011, 05:39PM
Trash it!
0
trashes
Props
0
props

I wish every goalie could have a breakdown of the shots against:

Shots "expected" to be stopped (E)- 1300 (saves 1260 = .969) Shots "not expected" to be stopped (NE) - 500 (saves 370 = .740)

Total 1800 shots, 1630 saves, .906SV%

Obviously subjective, like scoring a fight. But if criteria were established for "expected" vs. "not expected" then each shot could be categorized as such.

Then you could point to poor team play when a goalie faces a relatively high proportion of NE shots.

The goalie can not control the quantity of "defensive breakdowns" such as breakaways, odd man rushes, giveaways, or missed defensive assignments due to sustained pressure. The goalie cannot clear the crease from screens without compromising his ability to stop the puck.

It is probably not fair or accurate to assume that every team has the same amount/proportion of these difficult events.

I notice that Tim Thomas and Roberto Luongo had Shootout Sv% of .526 and .538 respectively (obviously small samples)

Avatar
#20 Reidja
October 03 2011, 07:36PM
Trash it!
0
trashes
Props
0
props

@Robert Vollman

Thanks for the clarifications. It really helps one understand some of the work that has been put this. Again great analysis!

Avatar
#21 Jeff Lebowski
October 03 2011, 09:23PM
Trash it!
0
trashes
Props
0
props

These stats are very interesting akin to sabermetrics in baseball. However I can see with my eyes that Kipper is the best goalie in the league bar none.

Kipper makes all the saves other goalies make but he also makes saves that NO one else can.

The question with him in recent years has been stretches of poor play by him or team play. Somehow he still manages to get W's.

If I needed to win one game I'd pick Kipper.

Avatar
#22 Geoff Lebowski
October 04 2011, 01:36AM
Trash it!
0
trashes
Props
0
props

I'm Mr Lebowski, you must be the dude.

Avatar
#23 Rain Dogs
October 04 2011, 08:29AM
Trash it!
0
trashes
Props
0
props

@Section205

I understand what you're getting at with this, but inevitably one would use it to compare to other goalies.... and it'd be so subjective it'd be meaningless.

Think of shots against. A shot itself should be pretty clear, but we see with scorer bias.... over 10,000 shots that bias can make a lot of difference.

(I'm looking at you Cam Ward and Tomas Vokoun... as prime candidates)

Avatar
#24 Rain Dogs
October 04 2011, 08:51AM
Trash it!
0
trashes
Props
0
props

Occam's razor is: as the Flames ice an older, less talented, less elite team against the NHL norm, their goaltending is going to suffer. Age, sustained speed and puck pursuit (backchecking) don't put the defensive core/goalies in a position for success.

Out-shooting is one thing (fenwick and corsi) and that off. zone time IS important to our goalie, but what you do after opp. transition is the MOST important thing for defense/goalie success.

Odd-man rushes against.

Time to set up for a shot because of light back pressure.

Availability of passing lanes

Sure, Iggy and Tanguay get two or three shots against team X (Luongo/Niemi/Howard), but team X (Sedins/Joe Marleau/Pavel Zettersyuk) counter attack and only need one three-pass play to bury us, while we're coasting back, or the goalie is required to make a lateral sliding save like in the photo above^^^ vs getting hit square in the chest.

http://www.puckprospectus.com/article.php?articleid=540

It's why you see the Don Cherry #541 so often: "Goalie makes a big save, other team comes back to score." Transition shots against are the highest opp. quality/opportunity for. Shot quality models I've seen only track proximity (ft. from the net) to determine quality.... not defensive situations, which then renders shot quality meaningless.

A transition two-on-one SA from 15 ft out is always going to be higher quality than a sustained-zone, strong-side, stuff play into the goalies pads from 1ft, or even more than many breakaways.

Comments are closed for this article.