The Value Of An NHL Draft Pick
Jonathan Willis
March 10 2011 06:58PM
Late last month, Jason Gregor looked back at the NHL draft from 1996 to 2005, in an attempt to put a value on NHL draft picks. Gregor’s far from the first to do so; after the jump I’ll look at his work and that of others to see what we can say about the NHL Draft.
Jason Gregor’s Take
Let’s start with Jason’s article. There are a lot of things to like about it; in particular the fact that the dataset is huge (10 years, ~2700 players) and the criteria is relatively straight-forward. That said, there are two things that I think could be improved: first, there is some subjectivity in the player ratings, and second Gregor breaks things down by round, meaning that we don’t know how much difference there is between a top-10 selection and a 10-20 selection. The latter is more of a problem than the former, as we will see later on.
It’s still quite a good article, and the key points I took from it were as follows:
- A first round selection is far more valuable than any other selection – a player taken in that range has a slightly better than 60% chance of turning out to be a “decent” or better player; nearly three times the rate of the second round.
- Second and third round picks are a cut above the rest of the draft, although the chances of landing a “decent” or better player are quite low: a little better than 1 in 5 for the second round, and slightly better than 1 in 7 for the third round.
- There’s little difference between picks made from the fourth to ninth rounds of the draft; over those rounds players are roughly half as likely to develop into a “decent” or better player as a third round pick.
Scott Cullen’s Take
Over at TSN, numbers guy Scott Cullen did something similar to Gregor’s article in mid-2009. Like Gregor, he used a 10-year period (1995 to 2004) and subjective rankings. However, Cullen broke the draft numbers down into groups smaller than full rounds – groups of five for the first 60 picks, and then groups of 15 afterwards. A few months later, Cullen clarified things even further for the first round – using 30 drafts and breaking down the top thirty picks by each draft spot.
The key points I took from Cullen’s work:
- There’s a huge gap between a top-three selection and a four through six selection, which is followed by another gap between the sixth spot and the rest of the draft.
- Again, there’s a big drop between the first round and the second/third rounds, and another big drop from about the 100th pick on, at which point things flatten out – there’s very little difference between drafting 105th and 195th.
- A player taken after the first round but in the first hundred picks has roughly a one-in-three to one-in-four shot at hitting the 100-game plateau in the NHL.
The Hockey Prospectus Take
Again from the summer of 2009, articles by two different writers at Hockey Prospectus. Richard Pollock considered both first round selections and late round selections, while Tom Awad considered the entire draft. Pollock uses both points and games played, while Awad uses Goals-Vs.-Threshold, a unified statistic that compares the value players of all positions generate above replacement-level players.
All the articles are good, but Awad’s method does the best job of comparing players across position objectively, and I like the fact that he weights his findings to reflect the fact that immediate production is better than later production. Plus, his fitted graph does the best job of showing how precipitously draft pick value drops off.
Key points:
- The first few picks in the draft – especially those in the top five – are vastly superior to other picks, including picks later in the first round.
- Once again, picks outside the first round but in the top-100 are roughly equivalent in value – there is some value in having a higher pick, but that value is not especially strong.
- After the first 100 or so picks, there is almost no difference in the value of a draft pick.
Conclusions
Despite the different methods used and the various writers employing them, the findings from one article to another are extremely similar. I would argue the following, based on the data presented:
Get top-five picks. The top picks in the first round are very, very valuable. Teams moving up into this range cheaply are getting real value. Generally it costs an arm and a leg to acquire a marquee pick, but there are exceptions – for instance, Tampa Bay’s trade of the fourth overall pick in 2002 to Philadelphia for Ruslan Fedotenko.
After the top-five, move down in the first round. Teams trade up all the time to snag players that they like in the first round, but as a rule it is a very bad idea. Teams trading down don’t really lose much value, especially if they aren’t trading down very far.
Top-100 picks outside the first round are not especially good, and are virtually interchangeable. These picks are all about quantity over quality; good players are there to be found but the odds are stacked against any individual selection. Trading a mid-second round pick for two third round selections would seem to be a wise move the majority of the time.
Picks outside the top-100 are all but interchangeable. These picks are all about quantity. Every selection is a long-shot, teams play hunches and snag the favourites of regional scouts, and basically there is very little to separate the players involved except the biases of individual teams and who their scouts have seen.
Contending teams can trade mid- to late-round picks with minimal reluctance. Draft picks after the first round, and especially those outside the top-100, can be spent with almost no hesitation on actual players. For a contending team, acquiring depth or quality via draft picks is an extremely sound strategy.
Take Adam Larsson FIST and get out of the basement!
Zach Parise
Gagner out for Season with severed Tendon
I told you all that Jones should have been traded. Now his skate cuts down the Oilers' most productive C. It's Cogliano and Horc the rest of the way as our 1 and 2 Centres.
Johnathan trading down isn't allways the best stratedgy as Zach Parise has shown us year after year. Draft stratedgy has more to do with the GMs confidence in his staff than anything. It depends if your scouts target the right players or just through darts at the wall. I trust Stu with LAs pick.
Throw.... not through
Good breakdown here. I'm sure some GM's have the same mindset when building their teams and their own little rules like this.
Thank god they took Ebs and didn't pull a Pouliot.
So, who are the feathers in MacGregors cap besides Jordan Eberle and Jeff Petry, any of MacGregors kids going to help us out next season?
@ James Scott:
I think betting that your scouting staff is way smarter than the average is almost always a bad move. I say that despite Stu MacGregor's excellent record to date; over the long haul I have difficulty believing that the Oilers' scouting organization is either much better or much worse than average.
My suggested conclusions are based on the assumption that any one team can't believe their scouts are brighter than everyone else's.
Jon, I hope the Oilers adopt that philosophy. Rightly or wrongly, I feel that the Oilers management have, for the longest time, taken the attitude that they are smarter than everyone else.
That's a HUGE assumption, akin to saying "I assume all teams are equal in player development.
There are teams that consistently draft better than others, and I expect that you can find widely varying degrees of $ spent on scouting. 12 very good scouts are going to do better than 6 average ones. It's also reasonable to assume that some scouts are better than others, just as some coaches are better than others. If the playing field on scouting was level then sure overtime it would all average out. Frankly I scouting is an areas where a team can gain a competitive advantage, particularly if it is combined with better player development (eg. not rushing prospects, vets to play with in the AHL, etc.).
This is especially true for picks outside the top 10 where a miss vs. a hit at pick #23 can make a big difference.
@Jonathan Willis
I cannot agree strongly enough. I was going out of my mind when they traded up for Riley Nash, and it had nothing to do with Riley Nash in particular. All Lowe did was trade away two birds in the bush for one bird in a different bush.
I once saw a special on Glen Sather talking about what a genius he was, and they asked him what makes a great GM. He said that if you can look at two full team playing hockey and figure out which two or three guys can really play, then you can run a hockey team. What a load of self-serving horse-feathers, this guy drafted Jason Bonsignore. Sather got lucky in the 80s and jinxed since, he didn't know anything more or less than anybody else.
The Parise argument doesn't completely hold because Kesler, Richards, Perry and Eriksson were all still available (so were Bergeron and Weber, but they would've been off the board.) Trading one pick for two is still a sound guideline.
Interesting note Willis that the top 3 picks are usually bona fide, picks 4-6 are less bona fide, and so on ... especially considering that there are 4 names often floated around as the top of the 2011 draft class.
So my question to you, fair nation, is which of the 4 top draft prospects do you feel is the WORST selection of the bunch.
I'm not sold on anyone in particular, but I for one am least sold on Sean Couturier.
What do you think?
@Westcoastoil
I agree with Westcoastoil. If you look back at teams success in drafting in the first round, you will notice that some teams perform much better than others (e.g., San Jose, Boston, Philly, St. Louis).
Although it can be a combination of scouting and player development - not just scouting. However, if our measuring stick of success is number of games played it really comes down to the whole package that gets the asset into the NHL as a regular.
All of this aligns well with the natural curve (bell curve) that one would expect hockey expertise (skill, motivation, intelligence, etc.) would be distributed by. So the empirical work lines up well with the theory.
However, the other thing that the theory will tell you is that the quality of the top picks will be more chaotic from year to year than the quality of the later picks (basically small 'sample' size effects). So, it would be unwise to assume 'regular' drop off points at 3 and 6 because there will be significant variability in this from year to year.
If Hedman and Tavares were born a few months later, the 2010 draft would have shifted by a couple of picks to the better and the 2009 would have shifted a couple of picks to the worse. This has a huge impact at the top of the draft, but almost no impact at pick 100.
@John Chambers
I think, I don't know but I think that Landeskog has the lowest ceiling, but is the surest thing. I really couldn't call myself "sold" on anyone else.
@ Westcoastoil:
Barry Fraser - savant or moron?
Based on his early work with the Oilers, you could argue the former; based on his later work, the latter.
There's also little evidence to suggest that NHL teams can solve drafting issues with money. For example, the Buffalo Sabres have gone the video route, largely in an effort to control cost. It doesn't seem to have hurt them.
Leaving that aside, the principal problem of aligning reports from multiple scouts remains. As head scout, your Swedish scout raves about Larsson, your QMJHL scout raves about Couturier, your your OHL scout raves about Landeskog, and your WHL scout raves about Nugent-Hopkins. How do you weigh those opinions? I suspect that particular problem is a big part of the reason tournament play gets such weighting - it's the one time all the scouts see players from outside their regional bases.
bookie:
Great call. Some years, Sidney Crosby's the first overall pick. Other years, it's Patrik Stefan.
@shau_co:
I don't think you can make that argument consistently. Let's look at your teams:
San Jose, '00 - '05:
Jillson, 14th; Goc, 20th; Morris, 27th; Michalek, 6th; Bernier, 16th; Kaspar, 22nd; Setoguchi, 8th.
Boston, '00 - '05:
Jonsson, 7th; Samuelsson, 27th; Morrisonn, 19th; Toivonen, 29th; Stuart, 21st; Lashoff, 22nd.
Philadelphia, '00 - '05:
Williams, 28th; Woywitka, 27th; Pitkanen, 4th; Carter, 11th; Richards, 24th; Downie, 29th.
St. Louis, '00 - '05:
Taffe, 30th; Belle, 30th; Schwarz, 17th; Oshie, 24th.
Aside from Philadelphia, it's difficult to argue any of those teams have consistently out-shot the average.
Barry Fraser - savant who jacked a bunch out of the park then mailed it in b/c Cabo beats Moose Jaw in January.
Fair enough that $ doesn't solve all your problems, but just because they are doing it via video doesn't mean that are spending less time/energy on it by better qualified people.
If your theory is based on no one team's scouts being brighter than another, then why wouldn't a bunch of teams just share a scouting staff or use Central Scouting and then go down the list. If all the scouts are as smart as the next guy their input doesn't help you align the reports anyway.
If I have a Swedish scout pushing Larsson, a Q scout pushing Couturier, etc. then I can either: a) go by organizational need, b) go by a scout's track record or c) rotate the scouts by flying the Swede over to watch the Q for a stretch and vice versa so their final reports are fully informed (or use video a la Buf.). Maybe moreso, in the case of a top 6 pick where it clearly matters the most to get it right because that's where you find your impact players.
I get what you're saying as to where the true value is. However, i think an assumption that all scouting staffs are more or less equal sets up a flawed argument right off the hop.
@ Westcoastoil:
Fair enough; I'll re-phrase:
Mostteams can't be confident that their scouting group is significantly better than the average NHL scouting group. In some special cases, where the track record warrants it, individual teams might have basis to believe they can beat the NHL average.
I'd still argue that betting on an average staff is the smart move for the majority of teams.
I think that drafting (and development) has to be a top priority in an organization and question how successful a team can consistently be without it. I think it's one of the areas where you might have the most opportunity to gain a competitive advantage. Because of the cap, having underpaid impact players is a huge key to success. You can only get that through outstanding drafting or (sadly in Edm's case) years of bottom feeding. Putting an extra 2-3 million into scouting gets you way more bang for your buck than trying to plug a hole with a UFA and ending up with Ryder's $4M for 3 years(or whoever).
Wow great read. Very interesting stuff. I hope the oilers management does this type of research.
@ Westcoastoil:
I believe in both drafting and development, but I think people forget how important support UFA's are. Detroit is the team I watch closest for management trends, and one of the things they've consistently done is grabbed value UFA's for depth roles - guys like Mikael Samuelsson and Patrick Eaves.
It's one of the reasons they're so consistently good.
I agree, Sean C worries me. He didn't look great at the prospects game and didn't make the WJC team. More to it than that for sure but my gut says the Oilers should pick someone else.
Mind you I thought Dustin Buff was going to be the next John Druce. Missed that one by a bit.
I guess if you didn't notice he was there he couldn't have been that great, but Sean C was on the WTC team.
@ Jonathan Willis
Trading down is a mistake unless you believe your guy will be available do to some overwhelming misconception about the player you want and you recieve a sure thing nhl player or prospect in return . Certain players are allways around later than they should be baecause of a generally accepted opinion Eberle his was size and skating, Parise size and Paajarvi finish Fowler toughness and shot. Thank goddness they didn't trade down and pass on those guys for a extra 2nd..... Opps, is 2 out of 3 bad. All kidding aside if the oilers had gotten a real player in that deal not a handful of magic beans it wouldn't have hurt so bad. Lowe and Prendergast had a bad 1st rnd record, as did Fraser after the first few years(The top picks are all well known prospects now days 1sts should not be off the board selections). Their 1sts reeked of arrogance like they had the whole world beat and everyone was missing something especialy Fraser(Doan and Miknov for example). The Lower the pick, the less chance of success period if you only look at one scouting staff(judging Team A vs Team B it is impossible to get data that means anything because a good staff will in all probability do better the higher they pick), Edm is going to have a better draft across the board when they finish 30th vs 15th, rnd 1,2,3,4,5 these are breaks that have nothing to do with talent(a title for the player selected). Talent in any year goes from high to low, the variations in success come from scouting bias or quality, post draft development(handling) / injury / luck(everyone can get lucky, on the Oilchange pilot it showed how lucky Fraser was when it showed the oilers 79 draft list), the first two can be controlled the last two can't be foreseen. IMO the best and worst staffs are probaly both getting guys that are way higher on thier list in later rounds. Being able to assess which players can overcome what ever warts in his game has caused their slide, (Hamilton it was injury, pitlick played for a defensive team hurt his offense, Martindale effort/maturity, Davidson late bloomer, Blaine late surge in the season, was it a anomoly) is the differance? I don't know if these guys are available 5,10 or 15 picks later, another team with a talented staff might have grabed them. Trading picks for picks doesn't make sense in most cases, it only works for the best scouting staffs as you have to have a specific target and you can't be wrong or you are most likely going to get burned and the optics are bad. Lastly IMO not taking the BPA is the downfall of most teams.
I agree Detroit is a good team to watch for mgmt, trends. By consistently drafting their impact players: Lidstrom, Datsyuk, Zetterberg, Franzen, Kronwall, Howard, Ericsson it then gives them increased flexibility when it comes time to plug a UFA in because the depth guys are more interchangeable.
As one of the few teams who've stayed on top without a top 5 pick, I'd argue they've made their hay by having a scouting staff better than most teams.
Good debate JW
Great article.
Are there any similar studies out there that broke it down further and looked at the type of players picked in terms of position, league, etc?
Someone at Lowetide posted a list of defenseman drafted in the top 5 over the last few years. The number of busts was astounding. It really makes me question the wisdom of drafting Larsson.
Excellent!
I'm glad this is getting more and more play.
I've basically come to the conclusion that I'd likely be trading every pick past 15-20. And for sure every pick past the 2nd round. (assuming the return is decent)
Of the four kids, Couturier,Larsson,Landeskog and Hopkins, is it a stretch to say only two of these kids make the jump to the NHL this coming year?
Has anyone compared Oiler draft track record to central scouting? I often thought the Oilers would have been better off firing the the scouting staff, using Central Scouting and then focusing on player development.
I haven't seen any of them play, but based on what I've RNH makes me a bit nervous - is he another Sam Gagner? Great numbers, but how much is PP and will he have the size/speed to be a first line player. I don't know, but I'd be tempted to chicken out and take the best "sure thing" over most potential.
Based on Tambi's intermission interview a couple games, as tough as it is to swallow i don't think they'd mind one bit if they drafted a player that wasn't ready and rode it out for one more lottery pick.
Say it with me,
"Don't Take, R-N-H!!"
@Westcoastoil
There must be a certain amount of pressure to step into the lineup right away if he's the first overall pick. Not sure how many kids have been selected first and then went back to junior, if any.
I remember the days when 6 or 8 kids drafted made that step their first year, now we're down to less than half that.
@Come join the Dark Parade...
I hear what you're saying, but which of the 4 ready to make the jump. Just the 2 Swedes? If Couturier wasn't able to stand out in the WJC, how's he going to look lining up against an NHL vet. In Stu we trust:)
Alright. Thanks for your thoughts. You too, Zissou, and Butters.
Too small a sample size, but I figured the Nation would be torn as to who should be the odd man out ... except for Larsson. I doubt many people have Larsson as their absolute least favorite lottery pick.
Great write-up Jonathan.
For more, there's a great write-up by Marc Foster: http://www.puckprospectus.com/article.php?type=2&articleid=849
and Iain Fyffe: http://www.puckprospectus.com/article.php?type=2&articleid=807
Unfortunately you need a subscription to read them. :(
However, I personally guarantee that the data and graphs within are worth the price of admission. They're both very careful and detailed studies of draft choices, similar to Awad's.
Re:Thinking that you (or your scouts) are better than others around you.
Check out this article by David Brooks in the NYT (it relates to biased self assessment)
http://www.nytimes.com/2011/03/11/opinion/11brooks.html?_r=1&hp
I've heard (don't remember where or from who) that most teams don't do any better, and some do worse, than Central Scouting.
So why pay for your own scouting department when Central Scouting is already doing as good a job for you?
Perhaps you've heard this same rumour. If so, where is it from and does it have any merit?
Don't worry lamers you will taket he best hard working third liner in the draft. NICE move on trevor Kidd too. Who would some guy named Martin Brodeur, when you have your usual West boys.
@James Scott:
I'm talking about essentially doing the opposite of the Riley Nash trade.
In any case, I know the Pouliot/Parise pick gets a ton of play here, but let's remember that if the Oilers had been able to pick one selection higher they would have snagged Robert Nilsson - which, IIRC, was the reason they traded down, as their player was gone when the 17th pick rolled around.
Pouliot/Jacques didn't work out, for a variety of reasons, but that one trade doesn't mean trading down is a bad idea.
And for the record, Fowler was an exception, a consensus top-5 pick who slid. You don't trade down when that kind of player falls to you.
@ Westcoastoil:
Believe it or not, the Red Wings really aren't that good at the draft. Sure, they've had some nice picks late (Zetterberg/Datsyuk/Franzen) but overall they don't do a great job - just average.
The difference is that they're just so much better in almost every other department.
RE: Central Scouting
Central Scouting's year-end list is generally pretty bad.
What they do well is identify players early on for NHL teams to home in on, but their final list is usually significantly worse than the consensus wisdom of the 30 NHL teams.
That said, in Barry Fraser's later years he got schooled by CSB.
@Jonathan Willis
Willis, I have to say that is one of the most interesting articles I have ever read on ON. I know this must have taken a lot of time and research so thanks a lot. You earned your writing credit on this one.