Benchmarking Bob McKenzie, Draft Projection Variability

With the final month of the regular season underway, realities are sinking in, playoff bubbles are bursting, and playoffs droughts are ending (I hope...). For teams like Florida, Tampa, Calgary, and Colorado, the 2013 draft couldn't come soon enough.

Speaking of the draft, I try to make it a regular routine to do some reading and podcast-listening on prospects over the weekend. In browsing some of the new projected rankings for the upcoming draft in New Jersey, I came across TSN's archive of Bob McKenzie's draft projections for the past 9 drafts.  His methodology involves an annual survey of NHL scouts, asking them which players they think will be taken in the first round. The final rankings are then determined by a consensus of opinion, and does not consider the order of teams, or team needs. 

"86%, bitches..."

The site claims that in the last nine years, "McKenzie is 231/270 in terms of projecting first rounders (~86%)", and after looking at the data myself, their claim is true. Now, 86% is pretty accurate and I'm not about to sling mud at McKenzie's record.  However, if we define accuracy as the degree to which actual picks deviate from the projections, how accurate is McKenzie, really?

In other words, TSN and I are usually two different definitions of accuracy. In McKenzie's case, of the 30 names he projects as first rounders, usually 86% of those names end up being picked in the first round. The accuracy I'm trying to get at is to see by how much the projections vary in their final position on the board. For those of you who are familiar with rudimentary statistics, what I'm basically looking at is the standard deviation of the projections at each position.

*(Ignore the next paragraph is you're already familiar with standard deviation)*

I don't intend for this to be a stats lesson but briefly, what standard deviation shows is how much variation exists from the mean, or average. For the mean, I took the average of the eventual pick position for each projection.  Finally, to determine the standard deviation, I squared the cumulative differences between the actual draft positions and the mean, divided the squared cumulative differences by the sample (9 drafts), and found the square root of that.  All you need to know is that a high standard deviation indicates that the data points (actual draft position) are spread out over a large range from the mean (projection), and a low standard deviation indicates that the data points tend to be close to the mean.

First, let's take a look at how McKenzie has fared. As the chart below indicates, over time, more and more of his projections have ended up as first rounders. In 2012, he matched his highest accuracy rates (90%) from the 2008 and 2009 drafts.  On the other hand, the two years prior to that , 2010 and 2011, have been below average (84%), but not by much. 

The chart below shows the standard deviation of each projection, at every position from 1-30. Assuming a normal distribution (bell curve ring any bells..?), the way to read this is, for example, 68% of the eventual positions of the projected 10th overall pick usually land +/- 4.6 spots from the projected position.  

What's interesting about the chart is that up until the 20th overall pick, deviations from the projections remain in a pretty tight band, ranging from 0 (1st overall) to 7.6. (20th overall). After that, things gets a little wacky and eventual picks deviate big time from their projections. Assuming the Leafs do not capture a division title, or the Stanley Cup this year (playoffs, sure - but let's be realistic), Nonis and Co. could be looking at a first round pick ranging from 14th-24th overall. Luckily for Leaf fans, the deviation from 14th-24th also happens to be lower, on average, than from 25th-30th. However, I'm willing to bet that no matter which pick the Leafs end up with, the TSN projection at that position will likely not be the name Nonis calls out at the Prudential Center this summer.

If you fancy yourself a draft prognosticator, or want to see how you fair against the insider of all insiders, feel free to use this as a benchmark of sorts for your own projections.

As always, feel free to leave any comments or questions below or hit me up on Twitter (@Mapleleafmuse)

Kadri & Co.: NHL Impact Analysis of Top-10 CHL Forwards

A little over a week ago, I wrote a piece on the NHL impact of top-10 junior defensemen in scoring from their respective leagues. The idea was sparked by a conversation I had with Curt from Blue Chip Prospects about the Leafs' track-record and effectiveness at the draft. With Morgan Rielly currently sitting in the top-5 in scoring among WHL defensemen, he's got some big shoes to fill. In this piece, I'll be taking a look at the top-10 junior forwards in scoring.

Nazem Kadri recorded 93 points during the 09-10 regular season with the 

London Knights - good enough for 5th place in OHL scoring.

Just like the prior article, I decided to use the All-Star, Impact, Replacement, and Bust methodology, and did so for the top-10 scorers between 1999 and 2008. Having defined categories for players kept the analysis nice and tidy, but I believe players like Bobby Ryan and Jakob Voracek should be included  in the All-Star conversation, especially Voracek who is tied in scoring with All-Stars like Tavares, Getzlaf, and Kane, and would certainly be an All-Star during the current campaign.

The OHL stands out with the highest number (18) and percentage (22.5%) of Impact+ players, and the lowest bust rate - not too dissimilar from defensemen it has produced over the years.  From 1999 to 2008, the OHL has produced 80 unique forwards in the top-10. In doing so, no one player was over-represented in the sample. Some notable graduates include Patrick Kane, Corey Perry, John Tavares, Steven Stamkos, Taylor Hall, and Bobby Ryan.

As an aside, it's impressive how many All-Stars the London Knights have produced (Kane, Perry and Tavares). This thought would have been nearly unfathomable at the beginning of the season, but if the All-Star game was miraculously rescheduled tomorrow, I believe Kadri is on the first ballot. The Leafs have something special on their hands and it appears that their patience, along with Kadri's resilience and perseverance, is paying off.

The QMJHL, overall, is in the middle of the pack in terms of number (15) and percentage (20.6%) of Impact+ players produced. From 1999 to 2008, there were 73 unique forwards who occupied the top-10 positions in scoring. With a bust rate of 72.6%, it's better than the WHL but not by much. Some notable graduates include Sidney Crosby, Claude Giroux, Mike Richards, Jakob Voracek, Radim Vtbata and P.A. Parenteau.

The WHL stands out, but not for the best of reasons. With a combined 8 Impact+ players, or 10.2% of all 79 unique forwards from 1999 to 2008, the top-10 forwards in scoring have only produced one All-Star and bust at a rate of 74.7%. Some notable graduates include Joffrey Lupul, Devin Setoguchi, Brayden Schenn, Jarret Stoll, Evander Kane, and Tyler Ennis.

After taking a glance at all three junior leagues, it is evident that the OHL produces the most, on an absolute and relative basis, Impact+ forwards.  Interestingly enough, the year Kadri came 5th in OHL scoring fell just outside of the sample timeline, and for good reason. It usually takes draftees a couple of years before they are able to make a sizable impact at the NHL level - and boy is he making an impact now!

His trials and tribulations were well-documented and it's refreshing to see a top prospect starting to pan out for the Leafs. With his ability to make others around him better, and hopefully with some more ice-time, the sky could be the limit for the kid they now call "The Dream".

Note: Other unique forwards who finished in the top-10 of scoring during the 2009-2010 OHL regular season include Tyler Seguin and Jeff Skinner. (Greg McKegg finished 11th that year) 

Rielly Among Great Company: Impact Analysis of Top-10 Junior Defensemen

Recently, Curt from Blue Chip Prospects and I exchanged articles that looked into the Leafs' drafting record.  I argued that the Leafs' positional drafting strategy was undifferentiated and that draftee impact was weak. Curt, on the other hand, argued that the Leafs' drafting was quite strong, given their low bust rate and late median draft slot in the first round. As an avid reader of sports blogs, especially of those that cover the Maple Leafs, I respect and appreciate Curt's attention to detail and thoroughness. Also, as a rookie blogger, I welcome constructive criticism and he was kind enough to offer some feedback on my methodology.  Be sure to check out his insightful response and some cool data over at Blue Chip Prospects.

Prior to his response, Curt wrote another interesting piece on positional draft analysis by round and tweeted about Morgan Rielly's impressive performance, including a game-winner against Prince Albert on February 16th. In a nutshell, the article showed that, at least in the first three rounds of the draft, defensemen typically have a lower bust rate and a higher impact rate than forwards.

The tweet on Morgan Rielly and the article got me asking, if I had to pick a defenseman in the upcoming draft, where would I look?

Given that I'm not a pro-scout, I decided to apply the same criteria Curt used (All-Star, Impact, Replacement, Bust) and began my search with the top-10 defensemen in scoring across the OHL, WHL, and QMJHL, from 1999-2008. Once again, this analysis would have benefited from some added flexibility. Players like Seabrook (Stanley Cup and Olympic gold) and Pietrangelo (2011 2nd-Team All-Star) were relegated to the "Impact" category as they were not "All-Stars", per se, but an argument could be made for both. In any event, the following charts represent the the number and percentage of players who fall into the aforementioned buckets.

The OHL stands out as producing the highest number (19) and percentage (28.8%) of Impact+ players, and the lowest bust rate (54.5%). Also, from 1999 to 2008, there have been 66 unique defensemen in the top-10 of scoring. This means that no one player was double-counted if he was a top-10 scorer in multiple years. Some notable graduates include Drew Doughty, Alex Pietrangelo, Mark Giordano, Michael Del Zotto, and P.K. Subban.

The WHL stands out with the highest number (5) and percentage (7.8%) of All-Stars. As a whole, there were 10 Impact+ players, or 15.6% of all 64 unique defensemen from 1999 to 2008.  Furthermore, the bust rate for top-10 WHL defensemen was 68.8%. Some notable graduates include Alexander Edler, Dion Phaneuf, Mike Green, Brent Seabrook, Josh Gorges, and Cody Franson.

The QMJHL also boasts 10 Impact+ players, including 2 All-Stars (2.8%). However, given that there were 71 unique defensemen in the top-10 of scoring, the "Q" lags the other two leagues in absolute and percentage terms. Also, the 81.7% bust rate is significantly higher than the other two. Some notable graduates include Kris Letang, Keith Yandle, Zybnek Michalek, Francois Beauchemin, and Johnny Oduya.

So, if we revisit my initial question on where to draft, it appears that the OHL may be the place to look. Obviously, other factors like international experience, size, translatable skills, and skating play a huge role in the overall player evaluation.

As of the writing of this article, Morgan Rielly is sitting in 5th place in scoring among defensemen, and Leaf fans should be ecstatic to have such a highly-touted prospect in the pipeline. I honestly can't remember the last time I've been this excited about a prospect. Now, the above data should not be used as a means to handicap the probability or likelihood of Rielly succeeding at the NHL level. However, what it does tell us is that Rielly is among good company.

Stay tuned for the next piece where we will cover the top-10 forwards from each league... 

Burke Was Right, and Wrong

A little over a month ago, Brian Burke was fired as GM of the Toronto Maple Leafs and it was arguably one of the most well-covered and talked about events surrounding the team in recent memory - a sad fact indeed. Media and experts where quick to assess the wreckage and perform their respective autopsies - how could they not? Brian Burke was a larger-than-life general manager and only his blockbuster trades rivaled his bombastic proclamations. Content with simply observing what others had to say about Burke's tenure, I was compelled to reflect on some of his most memorable soundbites, namely "I build teams from the net out..." and "Pittsburgh model, my ass...".

Brian Burke - truculent till the end

Photo courtesy of Global Toronto

Prior to the Leafs most recent 4-game win streak, the post-Burke life-cycle stage of the Leafs was debated and there are two main opinions. First, some argue that Nonis should trade for Luongo and push for the playoffs, and second, there is a contingent of fans who are of the "Tank Nation" persuasion, citing Seth Jones or Nathan MacKinnon as reason enough to "mail-in" the season.  I'll admit, I was conflicted as to whether I wanted the Leafs to tank. The possibility of adding an organizational gem like Jones or MacKinnon should have every fan salivating. But, as a former athlete, I couldn't fathom giving less than 100% effort as it defies the very nature of your being.

So, if tanking for a high draft pick was never an option for Burke, as demonstrated by his contempt for the Pittsburgh "model", then drafting well and identifying talent become imperatives for the Leafs. Combine this with the "net out" development philosophy, I wondered whether the Leafs' draft selections and performance reflected Burke's public statements.

The following graphic represents the positional breakdown of draftees for Northeast Division teams and the Pittsburgh Penguins, since 2005.

Before stating my observations, I think it's important to list some limitations in extrapolating sweeping conclusions based on how a team has drafted and how those draftees have performed. Drafting represents only a portion of a team's influx of talent and is a component of a larger strategy (trades, signings, etc.) for building a team. A team's developmental process also plays a big part in the future success of draftees. Unfortunately, as an outsider, a team's development strategy is hard to observe but if I were in a management position, development would certainly factor into the evaluation.  

However, the whole purpose of drafting is to identify and select potentially impactful players. So, how does Toronto compare to the rest of the division and Pittsburgh?

• Toronto places 5th, 4th, and 5th in goalies, defensemen, and centers drafted, respectively
• Buffalo and Pittsburgh appear to have similar positional drafting strategies
• Boston, Buffalo, and Ottawa all appear to have differentiated positional drafting strategies

Burke's influence on the data covers half of the sample of drafts, so he has had a chance to leave his mark on the team. As far as building his teams "from the net out", the Leafs have not drafted like it: 5th in goalies drafted (7%), 4th in defensemen drafted (37%), and 5th in centers drafted (22%). Juxtapose this with today's high-demand for goalies, defensemen, and centers, it comes as no surprise that the Leafs have had little success since the 2004 lockout. Building from the net out, my ass.

Another interesting observation, I believe, is the similarity between Buffalo and Pittsburgh. Granted, Buffalo has not had the "luxury" - and I say this lightly, as the rest of the blog will show - of drafting generational superstars like Crosby and Malkin. But, in terms of positional drafting, both teams are quite similar. Therefore, there doesn't appear to be anything particularly unique about how Pittsburgh has drafted. "Pittsburgh model, my ass", indeed.

In other words, from a drafting perspective, Burke was right and wrong. Wrong about drafting from the net-out. Right about the "Pittsburgh model". 

On the other hand, knowing whether Burke was right, or wrong, doesn't inform my opinion on whether I want the Leafs to tank or not. Many analysts describe the upcoming 2013 draft as containing a deep pool of talent. There are many ways to measure a successful draft, such as number of games played or goals/points scored by the draftee. Firstly, I decided to measure success by looking into the average draftees as a percentage of a team's top-10 and top-5 scorers.

Three points are worth highlighting:

• In both cases, Toronto is mediocre, at best, in terms of draftee impact on team scoring
• Compared to our division, Pittsburgh is average in terms of draftee impact on team scoring
• Buffalo stands out and leads the division in terms of draftee impact on team scoring

Pittsburgh's draftee impact, as odd as it seems, does not significantly outperform our division and reinforces the previous notion that there is nothing particularly exceptional about the "Pittsburgh model".  However, this is also a damning chart for Toronto and perhaps indicates some sort of weakness in our drafting.

What about the consistency of a draftee's impact on team scoring? Specifically, what average number, or percentage, of top-5 draftee scorers can we expect to be top-5 scorers again next season? The following charts give us an idea of a team's "turnover" of draftees as top-5 scorers.

Three points to outline:

• Toronto's turnover, on an absolute (1.17) and relative (50%) basis, is pathetic - one can expect only 1.17 draftee(s) or 50% of draftees from the prior season's top-5 scorers to replicate their performance
• Pittsburgh's turnover is notably below the Northeast Division average and is virtually the same as Montreal's
• Ottawa and Buffalo's turnover rates stand out as having the highest average number, and percentage, of draftees who can replicate their performance the following season

Once again, Pittsburgh's drafting performance seems average, at best, when compared to our division.  More importantly, the overall performance and consistency of Toronto's draftees is abysmal compared to the other 5 teams.

The two questions I asked at the beginning of this entry were seemingly unrelated at first. In fact, you may still be wondering what Burke's public statements have to do with whether the Leafs should tank, or not. The key takeaway, I believe, is that from a drafting perspective, Burke was correct in scoffing at the "Pittsburgh model", as the Penguins seem to be average at identifying consistently impactful talent through the draft. Conversely, the Leafs do not draft with the "net out" philosophy and, development aside, draft the least consistently impactful players.

So, Burke was right, and wrong, in his proclamations.  However, drafting is clearly not one of our strengths so even if we did tank for Jones or MacKinnon, would it matter?

I leave that for you to decide.

Kessel's Slow Start Not Unusual

January is now in the books and the Toronto Maple Leafs find themselves in 7th place (4W 3L 8Pts) in the Eastern Conference. There are many intriguing story lines: Reimer's continued emergence as a number one goalie, Kadri leading the club in points, and Phil Kessel's scoring drought, just to name a few. Understandably, Kessel has received the majority of the attention, including TSN's Jonas Siegel recent piece on Kessel's uncharacteristic slow "start". However, a concerted glance at history indicates that a slow January may be a seasonal part of Kessel's game.

With 4 points (0G 4A) in 7 games, Kessel's goal-scoring and point-production has certainly been underwhelming. Following 40 hard minutes of bullying from the Rangers last Saturday, a game which I attended, Leafs Nation and the Toronto media have really zeroed-in on Kessel's lack of scoring.

Phil Kessel doing his best Jordan Tootoo impression - Leafs v. Sabres (1/21/13)

Photo courtesy of MapleLeafs.com 

Some may attribute the lack of production to coach Carlyle's system, Lupul's absence, or just gripping the proverbial "stick" too tightly. Unfortunately, Kessel is known as a notoriously streaky scorer and his consistency often comes into question. As I pondered these allegations of inconsistency, my first instinct was to see if there was some sort of pattern to the purported variability in his game.

This led me to look into Kessel's goal/point-production and shot attempts per month, throughout his time as a Leaf.

The first chart presents Kessel's average goals-per-month (GPM) and goals-per-game (GPG) by month for the past three seasons. (Due to the small sample size of games during the month of April, results from that particular month should garner less consideration.)

• January is typically the weakest GPM and GPG month - on average, January comes 6th, out of the 7-month season, in goal-scoring (4.00 GPM and 0.31 GPG)
• February is typically a strong GPM and GPG month - on average, February comes 2nd, out of the 7-month season, in goal-scoring (5.67 GPM and 0.58 GPG)

The second chart presents Kessel's average points-per-game (PPG) and shots-per-game (SPG) by month for the past three seasons. (Once again, due to April's small sample size, results from that particular month should garner less consideration.)

• On average, January is the least (7th) productive month (0.72 PPG) and the 2nd highest month in shot attempts (4.28 SPG)
• On average, February is the 2nd most productive month (1.25 PPG) and the 3rd lowest month in shot attempts (3.89 SPG)

Now, there are numerous potential variables at play but the reason for the inclusion of shot totals into the analysis is because although Kessel hasn't scored as much, he is still shooting and generating chances. After Thursday night's win against Washington, no one can deny that he isn't getting his share of scoring opportunities. Subsequently, if we re-order the months from highest PPG to lowest PPG, an interesting relationship emerges.

In fact, the data produces a negative correlation of -0.53 between average SPG and average PPG by month. Although a -0.53 correlation may not be overly significant, what is does tell us is that Kessel is aware of his lack of production and in an attempt to change the tide, he shoots more! By shooting more, Kessel gets more chances and usually follows up a month of higher SPG with a month of higher PPG. In other words, SPG may somewhat serve as a (inverse) leading indicator of PPG. With Phil having shot at a 4.71 SPG clip for the month of January, he is well above his typical average of 4.28. He is also well below his typical PPG average of 0.72, having scored at a 0.57 PPG pace during January.

In short, a low-scoring and high-shooting January is nothing new for Kessel.

This lockout-shortened season brings with it added uncertainty and prior fast starts out of the gate may provide limited insight into what we can expect from players this season. In fact, one could argue that many things haven't turned out as predicted. However, given the inverse relationship between SPG and PPG, the historical productivity cycles, and with 15 games set for the following month, February may yet prove to be another productive period for Phil Kessel.

Joe Colborne: Rising to the Challenge

Apart from the Marlies' stellar Calder Cup run, Leaf fans had little to be proud of following a dismal 2011-2012 campaign. Then came the lockout and without NHL hockey, much attention was paid to the performance of the Leafs' young prospects on the Marlies roster such as Gardiner, Kadri, Colborne, and Scrivens.

After all, these were the players many believed, or hoped, would challenge for roster spots under coach Randy Carlyle. Kadri (27GP 8G 18A), Gardiner (22GP 9G 8A), and Scrivens (2.22 GAA .917 SV%) have performed up to expectations. Colborne, on the other hand, struggled early on, recording a mere 6 points (16GP 1G 5A) in his first 16 games.

Joe Colborne working the powerplay - Marlies v. Bulldogs (1/2/13)
Photo courtesy of bchan53 (http://flic.kr/p/dJMvim)

As of late, the Calgary, Alberta native has rediscovered his scoring touch - scoring 13 points in the past 17 games, and has exploded for 9 points in 5 games since the start of the new year. Anyone who followed the Marlies last year is well aware of Colborne's offensive talent and with the high likelihood of Frattin, Kadri, and Gardiner joining the big club, the former first-rounder will be relied upon to play a bigger offensive role with the Marlies.

With the gaze of the hockey world fixed squarely on training camps and the shortened 48-game season, the AHL will once again take a backseat to the NHL. However, the Marlies have delivered an outstanding season thus far and have given Toronto a reason to cheer. Be sure to add Colborne to the "watch-list" since not only is he a player that could prove consequential to the future success of the Leafs, but is also on pace to have his most productive month since he received the AHL Player of the Month award last season (October, 9GP 8G 8A).