Hockey Confidential Page 10

  As an aside, from a strictly neutral observer trying to grasp #fancystats, having two metrics so close in nature is more confusing than helpful and, therefore, hinders the “marketing” of new-age numbers. But that’s just me.

  Which brings us, finally, to PDO.

  “A brilliant number,” Desjardins said of PDO.

  Hockey’s advanced number crunchers use all three metrics in concert with each other, as well as other, more complicated calculations, but you get the sense that if they could have only one of them, it would be PDO.

  That’s because it’s perceived to be the most “predictive” of the Big Three.

  “Saying that guys or teams with a really high PDO will regress is sort of the ultimate shooting fish in a barrel of hockey analytics,” Dellow said.

  Here’s how PDO works, or at least what intrigued Brian King enough to come up with the initial concept.

  King figured if you take a team’s shooting percentage with a given player on the ice (his “on-ice shooting percentage,” not to be confused with his individual shooting percentage, which is calculated only on his own shots and goals) and add it to his team’s five-on-five save percentage when he is on the ice, there should be a total number that represents the baseline for any player who is neither extremely lucky or unlucky. As he said, “for shits and giggles,” he chose 100 per cent, which turned out to be pretty much perfect. When King calculated the PDOs for individual Oiler players at the time, he discovered those with PDOs of greater than 101 tended to get contract extensions, while those below 99 tended to get shipped out of town.

  The more research that was done on PDO, the more it became clear that players or teams (PDO works well for both) with exceptionally high or low numbers were often experiencing really good luck or wallowing in the misery of bad luck. It was determined that the outliers would, over time, regress to meet somewhere in the middle.

  Five-on-five team shooting percentage, for example, tends to run between 6 and 10 per cent, but the average is around 8 per cent. Five-on-five team save percentage generally ranges from .900 to .940, but the average is around .920. So, add the on-ice shooting percentage of .08 to the save percentage of .92, and there you have it: your midpoint of one—or 100, depending on how you want to present it. Players and teams can post PDOs much higher or lower than 100, but over time, that’s roughly where they’ll end up.

  King couldn’t possibly have known what an incredibly predictive tool PDO would become, both for players and team. Non-believers in #fancystats often don’t like PDO because its core concept can really rain on a hockey fan’s parade.

  Take Edmonton’s Jordan Eberle, for example.

  In 2011–12, his second NHL season, Eberle had a breakout year, scoring 34 goals and 76 points in 78 games. The widespread sense at the time was that it was a hint of things to come, that Eberle might be on the cusp of becoming an elite-level point producer who was only going to get better, to put up bigger numbers. The sky was the limit. It was exciting, so hopeful and optimistic.

  Dellow—like Ferrari, a passionate Oiler fan—looked at Eberle’s underlying numbers (as measured by PDO) from that season and didn’t like what he saw. The Oilers shot 12.7 per cent in five-on-five situations when Eberle was on the ice and had a .909 save percentage, making Eberle’s PDO 103.7. If Oiler fan Dellow didn’t believe so firmly in the power of PDO, perhaps he could have joined the cheery chorus predicting Eberle was on the launching pad to become a perennial point-a-game player (or better).

  But he didn’t. Or couldn’t. The numbers wouldn’t allow him, and the numbers, Dellow believed, don’t lie. He wasn’t shy about saying so, either. He didn’t say Eberle wasn’t—or couldn’t become—a good hockey player; he just said Eberle’s numbers that year were much more likely to be an aberration than the norm and that Eberle was likely to score less, not more, in coming seasons. A PDO of 103.7, Dellow told anyone who would listen, was unsustainable.

  And it was.

  Eberle scored 16 goals and 37 points the following year in the lockout-shortened 48-game season; he had 28 goals and 65 points in 80 games in 2013–14. His PDOs for those two seasons regressed from 103.7 to 98.9 and 100.5.

  The same principles of PDO apply to teams.

  That was, in large part, the rationale for the infamous #fancystats Maple Leaf Prophecy of 2013–14. Toronto had a PDO of 103.0 in 2012–13, which was viewed as unsustainable. Sure enough, in 2013–14, it dropped to 101.2.

  That mean of 100.0, for most teams, is like a magnet. The teams with PDOs significantly above or below 100.0 tend to regress towards it. Again, though, as with Corsi and Fenwick, it’s not inviolable. You can find exceptions to the rule. Pittsburgh posted back-to-back PDO seasons of 102.2 in 2007–08 and 2008–09 (regressing to 99.5 in 2009–10). Anaheim went from 101.7 in 2012–13 to 102.3 in 2013–14. Teams with chronically poor goaltending (bad save-percentage teams) can get mired in the range between 97.5 and 99.5 and not drift towards 100. Teams with really extraordinary goaltending can stay above 100 longer than they should.

  Dellow conceded PDO is not infallible, but in a game where luck plays such a huge factor, PDO is, relatively speaking, as good as it gets for identifying teams and players whose results are likely to change, for better or worse.

  Dellow reviewed team PDOs for six NHL seasons, from 2006–07 to 2012–13. During that time, only 32 of 180 teams posted a PDO of 101.0 or better. Of those 32 teams, only three didn’t regress the following season. Two of them—the Vancouver Canucks (from 2009–10 to 2010–11) and the Penguins (from 2007–08 to 2008–09)—stayed the same. The only team with a PDO of 101.0 or greater that improved the next season were the Ducks, from 2012–13 (in the 48-game lockout-shortened season) to 2013–14. Twenty-nine of the 32 teams with a PDO of 101.0 or more regressed.

  The trend is clear. One might call it overwhelmingly clear.

  That’s why the Ducks (102.3 in 2013–14) and the Avalanche (101.7) were the #fancystats gang’s picks to take a step back in 2014–15. One of them could prove to be the exception to the rule, but probability suggested otherwise.

  “Everyone [eventually] crashes towards 100,” Dellow said. “The best tend to stay a little higher than 100, but it’s by a small amount. So when I say Colorado is going to crash next year, I’m really saying, ‘Historically, teams with a PDO like Colorado’s do not repeat that PDO the following season. If Colorado doesn’t get a high PDO, they can’t win unless they improve their possession game.’ It’s very rare that the same team posts a high PDO two years in a row, so I’m comfortable saying Colorado probably won’t. In that sense, PDO is very easy to predict: just assume a team will be close to 100, even if it was significantly above or below 100 in the previous year.”

  If there’s a final exam on this at the end of the chapter, I’m going to ask to be exempt.

  Many more NHL teams than not are into analytics.

  That’s not a real news flash. The number grows each day. The real questions are, to what degree, for what particular purpose and to what end are analytics being used by these teams?

  That would qualify as difficult to quantify. NHL clubs are secretive at the best of times. When it comes to what new-age data they’re using, how they collect it, who’s collecting it, what specifically it’s being used for . . . well, suffice it to say they’re not too forthcoming.

  Again, though, that 2013–14 NHL season seemed to be, at least in terms of public perception, the year in which the profile of hockey analytics was greatly heightened.

  The New Jersey Devils, for example, have long been known as one of the most secretive organizations in the NHL. In January 2014, lo and behold, the Devils advertised on their own website a job posting for a director of hockey analytics.

  “We’ve had . . . I don’t know how many applicants,” Devils president and GM Lou Lamoriello told espn.com.

  If Lamoriello wouldn’t share how many people appl
ied, I don’t like anyone’s chances of getting the godfather of NHL GMs to explain what metrics the Devils would like to track and how they would use that information.

  On May 1, 2014, the Chicago Sun-Times published a feature on how Blackhawks’ GM Stan Bowman had fully embraced analytics.

  Also in the 2013–14 season, Oiler winger Hall, as Matt Fenwick noted, dropped some #fancystats knowledge in a pre-game interview on TSN: “Whether they’re use useful or not, I do think they are for sure, but the thing for a hockey player, if you’re an advanced-stats guy and you’re describing to a hockey player, you have to have, like, some kind of end point. Like, what does he have to do better to get this stat better? That’s the thing I’m lost on with Corsi and Fenwick and all this stuff—how do you improve a player by it, what do you tell him [to do]?”

  I would suggest the closer to ice level you get in the NHL, the further away you get in terms of an understanding and/or appreciation of analytics. The further up the food chain you go, the greater it is.

  That is, players are the least likely to know or care about them. Most of them just want to play. That’s challenging enough without bringing analytics into the mix. But for players who do understand new-age stats, they’re not easy to embrace because there often is, as Hall noted, a disconnect between knowing what a stat is and knowing what to do to improve the stat, assuming that improving it would make a positive difference on the player’s or team’s play on ice.

  NHL coaches are all over the map in terms of their views on analytics. Some do embrace them to some degree; others not at all. Mostly, though, I think you could safely say coaches are so focused on their own team, they tend to use two tools in particular: their eyes, while standing behind the bench; and expansive video, to draw conclusions on how the team is playing and how it would be better served to play. Don’t get the wrong idea: a lot of NHL coaching staffs are well aware of Corsi and Fenwick, and many get supplied with that information (or perhaps, for teams really into analytics, something unique and/or more advanced) period by period, but I’d wager the vast majority are more likely to trust what they see themselves or what the video coach gives them.

  Video is such a huge part of NHL coaching. The video programs have become so sophisticated that a single player’s entire game can be viewed with the push of a button. In the age-old battle between #fancystats non-believers (“I watch the games”) and believers (“The numbers don’t lie”), an NHL coach has the advantage of being responsible for only one team, and watching every second that team plays, with the added benefit of rewatching every or any aspect of it any time he likes.

  Dellow is a big believer that the onus is on hockey analytics people to answer Hall’s question—that it’s not enough to come up with the numbers, that there’s another step in terms of it being actionable. In other words, a player or coach needs to be told, “If you do this, this and this on the ice, it may better than what you’re currently doing.”

  Much of Dellow’s published work is in that vein, taking actual game occurrences, measuring and quantifying things, and suggesting tactics or different approaches to improve the results.

  Gabe Desjardins said the cost of one really good analytics expert working for a team would be saved with one good decision from that individual—that using analytics for the acquisition of one good player, or not overpaying on one contract extension for another, would be the wisest investment a team could ever make.

  All of these guys—Desjardins, Ferrari, Dellow and their growing legion—have, in one manner or another at some time, worked for or consulted with NHL clubs or have been asked to.

  In fact, for a relatively brief period of time, during or sometime around the 2010–11 NHL season, Desjardins, Ferrari, Dellow, Tom Awad and Sunny Mehta, a former pro poker player who was also a notable contributor to Ferrari’s website, worked as consultants for the Phoenix Coyotes. They were among a group of analysts who were known within the Coyotes’ organization as “the Quants,” named for Scott Patterson’s bestselling 2010 book that looked at quantitative analysis of Wall Street hedge funds and the men who operated them.

  This hockey statistics equivalent of the Dream Team, in a manner of speaking, was put together by prospective Coyotes’ owner Matthew Hulsizer, the financier who seemed to be on the verge, at that time, of gaining control of the bankrupt NHL franchise. The hockey ops people in Phoenix did utilize the Quants for some input on statistical analyses, but the Quants didn’t last long and neither did Hulsizer, who ultimately pulled out of the running to purchase the Coyotes in 2011.

  More and more teams now have full-time directors of analytics, applying their work not only to the team’s play on the ice but everything from contract negotiations to the drafting of players.

  Well-respected advanced stats blogger Eric Tulsky wrote a piece, an effective NHL status report, on October 21, 2013, chronicling just how many teams are hopping on the analytics bandwagon, but it’s well known in hockey circles that some teams—Boston, Los Angeles, San Jose and Chicago, for example—have been at it for years, and likely at relatively high levels of sophistication.

  Each year, more and more NHL personnel are turning up at the renowned MIT Sloan Sports Analytics Conference in Boston, which used to be the almost exclusive preserve of baseball, basketball and football executives.

  In the summer of 2013, the Edmonton Oilers hosted the Oilers’ “Hackathon 2.0,” where the club invited any citizen with an idea for a metric or statistical analysis to submit it for consideration. The written submissions were then winnowed down to allow a few of the very best to make their case in person before Oiler management personnel. Also, in August of that year, Dellow and others were part of organizational meetings with Oiler management and coaching staffs.

  The Oilers, for years, have had an interest in analytics, employing Dan Haight, who is the chief operating officer of Edmonton-based Darkhorse Analytics and managing director of the Centre for Excellence in Operations at the University of Alberta.

  There’s no doubt the Oilers’ acquisition of David Perron from the St. Louis Blues on July 10, 2013, for Magnus Paajarvi and a second-round pick, was driven in no small part by the team’s advanced-stats analysis.

  Haight’s work with the Oilers has been applicable on numerous platforms throughout the organization, but perhaps most noteworthy—at least from a public or measurable way—was Haight’s involvement in the Oilers’ 2013 NHL draft.

  Remember the name Marc-Olivier Roy.

  The Oilers made the six-foot-one, 182-pound forward the 56th-overall pick in the 2013 draft. Oilers head scout Stu MacGregor and his staff had liked the 1994-born forward with Blainville-Boisbriand of the Quebec Major Junior Hockey League as a potential Oiler draftee, a prospect they had some interest in. But Haight, using an advanced statistical evaluation of draft-eligible prospects, had flagged Roy as a high-priority target.

  “A lot of data gets put into the hopper and this particular player, with his statistics, the team that he plays on, the league he plays in, the numbers he put up, all of those factors and others, his profile was one that was identified as having a very good chance of being successful in the NHL,” Edmonton GM Craig MacTavish said in September 2013, when Roy was playing for the Oilers’ prospects in a rookie tournament in Penticton, British Columbia. “So we made drafting [Roy] a priority.”

  The Oilers selected Roy late in the second round, though they used their interest in him to employ another draft metric, the relatively common practice of parlaying a higher second-round pick into multiple lower picks. In fact, Oilers’ senior vice-president Scott Howson has his own metric or formula to determine the value of specific picks, when it makes sense to trade a higher pick for multiple lower picks and how all those numbers should add up.

  The Oilers were scheduled to pick 37th overall in the second round of 2013. The Oilers could have taken their primary target, Roy, at that point, but knew he would likely still be
available later in the second round, when they were scheduled to pick again (56th). The Los Angeles Kings, meanwhile, had a particular prospect they were after—Valentin Zykov of the Baie-Comeau Drakkar—and recognized he likely would not stay on the board until the Kings picked at No. 57 overall.

  So, reasonably certain Roy could be had at No. 56, the Oilers traded the 37th pick to the Kings, who took Zykov. In exchange for that pick, Los Angeles traded to Edmonton the 57th, 88th and 96th picks. Edmonton then traded that 57th pick to St. Louis (who took William Carrier) in exchange for the 83rd, 94th and 113th picks.

  Edmonton selected Roy with their existing No. 56 pick, but the knowledge they would get him there allowed them to parlay that 37th pick into a total of five more picks that became prospects: Russian forward Bogdan Yakimov (83rd), Russian forward Anton Slepyshev (88th), Vancouver Giant forward Jackson Houck (94th), London Knight forward Kyle Platzer (96th) and forward Aidan Muir of Victory Honda (a Detroit-area midget team).

  Even within the Oiler organization, how Roy fares as a prospect—to say nothing of the additional five prospects who came to be Oilers mostly because of the analytics-driven desire to draft him—was being viewed as a noteworthy test of the analytics process, and the application of #fancystats to the evaluation of draft prospects.

  One suspects this is, in various forms, an exercise repeating itself throughout the NHL. For every NHL club that rejects #fancystats, I would wager there are two or three more clubs that are far beyond debating Corsi and are working on far more sophisticated concepts and metrics tailored to their individual needs and preferences.

  If there is a battle between the traditionalists in hockey who eschew #fancystats and the growing legion that embrace them, I have bad news for the old guard.