I been Nate Silvered

    Ken Jennings Posted by Ken Jennings.

    Ken Jennings, Ph.D. is a senior partner at VentureWorks and a managing partner at Third River Partners, a consultancy that specializes in leadership development and strategy execution.

    I been Nate Silvered

    FiveThirtyEight.com ran a smart analysis this week on the probability of another 75-game Jeopardy! run. On the whole, I agree with David Goldenberg’s main conclusions: the run is unlikely to be repeated anytime soon because

    (a) the competition is better curated and better prepped today, and
    (b) 75 games was an incredibly lucky run in the first place.

    (I don’t actually think that Goldenberg’s contention that strong players are playing more aggressively today, and therefore are less likely to build long streaks, is actually a thing, even though I think he’s largely basing this on a conversation I had with him about Roger Craig. But I’d be interested in data that says different.)

    The most interesting (and controversial) part of the analysis is the finding that my odds of winning any Jeopardy! game during my run were 97.9% (based on my average score going into Final Jeopardy, my average accuracy at Final, and the historic overall Jeopardy! accuracy at final.) Goldenberg’s simulations expect a player with those stats to win 47 in a row, on average. So there you go: finally, a mathematical answer. My streak was 64% skill and 35% luck.

    Except, as commenters point out, that 97.9% number is a little suspect. It doesn’t take into account games where my lead is narrow enough that I can lose to a small wager even if the second-place player doesn’t answer the Final correctly. (For the record, of my ten non-lock games, five were close enough that a wrong answer and $0 bet would have beaten me, including my first and last games.) That 97.9% inches downward somewhat if you consider that. Anyone want to do the actual math? I don’t.

    (Another commenter points out that Final Jeopardy correct answers aren’t independent events: a question that the leader gets wrong is more likely to be missed by other players as well, because it’s probably hard. I think this is a real but fairly small factor. Of my wrong Final Jeopardy responses, 58% were answered by at least one of my competitors. 42% were correctly answered by the player in second place. In other words, I missed plenty of answerable Finals.)

    In my opinion, the most important factor not addressed by Goldenberg is what happens over the length of the streak. My clear sense at the time was that playing Jeopardy! got easier the longer I was on, especially after the nerve-wracking first few tape days. The ever-snowballing advantages for the long-term champion are obvious: his or her increasing comfort at playing the game, and the psychological effect that playing a long-term champ has on his or her opponents. That would produce the “long tail” you see here: lots of short streaks because nobody gets over that hump, one long one for the player lucky enough to stretch the odds and pass it.

    In the article, Julia Collins opines that there is such a fatigue-related threshold that might explain the gap between the 5-7 day champs (63 of them since the “no 5-day limit” rule change) and the 8+ club (just ten of them). There might be a second comfort-related threshold that explains that big gap between Arthur Chu (12 games) and David Madden and Julia (20+). But I wonder if the most pronounced threshold isn’t somewhere north of 20, a place where the game suddenly gets a lot easier. If I had lost after a 20-game run, like Dave and Julia did, my lock game percentage would drop from 87% to a much more vulnerable 75%. But after those first 20 games? Twenty-eight locks in a row.

    So maybe there’s a second wind up there in the fourth week. Hang in there, Jeopardy! hopefuls. The record may be much more beatable than FiveThirtyEight thinks.