A brand new examine by one of many world’s main statisticians has discovered that GM Hikaru Nakamura‘s spectacular streaks towards high-level opponents on Chess.com are effectively throughout the anticipated chances. GM Vladimir Kramnik strongly criticizes the report, dismissing it as “utterly ridiculous” and accusing Chess.com of “apparent manipulation.”
Jeffrey S. Rosenthal is a Professor of Statistics on the College of Toronto and a globally acknowledged professional in likelihood and statistics. He’s the creator of a number of acclaimed books, reminiscent of Struck by Lightning: The Curious World of Chances, and is well-known for his potential to clarify complicated statistical ideas to the general public. He additionally has a lifelong ardour for chess and enjoys the sport as a pastime.
Rosenthal’s Conclusion: “Not Significantly Shocking”
The Canadian professor has launched a brand new 12-page paper titled Chances of Streaks in On-line Chess, which was accomplished in response to a suggestion by Chess.com. Within the examine, he meticulously analyzed Nakamura’s streaks primarily based on the American grandmaster’s 57,421 video games on the platform from 2014 to 2024. Rosenthal famous that his examine ought to be seen as an investigation of unlikely streaks, and never the broader problem of dishonest in on-line chess.
In his detailed evaluation of Nakamura’s efficiency, Rosenthal centered on one of many streaks in on-line blitz the place he scored 45.5/46: “This statistical evaluation signifies that Hikaru’s on-line chess successful streaks usually are not notably stunning. His current controversial streak of size 46 is effectively inside anticipated ranges,” Rosenthal concluded in his examine.
This statistical evaluation signifies that Hikaru’s on-line chess successful streaks usually are not notably stunning. His current controversial streak of size 46 is effectively inside anticipated ranges.
—Jeffrey Rosenthal, Professor in Statistics, College of Toronto
Nakamura, at the moment the world quantity two in classical and blitz, got here beneath scrutiny by Vladimir Kramnik final 12 months, with the previous world champion questioning his successful streaks, saying considerably cryptically: “I consider that everybody would discover this fascinating.” Kramnik referred to as for additional investigation of the likelihood of such a streak, and whereas not explicitly accusing Nakamura of dishonest, the worldwide chess neighborhood has interpreted Kramnik’s feedback as a transparent insinuation of foul play.
Kramnik wrote in a weblog publish on Chess.com:
Having checked Hikaru’s statistics fastidiously, I’ve discovered NUMEROUS low chances performances each of him and a few of his opponents. A few of which have EXTREMELY low mathematical likelihood, in response to mathematicians. Manner beneath one p.c, in response to the calculations of these skilled mathematicians.
Rosenthal: “Issues That Appear Uncommon Are Not Essentially Uncommon”
Nonetheless, Rosenthal’s examine discovered that Nakamura’s streaks, whereas distinctive, fall effectively throughout the statistical expectations for a participant of his caliber. “Issues that appear actually uncommon usually are not essentially actually uncommon once you take a look at them in the precise context,” Rosenthal defined to Chess.com in an interview.
Issues that appear actually uncommon usually are not essentially actually uncommon once you take a look at them in the precise context.
—Jeffrey Rosenthal
Utilizing the Elo ranking system, Rosenthal calculated the probability of such streaks occurring naturally. He famous that Nakamura’s 45.5/46 streak had a likelihood of about 1 likelihood in 830, which isn’t not possible given the variety of possibilities he has had over the course of greater than 50,000 video games.
A sequence of 57,421 video games has about 57,421/46 ≈ 1,248 totally different non-overlapping impartial possibilities to realize a streak of that size, so discovering one with a likelihood of 1/830 is definitely very probably.
Evaluation Of Nakamura’s Most Notable Streaks
The examine additionally analyzed the likelihood for Nakamura’s longer successful streaks, outlined as a sequence of video games with no losses and at most one draw. Rosenthal discovered a complete of 226 streaks that lasted greater than 30 video games, with the longest 121 video games.
Nakamura’s common opposition was lower than 1600 in these video games, and Rosenthal famous {that a} lengthy streak doesn’t essentially have to be unlikely: “His likelihood of scoring at the very least 120.5 on these 121 video games then works out to 1/8.9, which isn’t notably unlikely in any respect.”
Amongst Nakamura’s most notable streaks, solely two are significantly much less probably than his 45.5/46 rating, in response to Rosenthal’s examine: “These not possible issues, one likelihood in 9,000 or one likelihood in 11,000 streaks… one thing like 43% of the time, there was at the very least one such streak,” Rosenthal mentioned, referring to line 4 and 6 within the desk beneath.
To additional validate the streaks, the professor performed Monte Carlo simulations, a way that makes use of repeated random sampling to estimate the probability of various outcomes. By simulating 1000’s of random recreation sequences primarily based on Nakamura’s scores and the scores of his opponents, Rosenthal discovered that the likelihood of reaching a streak like 45.5/46 was truly fairly affordable.
In every simulation, Rosenthal recorded the size and frequency of successful streaks that matched or exceeded these noticed in Nakamura’s precise video games. The outcomes supplied a distribution of streak lengths and frequencies, which Rosenthal then in comparison with Nakamura’s actual efficiency.
Rosenthal famous:
We see from the graph that, whereas the precise values 11,570.6 and 9,452.1 are bigger than most of the simulated most 1/likelihood values, there are additionally many simulated 1/likelihood values that are a lot bigger than that. Certainly, the biggest simulated 1/likelihood worth is over 284,000, and the imply simulated 1/likelihood worth is over 26,000, and even the median simulated 1/likelihood worth is 10,461.92 which is near Hikaru’s 11,570.6 worth. In reality, in 43 of the 100 simulations (almost half), the least probably streak is much less probably than the noticed 1/11570.6 one. This additional confirms that Hikaru’s least probably streaks usually are not stunning over such a protracted assortment of video games.
In reality, in 43 of the 100 simulations (almost half), the least probably streak is much less probably than the noticed 1/11570.6 one. This additional confirms that Hikaru’s least probably streaks usually are not stunning over such a protracted assortment of video games.
—Jeffrey Rosenthal
In his conclusion, the professor famous that regardless that the uncooked chances of Nakamura’s least probably streaks are every about 1/10,000, the prospect of reaching such streaks over the course of so many video games, remains to be proven above 10%, and about 43% in Monte Carlo simulations: in different phrases, not not possible.
Having two such notable streaks is considerably much less probably, however nonetheless happens about 18% of the time, effectively inside regular statistical variability. Total, the streaks noticed in Hikaru’s Chess.com report are pretty typical given the scores of the gamers over Hikaru’s lengthy report of video games.
Kramnik’s Response: “Utterly Ridiculous”
Whereas the findings recommend Nakamura’s streaks are statistically regular, Kramnik has dismissed the examine in sturdy phrases. Responding to Chess.com’s request for a remark to this story, the previous world champion despatched a number of emails describing it as “a ridiculous report that simply does not make any sense”, “a joke”, and “disgusting”.
“With all due respect, this ‘analysis’ is simply one other manipulation. I can stand by my assertion and show it in direct dialog with this gentleman. I feel he’s misinformed or does not know a lot about chess,” Kramnik mentioned of Rosenthal.
With all due respect, this ‘analysis’ is simply one other manipulation. I can stand by my assertion and show it in direct dialog with this gentleman. I feel he’s misinformed or does not know a lot about chess.
—Vladimir Kramnik, former world champion
He additional claimed the professor made at the very least “5 main errors” within the examine, reminiscent of mixing totally different time controls in analyzing the streaks and utilizing the Elo Ranking system to calculate chances with Chess.com’s Glicko system.”It is a fundamental primitive mistake. It makes the entire analysis fallacious. It is simply actually ridiculous. Imagine me.”
Kramnik additionally accuses Chess.com’s CEO Erik Allebest of “apparent manipulation”, suggesting he deliberately supplied the professor with “utterly fallacious information”. He challenged Rosenthal to a reside dialogue to debate the examine’s findings and said he has already recorded a YouTube video detailing his criticisms in additional depth.
Conclusions Align With Impartial Evaluation
The findings in Rosenthal’s examine are supported by conclusions in two impartial research. Kramnik vs Nakamura or Bayes vs p-value, performed by Shiva Maharaj, Nick Polson, and Vadim Sokolov, used Bayesian strategies to investigate Nakamura’s efficiency. Their evaluation concluded that the likelihood of Nakamura dishonest was extraordinarily low.
One other impartial evaluation, by software program engineer Kiril Bobyrev, additionally concluded that “successful streaks present in high gamers’ on-line blitz video games on Chess.com usually are not statistically unbelievable”.
Final week, IM Kenneth Regan, the American professor in statistics, additionally commented on Kramnik’s many posts on X/Twitter: “They are not normalized. He doesn’t do the statistical methods which might be required to determine a benchmark of reference, whereas I’ve. I’ve a predictive analytic mannequin, I set expectations, I do know the boldness intervals round them. These are fundamental statistical vocabularies which were recognized for the reason that 1700s however absent from his posts,” he mentioned through the broadcast of Conflict of Blames.
The College of Toronto printed their very own video with Rosenthal.
Professor Jeffrey S. Rosenthal famous that he didn’t obtain any compensation from Chess.com for his work.
