Chess.com's Danny Rensch on Identifying Cheaters
Vložit
- Äas pÅ™idán 22. 09. 2022
- Chess.com Chief Chess Officer (made up title) talks to Hikaru about some of how Chess.com identifies cheaters based on not just their number of best moves but also their number of bad moves. I found it fascinating and i'm sure you will to because i'm telling you to.
-----------------------------
👕 MERCH ► streamlabs.com/gmhikaru/merch
â™Ÿï¸ LEARN CHESS & PLAY WITH ME â–º go.chess.com/hikaru
🎠GIVE CHESS ► www.chess.com/membership/gift...
ðŸ“½ï¸ GMHIKARU MAIN CZcams CHANNEL â–º czcams.com/users/GMHikaruCli...
ðŸŽžï¸ MORE GMHIKARU CZcams â–º czcams.com/users/moreGMHikar...
💜 TWITCH ► / gmhikaru
💖 INSTAGRAM ► / gmhikaru
🦠TWITTER ► / gmhikaru
✨ TIKTOK ► / hikarugm
💛 DISCORD ► / discord
💙 FACEBOOK ► / gmhikaru
💪 GFUEL ► gfuel.ly/2LS5Rct​ and use CODE HIKARU
💚 SUPPORT ► streamlabs.com/gmhikaru
🤣 REDDIT ► / hikarunakamura
â”â”â”â”â”â”â”â”â”â”â”â”â”
🎥 Edit by: ChaeDoc - ChaeDoc?s=20
👌Channel Management - ChaeDoc - ChaeDoc?s=20
📧 Business inquiries only: TeamHikaru@WMEAgency.com or teamgmhikaru@gmail.com
â”â”â”â”â”â”â”â”â”â”â”â”â”
Music in this video:
Song:
Artist:
â”â”â”â”â”â”â”â”â”â”â”â”â”
#gmhikaru​ #clips​ #twitch #chess - Hry
*FIDE:* "Our method is a bit more involved, first we ask the suspected player to pinky-swear they didn't cheat. If the player passes this step, we have Ken Regan review two random games of the player's to see if the games were 100% perfectly accurate. If the games were not 100% perfect, the suspected player is considered innocent. We are very proud of the rigor of our system and can proudly say that not a single cheater has been discovered in FIDE history."
I feel like Ken Regan's method and the chesscom method don't seem that different. I think Danny is just better at explaining
"Additionally, we now monitor all chess players' butts for any suspicious vibration during critical positions."
I love the "We are very proud of the rigor of our system and can proudly say that not a single cheater has been discovered in FIDE history.". That’s corporate disclaimers in a nutshell
@@elijahbuscho7715 Ken Regan’s method still isn’t anywhere near perfect
I think it’s good enough to detect any blatant cheating, but again…….. if you cheat subtly enough it’s simply impossible to detect through any statistics
@@elijahbuscho7715 I think the chesscom method is actually better (i.e. not just a better explanation).
This actually follows more in line with general statistical analysis than simply comparing games to top engine moves like others are doing
If you're trained in statistics, Danny actually just gave away a huge amount of information on how the algorithm works. But yeah, it sounds like they're doing a lot
​@@hitppohiman Actually, that's not true. Mr. Rensch merely explained an adjustment made in the data preprocessing step for the data used in their algorithm. The bucket analogy is a well-known technique in which you change how classes are determined, so that hopefully you're giving an easier task to your classifier.
@Edwardian23 they could be but I doubt it, the bucket of moves would be the same for most engines so why bother computing the same thing over and over to achieve the same result.
@@hitppohiman He didn't. And it's very unlikely the algorithm can do anything more than detect the outliers in the sport: the ELO1000 that suddenly make 2500 moves all the time. Not the ELO2500 who cheat a little bit by just blundering less often, something that is likely to occur when a player is naturally improving.
There's a category of statistical outlier which may be beyond the reach of any algorithm. Let me explain with an anecdotal example:
In the period before Kasparov's matches with Deep Blue, the developers honed their results with varied approaches, among which was putting their boy into the ring with ICC (or equivalent) players of all strengths.
A certain player, who himself had a background in high level programming and was a classically trained trombonist as well, was well familiar with the principles of the game -- but for whatever reason languished among the middling class players.
So, his moves against the machine could not be fully evaluated as chess moves only -- since, against an opponent of his caliber, the machine was overmatched. This was born out by his record, which was far enough outside of the norm for the site administrator to 86 him. This in spite of the fact that any chess program available to amateur or computer professionals of the day was far inferior to Deep Blue!
Hans taking notes 😂😂
Hikaru is the guy who says 'uhum, look, I need to go' the moment you've finished pouring your heart out.
Thats on point lol
All these top chess master are on spectrum
As much as I understood, so long short is that it's not only about how often do you play the top moves but also how less you play blunders/mistakes/worst moves
yes, danny just had a very longwinded explanation of such a simple concept
I think this can be the solution to the "smart cheating" problem
Wow so innovative 💀💀💀💀
@@nehcrow Sounds like he had to keep filtering info on what and what not to share regarding the algorithm.
No, that's not it. The frequency of blunders doesn't necessarily decrease significantly. As far as I understand, what does decrease is the frequency of incorrect critical decisions (i.e. best moves if they work, blunders/bad moves if they don't).
Funny how Hikaru hit em with the "ok nerd but did you see that catch?"
As a sports fan myself, Hikaru did a good job of handling it. He kept it to a minimum by only interrupting on the biggest catches, and he was open about the situation in advance (it's literally on his screen)
Truly fascinating and I do love it. I'd listen to a full talk with Danny about this topic
Or actually one of their statistical engineers/mathematicians who (hopefully) work on these things.
It's never going to happen. He already mentioned that he's saying too much. While it is incredibly fascinating, revealing their analysis strategy would also reveal to the cheaters how to cheat around the system.
Fascinating and would love to hear more
as someone with stats background my insight to this is that your moves are being plotted and tracked on a distribution as well as just the mean. while everyone expects a player's mean move score rating to shift over time along with skill improvement, i think what he is saying is that shape of the distribution is also changing and how it changes matter. we expect good players to make more and more good moves and less and less bad moves, meaning some skew develops as ELO goes up and there is an expected rate here as well. when engines appear they probably have a very tight distribution and almost no skew.
This makes a lot more sense than what Danny was saying (no offense to him: I have received some statistical education and have never watched american footbal). Thanks a lot :)
So a cheater would never swing outside the strike zone, but be perfect when he does. A super GM would swing and miss when he thinks he knows a pitch is coming outside the strike zone. So you can identify cheaters by no missed attempts outside the strike zone.
Thanks for helping my pea brain understand this lmao
Yea that Astros analogy was good. Not looking at how often they swung correctly at strikes, but how often they didn't swing at non strikes (pitches outside the zone). Top engine moves vs top engine moves at the right time? Or perhaps knowing when to try for that top move. Something like that. Man, I want him to write a paper or talk more about their process using the Astros.
@@jersey282 I had a bit of difficulty following the nuance of the analogy. By not swinging on a ball, he could also mean avoiding playing the "obvious" move. Tbh, I would have thought a top GM could pick out the best engine moves relatively often. So to catch a "smart" cheater on a handful of games seems impossible ; the cheater would have to be doing it a lot for them to have enough data for it to be significant - which is maybe why they don't buy the "I only did it twice" explanation.
Basically it’s impossible to cheat without getting caught unless if you do it once a year which’s not gonna change your elo noticeably lol
@@Unknown1Percent but will land you prize money at lower rating points.
danny boi is smart asf
is it possible he has smart data scientists that works on this problem?
@@nawzyah It's certain and self-evident.
"i'm sure you will to because i'm telling you to"
damn hikky chill
Basically players' ELO is a mean of a normal distribution, but not only mean is important but also a spread. They probably calculate the strength of each move and if standard deviation is too narrow then something is fishy - computers play always at the same level, but humans have better and worse days in the office. Also, normal distribution should be symmetrical, not skewed (more good moves than bad) - that would mean that the player is cheating by selecting too many good moves or by eliminating the bad ones.
Sounds a lot more like a machine learning predictive analysis than simply comparing differences between variances at different intervals of time. It’s possible, but very reductive in the info it uses compared to ML. Plus he said they convert them into buckets (perhaps like their game review uses, “brilliantâ€, “bestâ€, etc.). There’s no real way to do that in the way you mentioned
But as a player improves, won't you see a shift in the occurrence of both good and bad moves, creating a deviation? Players can climb from 2200 to 2400 to 2600. People get better, learn new things, refine their skills, and have mental break-throughs. Like if you're learning to play the piano, the number of mis-struck notes is going to fall through lessons and practice until you're excellent. How can you tell the difference between organic improvement (Neimann's claim) and cheating (Carlsen's claim)? I appologize if this is an incredibly stupid question, hahaha. Understanding statistical analysis is not among my strengths, haha.
​@@eckroattheeckroat4246 I suppose they don't just compare between the past and present performances by the individual player (i.e., Player A). It could be that Player A is being compared to a cohort of many other players whose rating is similar to Player A. Hence, we don't need to concern with the player's own improvement interfering with the analysis because Player A's performance is constantly being compared to many other people with a similar level of playing. If Player A performed just like those other players, who are good citizens and don't cheat, statistically speaking, Player A will show a similar amount of many average moves with some exceptionally well (best // brilliant) and some exceptionally bad (blunders) across many different games compared to the cohort. If Player A cheated across many games, his performance will show a lot more best moves + a lot less worse moves compared to his cohort. This is a much simplified version of what an anti-cheating engine looks like, but I hope it covers the general concepts. The real engines have to more complicated than this haha. Please take my words with a grain of salt because I'm not an expert in statistics either. The problem the current chess world has, as mentioned in one of Hikaru's recent videos, is whether the real, more advanced anti-cheating algorithm we've got can detect cheating in one or two critical moves. More, the current anti-cheating algorithm doesn't seem to have the ability to do so.
​@@eckroattheeckroat4246 Not exactly. Think of a class being graded on an exam. A normal distribution of grades would have the number of As equal Fs. If the class has more As than Fs you know something is wrong. The class can still improve, however the distribution is agnostic to that.
Think about it like this. You can tell the difference between an expert pianist and a computer rendering the music because a computer plays strict times and follows the dynamics perfectly. A human does not. They vary the time and interpret the music slightly to add their own push and pull. If you analyse the music statistically you can see that even in the expert's performance.
@@agravphili But some people improve faster than others right? Also, people don't always improve at a constant rate, some people take rests, some people go all-out all of a sudden. Calling someone a cheater just because some statistics says that he/she is expected to play more poorly doesn't make any sense, to me at least. A cheater is only identified when caught in the act, there is no other way.
My DNA of moves is to play E4, knight to F6, and then hang my Queen a few moves later. No one ever accuses me of cheating.
Way to keep it honest! lol
Quality analysis and content, guaranteed! Ironclad academic discourse without innuendo, this is it - everything I love about Chess - a reflection of societal perfection! Love the comments on this vid BTW.😂
danny ranting at hikaru reminded me of "guy yelling at girl meme" lol
to say it simple: cheaters elevate their level of play which also means they mitigate the blunders. that way of hightened consistency is not human-like and it marks them as cheaters.
Bro I cheat this is what I do he is right
Love how Hikaru is just looking at Thursday night football while Danny gives an insightful rant on how they identify cheaters. Never change, either of you.
Danny: "....So, over time when you aggregate data... Hikaru: TOUCHDOWN!!!!!!!!!!!!!!......Now that would have been funny😄
Since the whole cheating scandal with Magnus and hans, there have been floods of cheaters in the online pool. But in fairness most of the people iv reported have been banned so big ups on responding to reports!
I am pretty sure I got cheated. I am 1800 got crushed by a 1000. It's actually rare at my level to get cheated I think.
@@peterhardie4151 Everyone that I noticed cheating was 900-1200. the
@@wee_pizza that's funny.
I was using a weak engine about 2400 online blitz rating. Just to see how it performs and how far it could go. I was still caught although I was pretending to wait, think etc.
Already finish? I think I felt asleep. Fascinating stuff!
I love how you’re giving it all away. The top cheats will appreciate that.
Playing a blitz vs classical games has to change these stats and analysis. Furthermore, do they document every player on all formats/tournaments in each and every move as well as time spent to make a move?
"My analysis speaks for itself" - Hans Niemann
Not understanding your own move is honestly proof someone is a cheat. It's why other GMs like Hikaru, Nepo, Fabi, etc are very sure that Hans is a cheater. The analogy would be like let's say you're a phd student you read a phd paper made by a guy, then you talk to the guy and you figure out that his knowledge is at high school level, you have no proof that his paper is not his own but you are sure he is a cheat.
This is lowkey amazing to hear about.
Hans taking hella notes rn
I might have turned the game volume down, but at the same time it’s nice to listen to a real conversation.
danny explained very well
(when he starts talking about buckets)
"mm yes bucket sort"
(my mind declines to a point to where it's putting numbers into buckets instead of figuring out what he's talking about)
Hikaru always have that sarcastic teacher look when he ask you about a question. He nods in a way "Hmmm really?" sarcastically.
I know Danny is refraining on revealing too much but DAMN I should have paid more attention in statistics class
"Correlation does not equal causation."
@@munchkinmatt1670 If you're not careful with data, two very different random things can appear correlated. Commercials often do this in their ads to persuade you to buy their products (by not being entirely honest with you).
Basically Ken Regan is looking at data as correlation of your moves to best engine moves, and doing that by putting huge chunks of your games into it. Meaning every game you didnot cheat, every move you didnot cheat will prove you are innocent, meaning even if you cheat half of your games with half engine moves, you will come clean.
These guys are looking at much more detailed aspects of your data, inspecting every suspicious game on its own, as well as your chronological play level.
First method is complete garbage, 2nd is apperantly best we have, and at least somehow promising.
@@munchkinmatt1670 Did they teach you what 'causation' meant?
@@yzfool6639 By pure logic, causation is not correlation.
But what if people improve quickly or just have a good game or three? Would be interested to hear more.
If the important aspect of this moment (rather than the overvalued drama) is learning more about cheating and how to prevent it, having an experienced professional giving extremely relevant insights to the conversation (which should also be news to Hikaru) being met with "hold on, football" hints at how much of this moment is generated drama by our "drama seeking glands". We should keep that in mind.
I like Hikaru, but was very poor taste to cut off his guest in the middle of their explanation like that.
I believe it might be code for "don't say more".
Just cuz Hikaru is distracted doesn’t mean everyone is. I want to know how and why people are cheating. I find it extremely interesting. An alternate theory is that we may just be witnessing Hikaru transitioning from unique, quality chess content to just another CZcams streamer.
@@AronMarkCsernak wow that actually makes sense
No, the important aspect of this moment is if Hans cheated or not lol
Neimann nearly broke their system...
Statistics seems to be the key. Seems like in a match between two grandmasters, if you can play that one move that bumps your eval bar up by .5 in a game then you are going to win a lot of games. I had always wondered how often GMs find that move in a match statistically and if you could use that metric to spot people who seem to find it much more often than others.
… or not play that one slightly negative move
love the Astros analogy/comparison. It's easy to think people would be cheating to win, but the idea that the engine will help you 'not lose' at a key point is mega interesting.
Interesting part about the Trashtros was different guys interpreting the trash signals differently. So Altuve was jumping on first pitch fastballs all those years with remarkable consistency, meanwhile Bregman would wait for the bang and turn on breaking pitches at a historic rate. What an absolutely massive advantage it is to know a breaking pitch is coming...anything in the strike zone should be smashed while literally everything out of the zone gets spit on.
That’s exactly what stood out to me. Danny said he didn’t wanna give away too much involving their algorithm. I made me think… what if they had some crazy statistic like Hans has a < .5% blunder rate in critical positions for all chesscom games. Or something like that.
@@Coreyinthehouse got me thinking of recaps and hearing "the only move that doesnt lose is..." - its just as critical as finding the winning idea. Agree they will have a lot of interesting data on how people play in those situations
@@willc3421 wow that’s a good point. He said that a lot. If you play deep into a game where you know your moves won’t be detrimental, you can patiently wait for your opponent to slip up
Someone please can explaine what he is talking exactly? I am not good enough in english sadly.
Omg more!
Hikaru: how dare you make a point when someone makes a good catch
Danny: sorry
Great correlation with the astros.
I didnt get the impression that Danny really understood what he was talking about.
Next we need Danny to establish the DNA profile of the real William Shakespeare.
This is a good strategy to catch cheaters. I always try to play the types of moves the computer suggests when I’m reviewing my games, but because the position is slightly different my moves are always wrong
He said DNA. Danny nos all.
And how Danny Rensch let the cheaters back into circulation.
Makes sense why when you play engines they play tons of best moves then just hang there queen for no reason to try and fit in 1200 strength instead of playing just average moves throughout
Honestly, as a titled player I would love to have insights into my "DNA" by position type - seems like a great way to improve and to find holes into your game.
As a MSc and a guy who worked in ML & data analysis field I'm still surprised that there's so little data mining in top chess (or in chess in general); instead it came down to memorizing boring Berlin and Italian lines up to a draw smh
I'm glad you worked in ML. I actually think this DNA thing is simply an analogy and what is described here is a data preprocessing step to classify your data in a more robust manner (assuming that was done because of noisy outcomes) while at the same time, giving an easier task to your classifier (instead of classifying digits from 1 to 10, let's just classify odd number vs even number). If we assume you have enough data for each player and you can build a distribution that matches the likely moves such player can reasonably make, then that could be useful to catch normal cheaters. However, if it is true that top GMs only need to cheat for a few moves in critical positions, the same approach gives enough room to make choices that are within the known distribution and still provide an advantage to win games.
@@rsmith31416 From his description, it looks more like an anomaly detection system than a classifier. Honestly, it also sounds like a fun project to work on but maybe with a twist - instead of cheating detection system one could build something like a player profiler tool to automate the process of opening prep.
@@boccobadz I didn't get the feeling that they are working on anomaly detection but who knows... It is certainly an interesting project anyway.
These football references are great.
So basically, whenever a cheater plays a risky-looking or complicated move, they're disproportionately more likely to play a best move than a blunder.
That's not how I understood it. I understood it as that I am likely to make a certain number of suboptimal choices and a certain number of optimal choices given my chess DNA. When that discrepancy swings too much in my favor, I am likely cheating.
​@@yzfool6639 Okay but what if you are a young player that is dramatically improving and you just started working with a new coach who is encouraging a dramatic shift in your style of play, for example: more or less risks. Does this then make your chess DNA irrelevant?
@@HEEHHOOH Even that is not enough to create a dramatic rise at the highest level.
Very very interesting stuff
Is there a full video? Couldn't find it
Cavity search. The gold standard in cheat detection
Just as good, an airport whole body screen (the one you walk through and raise your hands after already emptying all your pockets, watches, jewelery, etc).
I cheated using Leela (lc0) and I'm still not banned in spite of 2450+ blitz rating lol. So that DNA not working (I used 1st or 2nd line of multiPV)
nice cheats :)
My suggestion is, at a minimum, have played in faraway cages that will keep out radio waves. Then work on machine learning that catches cheats. You will have the Stockfish level algorithm that detects cheating.
My question is, how do we catch a cheater in that case who cheats for a move at a critical time (not the top engine move but second to top or something) and besides that just because the player is in general a strong player, he wins the game and rest all of the moves are good enough?
You cant catch these people
Bottom line: when we say Hans cheated more than once at 16 yrs of age we are comfortable enough to go to court with it. He is a cheater.
Where can I find this full stream?
Just bring a bible and make every palyer swear with their hand on it that they'll never cheat. Works well in the court room.
Obviously you are highly delusional.....
Their algorithms are simple it’s just establishing priors and updating them based on a time periods current approximated prior so they have a population (total individual history) and some other densities and it’s quite straightforward they do max, averages, top N
That is... the most wordy way to say that as the probability/# of occurrences relative to # of moves played of the best and top 5 moves are played inflates, the probability/# occurrences relative to # of moves played opposite moves (worst and bottom 5 moves) must deflate in the data set...
Top players would really only need too use a computer for a few moves a game, too gain a massive advantage, how would they catch this?
Danny always drawing gaussian distributions with his finger in the air
to sum it up. somone knew the prep of an opponent and played it to his own advantage?
So they build a Biometric of a player, pretty fascinating. So have a "fingerprint" of a person, based on their play over many games. It's still not 100%, but definitely great idea. You also need to account for different time controls. You might also have some things like sickness, playing on a bus on mobile phone, etc. that can make a player play outside the norms. However, these would be in the "bad move" area and not the good move area.
So the probability of how someone plays all types of moves good/bad/other/ give an accurate picture of whether someone is cheating.
That's a weird statistics to measure skill, unless you monitor cheating for top players only.
Players are using the engine to avoid inaccuracies but not technically playing the "best" move?
Danny sounds more like a data scientist than a chess player
He is the Cheese Chef Chi-Cha-Cha Officer
So a king sacrifice is considered top move or best move?
Why did hans turn off engine his last game?
He already crushed 2 more GMs way above his skill level. I bet he was planning to lose to Magnus as well, if he played.
If indeed Hans is cheating and his method of cheating involves some sort of relay from an accomplice, and that relay involves vibration (whether or not it takes place in his rectum), it's feasible that their mode of communication only identifies which piece should be moved and not where precisely it should be moved. So for example a short burst for a pawn move (with an increasing amount [but not duration] of bursts from a pawn to h pawn), slightly longer bursts for a knight move, slightly longer for bishop and so on up. Hans' losing move in the last game against Le was moving his knight backwards instead of forwards to b3, which would have maintained good drawing chances.
the dog that didn't bark....
I bet they also use "time for the move" as a variable.
okey and what if the player arrtificialy makes his own DNA by playing best moves/blunders at the same rate each game with the help of computer and using that to cheat in every/most recorded game/s ?
Yes. A Prestige type swindle is possible.
Exactly this is still an easily flawed cheating system lmao.
That wouldn't be considered cheating. That is what top players do right now. They simply memorize the best moves and agree to play them against each other. Their ratings only fall precipitously once they run out of memory. That's why they agree to draw before they reach that stage of decline whenever possible.
@@yzfool6639 I think that you misunderstood what I said. I am not talking about memorization of any moves, I am talking about cheating algorithm that will suggest moves based on your best move/blunder ratio.
Your DNA (best move/blunder ratio) could be artificially changed by always using that cheating alghorithm in every recorded match. Your DNA will be stable (just as if you wouldn't use the alghorithm), so the anti-cheating software wouldn't recognize the cheating. But your ratings will be artificially higher (or lower if you would wanted to).
Of course you would have to know how the anti-cheating software classifies best moves and blunders, but that could be done in a few ways. (easy example would be to have insider who can directly look at the code)
How many games does it take to bust them? I'm thinking 30+, which is still horrible. I remember cheaters busted after winning 25 straight...... I say put them in a different pool than veteran accounts
The takeaway is that they not only collect your best moves but your worst moves also. And when you cheat, not only do your top moves increase, but your worst moves decrease to the point of disappearing altogether.
That's how I understood it too.
Not even dissappearing bad moves, just having a statistically significant portion of your bad moves vanish could prove that you are cheating
Think of it like this. Say that as a chess player, at critical points in the match I make a really good move 30% of the time. But it's also not just that. When I don't make a good move in the original, out of the remaining 70% of the time I make a move that puts me behind 20% of the time.
Even if I'm not making the best move every single time, those numbers, how often I make a really good move and how often I make a move that puts me behind will stay reasonably consistent or go up over time (or down).
If I suddenly had a tournament where I was making a really good move even 35% of the time, and I only make moves that put me behind 10% of the time in critical situations, then you already know something is weird. People don't magically get way better over a weekend, or heck probably even a year. You don't go from losing against nobodies in a foreign local tournament to beating the best players and becoming the front runner of a world level tournament. That's what he's saying
So, no matter what age of engine or whether it is top or 2nd or 3rd computer choice you will be screwing up your appropriate blunder value. None of the top 100 engines will do a 3 move blunder like you usually do, and you will be messing with your human skill floor value instead of just the usually examined human skill ceiling value.
And that's why I know people who suspect Niemann of OTB cheating haven't done their homework. He has played his worst moves just as much as he's always done, including the game against Carlsen. People are selectively looking at his best moves only, but his worst moves follow his usual pattern, and the aggregate reveals a cheat-free Niemann OTB.
thanks now i know how to break the algorithm
I know all about statistics, markov chains, labeling, machine learning and data science in general. It is my dayjob and it has helped the companies I've worked at.
But I have no idea what he just said.
I imagine you are already exploring visualizations Danny, et al. Pictures are worth thousands of words. One obvious picture is simply to look at the graph of rating progress.
There's definitely a crazy machine learning AI that analyses every game and can flag suspicious activity. Using machine to detect machine
let's run a lie detector test on that lie detector
" OK machine, are you telling us the truth ? "
Couldn't a cheater have the engine list out the top say 30 moves and the cheater first check to see if the move they want to make is in the list and second whether it's a bad move?
Don't interrupt Danny!
When you aggregate buckets at t1 t2 t3, etc you get a "bucket DNA" its quite simple
Hans watching this vid taking notes
I never make bad moves and engines use me as training data! Do you have advice for someone who is perfect like me? If I make a wrong move will it be considered cheating because I deviated from my chess profiling?
As I understood it, basically the Big Data model computes your "DNA" of moves, how you usually move within the classification of those moves e.g. E1, E2, E3, buckets etc. If your game deviates from your "DNA" then you cheated. This is statistically known as the Law of Large Numbers.
You can imagine Magnus having a distribution in E1, E2, E3, mistake, inaccuracy, blunder buckets different than Hans. Hans's DNA of moves is different than Magnus's.
What about drunk Magnus?
I think the core problem is that many do not formulate the correct null hypothesis. Their null is no cheating occurs if there is not an `unusual' occurrence of top engine moves. This does not work in high-level chess. Your null should be: no cheating occurs if the distribution of move evaluation values (doesn't have to be even top moves) is consistent with his previous performance (his DNA) and reference players of the same elo range. An important point is that these tests will not be very powerful for individual games, so high level cheating probably cannot be proven on the level of individual games, but only historically for multiple games. As a result, I the logical approach is to ban all players who have been cheating online or on the board from ever participating in any tournament ever again.
I hope this doesn't flag experimentation. I've been using this old favorite opening, but suddenly used a new hidden novel opening that I studied intensely under a GM, taking classes and buying his books, etc. A secret technique to play high level players, so to speak.
@@simpletongeek You can tune the model to give different weights to moves beyond prep which I'm sure they do
Putting aside the misleading analogies, increasing the range for each class is not related to the specifics of the algorithm used. However, for some models, you can certainly build an actual distribution for each class as you described in your second paragraph and that could have some value to flag outcomes that are consistently unlikely. Having said that, if a top GM only needs a handful of moves during a game to win, then this sort of distribution is not very useful since a smart cheater can always pick and choose their next move that guarantees an advantage and still keep their moves within the expected distribution.
The football noises in the background were absolutely not annoying at all, please make sure we can hear them in all chess clips from now on.
This was a football stream on Twitch, some of which they talked about other subjects - like this one.
Really interesting. What a shame all the noise from the football, makes it uncomfortable and difficult for non natives
I've cheated a lot in chess, i only would use engline after i blundered, it works very well on amateur level
Basically doing the same as how they caught Lance Armstrong with his changing hemoglobin levels. Oprah-Hans interview ASAP!
Nice! Giving it away for potential cheaters!
If I cheated I would not be trying to remember all these stupid Gambits and my rating would have not dropped back down to 940 ha ha
It will tell you that something (may) be wrong
My bucket is full of vomit and diamonds.
to me it sounds like danny is using a ML model… like a classifier
I was banned because as a 1700 or so blitz rated player I had the audacity to win a game against an FM. Why play the game if the only acceptable result is defeat? Hmm, interesting
Definitely, time controls should definitely be taken into account. Also another thing to take into account is that if the other person blunders, you finding a great move shouldn't be too suspicious.
The thing is: you need a very, very large database of moves for every single player before you can capture the DNA of a player, and every single player isn't a static one: players evolve, have good and bad games, are on the way back or the way up.
You have to take into account that players in a winning position are MUCH more likely to make more accurate moves than in a complex position. So you have to take into account the evaluation score WHILE evaluating a player's moves. Another strong influencing factor is the opening vs. middle game vs. end game: a player is much more likely to play accurate in the opening, as this can be learned through studying. It's much less likely to consistently play the best moves in the middle game, I don't know about the end game (it depends on the number of pieces on the board and the position: some endgames can be ridiculously complex.
So whatever analysis they are trying to make: it's very complex, especially once you analyse >2500 ELO players. You could easily detect a cheating 1000ELO player using a computer, but it's far more difficult once the level goes up: Top level players are less unlikely to make good / very good moves, and as long as they don't cheat on all moves, it's much harder to detect: it's merely an indicator.
Good luck, Danny. Your fight against random cheaters for sure will be successful. But against bigger foes with intention, it will be really hard.
For sure. If i was a super gm using an engine i would KNOW which moves are sus and I'm pretty sure I'd be able to cheat forever without any algorithm catching it.
So this would mainly work on non gm players that don't understand what move is sus and just follow the engine's top moves.
Hell GM's would get a massive advantage without even looking at the engine's moves but only the evaluation, how you catch that?
"Just use normal people words, then maybe I could understand you" - Ricky, Trailer Park Boys.
I think, the key thing is to catch the outliers and compare them to the "value" of the specific games. Every top player once in a while has that sensational game above his standard strength. If you find a greater number of such games, there is suspicion. If you find a lot of those games in important situations (GM norm tournaments, high rated opponents, prestigious events) the suspicion rises. If you find over the top performances mostly or only in crucial games, suspicion becomes near certainty.