I Hacked Stockfish's Code To Learn How It Thinks (Part 1)
Vložit
- čas přidán 27. 11. 2022
- What is FEN?
www.chess.com/terms/fen-chess...
👉🏽This will take you from 1000 to 1500 📈
🎓 chessvibescourses.thinkific.c... 🎓 Reviews: Overwhelmingly Positive (32 reviews)
✅ My Courses! ✅
chessvibescourses.thinkific.com/
The chess set behind me:
The Pieces: bit.ly/3zBJzlO
The Board: bit.ly/3b3B8pb
👕 New Shirts! 👕
Chessvibes.shop
☑️ Play Chess Here:
www.chess.com/membership?ref_...
☑️ Support Nelson Here:
paypal.me/ChessVibes
Links are affiliate links and help support the Chess Vibes channel via a commission. - Jak na to + styl
CS major here, the benchmark file is for testing, the listed position are likely test cases, where you expect the engine to behave in a certain way or to find a certain move, or you're just interested in how it behaves in that position.
I mean, I'd imagine it's probably for benchmarking, I know stockfish devs are very concerned with performance regressions
on top of that it appears to also take in potentially other FEN starting position and which already have a predetermined outcome and evaluates the system versus those using the file CURRENT
This would make sense. With the name of Benchmarks, performance testing would be reasonable. It might even be a speed test for loading FENs.
it's a way to get a nodecount (the number of nodes the engine searches) under certain circumstances (the default is a search at depth 13 i think), it can be used for
1) having a unique identifier for a specific version of the engine
2) test the engine speed on a specific cpu to scale fishtest workers
They basically reject a stockfish development build if it fails any of these tests. It is for curation.
StockFish’s evaluation function has been replaced with the efficiently updatable neural network (NNUE backwards), that roughly halved the number of moves/sec it could consider, but the better evaluation counters that. All those carefully tuned “bitboard” tables about how many points to give pieces on different squares depending on phase of the game are now irrelevant. But if you look at Leela-0 you’ll see that its doing 100k nodes/sec vs StockFish’s 5M nodes/sec. That’s the critical difference: LC0 looks at way fewer moves, but has a much better understanding of the board. But I’d argue it’s still not a human level of understanding, as clearly no human evaluates 100k moves/sec.
Leela 1 node is about 2400-2500 elo tho. it is about GM level of understanding.
Yeah. It's not human level. It's above. That's why it has to consider that many positions. Duh. It could still probably be even more efficient, but also you just *need* to be able to crunch way more positions if you want to be >3600 elo lol. I think it's kind of ridiculous to think it "doesn't understand the game as well as humans" when it clearly understands it *better,* and not in just a tactical sense.
The key difference is not merely an evaluation function, but also a search tree expansion algorithm. The original stockfish was based on so called minmax strategy, while alpha zero uses Monte Carlo tree search, which is somewhat different.
Benchmark is self explanatory. But the 'moves' part of FEN actually defines the position. For example, you can reach an opening position by inputting '*start pos FEN* moves *all moves*' and UCI would translate that as the opening position itself.
The point of 9:44 is that the moves to play mean that Rg6 would produce a draw by threefold repetition.
Nelson, you always have very interesting content. I use a modified stockfish 13 I made to help me analyze my own games. I modified it to have increased valuation on mobility/space and opponent king safety. It doesn't play better than the original, but it plays a more sacrificial, aggressive game that I'd argue is more anti-human. I'd rather learn lines which gain initiative and aren't as deep. Moves which only make sense 10 moves later don't really teach me in any meaningful way.
I made another stockfish 13 which only plays hardcoded gambit openings and accounts for permutations, as well as shifting to a different compatible gambit if possible. This helped me train my openings a bit.
I'm working on another one which has a ridiculous amount of hardcoded trap positions it will always accept(made in a way somewhat similar to that Defaults list you were looking at). Stockfish won't otherwise play dubious anti-human traps which are actually effective against human opponents around my level.
I have an insanely tough crowd at my chess club who have all been playing 20+ years, so the only way I'm able to win is through taking them out of their comfortable lines with gambits, keeping initiative, and the occasional traps they still fall for. Plus it makes for some really fun games. I've only been playing for a few years so winning makes feel like a swindler sometimes, but you gotta adapt and overcome.
Starting with a very simple chess engine like Sunfish is a good idea if you want to understand some of the foundational ideas without going too deep.
Thanks for sharing, really interesting!
send me ur engine.
This is very interesting. I hope Nelson or someone else makes a follow up video on this, or maybe you could make one :)
So a benchmark test is usually for testing how quickly the program can calculate something. I wouldn't expect those test cases to have much significance, but they could include edge cases that will exercise unusual code paths.
As others have said, I imagine that the positions are there for some sort of testing. They're not positions for which Stockfish has a pre-programmed answer, instead they can be used to see how well the program is working.
You seemed confused commentating on two of the positions. At 15:50 you seemed surprised that the black king was being forced towards its pawn. If white can't win with a king and two knights against a bare king, then it can't win by having one knight block a pawn and the other knight and king dance around the black king in an unrelated part of the board. You need the black king close to the black pawn to provide some sort of blocking that the white knights can't do alone.
Also you seemed puzzled that the three bishops at 17:27 was a draw. The point is that the bishops are all on black squares. If black keeps his king on white squares then white can never put him in check. As long as he keeps his king towards the middle of the board (so that the white king can't block off all the white squares) he can keep going forever.
Because I'm a C++ junkie, I decided to look up Stockfish's code for the fun of it.
What happens with "fens" is that it's just a temporary variable used inside the function "setup_bench()". This function returns a list of fen positions that gets used in a file called "uci.cpp". What this particular function seems to be doing is getting a list of fen positions that can be sourced from an external file, or use the default positions (hence, "Defaults") from Stockfish's own source code.
And now that you got me looking at this, evaluate.cpp is an interesting read even though apparently Stockfish doesn't even use it all the time to evaluate positions. Instead, it uses NNUE Evaluate to evaluate positions most of the time. Stockfish's code claims that NNUE evaluate is more powerful, but when I glanced at it, it seemed less interesting for a human to read.
'push_back' is a method of std::vector (see vector<string> fens, list; on line 113) that appends the string to the end.
Douglas, beyond My paygrade-but I appreciate the input.
Looks like you could use some help from someone who knows C++ pretty well. 🙂
You hit maybe the least interesting file when it comes to understanding what stockfish does. All it does is performance testing.
Also GitHub search sucks since it sometimes misses things. In this case fens is used in a for loop at line 156. The loop does only a bit of processing to construct a new list of commands to be processed by the caller.
Edit: material.cpp looks interesting, I suggest looking at it first.
Those endgames are quite interesting and definitely deserve a video. The two knights one is not too complicated, and all the key themes can be explained in 1 video. I would probably start with the longest checkmate which is over 100 moves I believe. It transposes into the position that you gave.
That file looks like it's for checking the engine can correctly generate all legal moves in weird but known positions, ones that commonly trip up devs when creating an engine
eg. the position at 6:00 is for testing the combination of checks, double pawn moves and en passant
White can play e4, then normally black should be able to take it en passant with fxe3, but then that would put black into check so it's illegal
en passant is a forced move so e4 would be a stalemate
This was really fascinating, quite interesting getting a better idea of how stockfish works.
Interesting video. Stockfish code puzzles are fascinating.
and FEN code makes sense now.
I first encountered Forsyth notation in one of my dad's old notebooks from 60 years ago.
for you young whippersnappers, there were no public computers in those days
Super interesting. More please!
The file with the "random positions" is called benchmark.cpp. Those positions are obviously used for benchmarking, i.e. testing if the engine can find the solution.
They are not used for testing if engine can find a solution or not.
They are used to calculate bench. Bench is a number that is basically "number of total nodes searched that is needed to calculate this positions to given bench with given hash". Default parameters are 64 MB hash and 13 depth.
This has one single use - every attempt to improve stockfish uses this number as a "hash sum" - if it's not stable then it means smth is not initialised so patch is bugged, and if it doesn't change patch is (most likely) not doing anything.
No one really cares what stockfish really outputs in this positions - it's just used for this and only this.
@@user-mt2jm7vy6i Yeah OK, thanks. I don't know the details for Stockfish. Benchmarking is a commonly used term in a lot of disciplines and it's relatively close to "testing" in meaning.
@@elmiraguth Not necessarily testing in general but testing performance is what it means
These positions have absolutely no significance whatsoever. A benchmark is a piece of code that runs another piece of code, repeatedly and probably with different parameters, to test that code's performance. You just dug through their test-/benchmark cases and kept asking yourself what the significance of these positions is - there is none. You could maybe have learned something by looking at the actual benchmark code instead of the input data. But if you really want to learn how a piece of software works, you're usually better off looking at test code than benchmark code.
to be fair, the benchmark positions might have some sort of significance in the sense of have been chosen for a reason, but yeah, that may not correlate much with "being significant to play chess well"
Those benchmark positions are just random positions for running benchmarks to test the program, lol. There's nothing special about them. The reason it stored that "bad" move is just to test the UCI command position fen moves . It's just for testing.
Great video as always
I want to point out that "syzygy" is a word and should be pronounced as one, rather than the individual letters. It's meaning is an alignment between three celestial objects.
Syzygy (pronounced siz-a-gee) is also a term in poetry and rhetoric that means when the sound at the end of one word is the same sound as the next. For example, “Lizard dreams.”
Thanks, I was just about to write the same thing. By the way, it has a wider meaning than just the alignment of celestial objects.
Feels so weird to have you go over Stockfish's Github page. Never expected to see this day.
There is a bug in github search. fens variable used later in the same file to generate a list of commands to run benchmarks. I doubt that these positions are used inside of Stockfish for evaluating next steps, it's just random position that stockfish is using for measuring performance of the Stockfish.
Maybe it is not a bug, but a missleading feature. It shows the file count and the first occurences per file.
The fact that just a king and a knight can force the king to the other corner is just crazy.
Also, isn't the basis of Stockfish neural networks? Those are notoriously difficult (if not impossible) to understand.
not the basis, but some parts of it uses it (i think the NNUE is the part with it)
@@schwingedeshaehers The NN stands for neural network so yes
Not really. Stockfish only started using neural networks very recently, and you can still use it without a neural network.
Can't wait for part 2.discovering stockfish's secrets
Nelson wasn't cheating; merely a "peek"(or poke) behind the curtains.
@@robertakerman3570 And I didn't say he's cheating
Yeah, dissecting evaluate() would be actually fun
@@dilshankavinda9911 Correct Dilshan. Nels said that He felt that way. I apologize. The peek/poke were very old computer language terms, so "ha-ha". I made a poor joke. Cheers
Yay, that's the thumbnail I voted for!
22:07 It says Rookh1 is a draw for some reason, which is quite fascinating, as Rh1 gives up a full rook.
lol he wants to learn from stockfish but didnt open the evaluation file and opened the benchmark file and wonders what those testing positions are.
I found that "Bench is a number that is basically "number of total nodes searched that is needed to calculate this positions to given bench with given hash". Default parameters are 64 MB hash and 13 depth."
could you make a video explaining stockfish code base ?
I found out trying a random fen in one of those that one fore some reason on board editor started on move 18 rather than move 1.
Hacking stockfish speaks for itself
and... what does that say?
Is the FEN positions things that the direct code of stockfish cant compute and figure out fully by itself but the positions were known and figured out by people already?
No.
They're random positions for testing purposes.
@chessvibes these positions better be a puzzle video in the near future so I can pretend to know the solutions.
You literally spent the entire video discussing random positions meant for testing.
16:00 these speed clones bruh
i didnt knew tuna can ride horses like that these positions were for sure a thing
Where’s part 2?
That first idea you showed is book.
"I can't notice a thing"
-non coder
HAHAHAHAHA i love the duck chess board in the background
3 bishops or even 10 bishops on the same tile colour cant make checkate. The besg thing u get is fotcing the king into a orner and then he has a stalemate.
Not sure I would call any of this hacking. Pretty much its intended use...
Poked around source code files is just that. Hack would require changing a program's behavior. Driving clicks is a kind of hack though.
Stockfish is super weird. It stalls the random positions. Why
wheres part 2
new series - stockfish puzzles
I didn't think you'd actually go through the code
He basically didn’t though. He just looked at some random benchmark positions. Honestly he picked the most boring file to look at. evaluate.cpp would be a lot more interesting and would actually have code that shows how stockfish ‘thinks’.
Tbh the game in the intro was pretty obvious to sac your knight
he opens up a random text file that isnt even code for stockfish
19:34 this could make for a black to play and draw puzzle
I voted for the other thumbnail
Who else noticed he has a game of duck chess on the screen
Try searching for the comments. I mean they put 5 man position mate nd4 thing as comments. If searching comments are possible then why not
part 2?
Did you get part 2?
@@LeaderTerachad what
Very interesting
Haha! "Why is it storing the starting position?" Where else to start from in a standard game??
how can you be a Nm and not know the draw with a pawn and a rook
Because you don't have to know the draw with a pawn and a rook to be an NM.
I tried the same thing before.it's just weird😵💫
20:00 shouldve shown how to draw that..
Nice to meet you buddy we're on the same page
It's just nice to be a programmer and a chess player it's just 8640 IQ
Stockfish is super human
Github is bad for browsing. Apart from the file name you have no clue what it is all about. The text besides a file name is the description of the change the file is lastly changed. This change could be unsubstantial. Browsing is only useful for people thoroughly familiar with the code and knowby heart the purpose of each file.
Wait, is stockfish stealing our data on chess? 😡💀
These are just test cases...
21:36
why dont we just learn the same stuff stockfish knows
Pro position
Real stock fish beat him
👍
hi Qh4 is mate in 1 on 26:00
You must be smarter than stockfish, then.
No no knight will block
well there is a bot that beat stockfish15 ask GothamChess or Levy
Hidden gem :d
Those fens are the puzzles that stockfish gives on lichess!!!
cap alpha zero is better
Misleading title and extremely disappointing material.
Any half-decent follow-up on this?
If there are 1000 Nelson fans, I am one. If there are 100, I am one. If there are only 10, I am one. If there are no Nelson fans, I'm dead.
Are you Hungarian?
Sounds like a generated response 💀
@@mr.cynical520 yes
@@gkristof6504 az úgy király
oh my
☠
Did you get in trouble???
please pin me ❤️
now we see it
stockfish is NOT a chess engine
actually, it is an alien intelligence, sent to conquer this planet
Umm...but I thought Alpha Zero was the best engine in chess...
Stockfish is the best _open-source_ engine.
Alpha zero isn't a chess engine it's a research project and they cheated in the stockfish matches by giving alpha zero a stronger computer. It was just a proof of concept.
Leela Chess 0 tries to implement Alpha Zero's technique but is currently slightly worse than stockfish
Looking you be amazed at everything while having no idea about programming is fun to watch, but your title is widely misleading.
alpha zero is stronger than stockfish
Nope , stock fish 15 is much stronger than alpha zero
Stockfish, as opposed to alpha zero, exists and you can use it
Nelson do you love me
My name is Roberta and I'm with your sister right now driving to ashville!
Alpha zero is beter
Yikes, you are like that starwars guy