What is error correction? Hamming codes in hardware
Vložit
- čas přidán 23. 05. 2024
- 3blue1brown's video: • But what are Hamming c...
See eater.net/crc for more error detection and correction stuff.
Support these videos on Patreon: / beneater or eater.net/support for other ways to support.
------------------
Social media:
Website: www.eater.net
Twitter: / ben_eater
Patreon: / beneater
Reddit: / beneater
Special thanks to these supporters for making this video possible:
Aaron Todd, Adrien Friggeri, Alexander Wendland, Andrew Vauter, Anson VanDoren, Anthanasius, Armin Brauns, bapu, Ben Cochran, Ben Dyson, Ben Kamens, Ben Williams, Bill Cooksey, Binh Tran, Bouke Groenescheij, Bradley Pirtle, Brent Reusing, Bryan Brickman, Carlos Ambrozak, Christopher Blackmon, Daniel Jeppsson, Daniel Sackett, Daniel Tang, Dave Burley, Dave Walter, David Brown, David Clark, David Cox, David House, David Sastre Medina, David Turner, Dean Winger, dko, Dmitry Guyvoronsky, Dušan Dželebdžić, Dzevad Trumic, Emilio Mendoza, Eric Brummer, Eric Busalacchi, Eric Dynowski, Eric Twilegar, Erik Broeders, Eugene Bulkin, George Foot, George Miroshnykov, Harry McDow, HaykH, Hidde de Jong, Ian Tait, Ingo Eble, Ivan Sorokin, Jason DeStefano, Jason Specland, JavaXP, Jay Binks, Jayne Gabriele, Jeremy A., Jim Kelly, Jim Knowler, Jim Van Meggelen, Joe Beda, Joe OConnor, Joe Pregracke, Joel Jakobsson, Joel Messerli, Joel Miller, Johannes Lundberg, John Fenwick, John Meade, Jon Dugan, Joshua King, Kefen, Kenneth Christensen, Kent Collins, Koreo, Lambda GPU Workstations, Larry, Lucas Nestor, Lukasz Pacholik, Maksym Zavershynskyi, Marcus Classon, Martin Roth, Mats Fredriksson, Matt Alexander, Matthäus Pawelczyk, melvin2001, Michael Garland, Michael Tedder, Michael Timbrook, Micro Hobbyist, Miguel Ríos, Nicholas Counts, Nicholas Moresco, Örn Arnarson, Paul Pluzhnikov, Paul Randal, Pete Dietl, Phil Dennis, Philip Hofstetter, Randal Masutani, Randy True, Ric King, Richard Wells, Rob Bruno, Robert Diaz, Ryan Trunck, sam raza, Sam Rose, Santiago Bustos, Sergey, SonOfSofaman, Stefan Nesinger, Stefanus Du Toit, Stephen Riley, Stephen Smithstone, Steve Jones, Steve Gorman, Steven Pequeno, TheWebMachine, Tom, Tom Burns, Vlad Goran, Vladimir Kanazir, Warren Miller, xisente, Yee Lam Wan
*Ben Eater + 3b1b = a perfect Friday*
Completely agree
I feel like I've been seeing you everywhere ever since your first biology video.
They're duch a humble, intelligent very interesting human beings
@@jamescollier3 they did, and maybe for a month
Both popped up together in my feed and I had a double take
1ben3eater
3ben1ben
1men1jar
This turned perverse quickly....
@@works4me89 Unfortunatly I have seen that one... and I don't want to be reminded of it... ;-)
blackbenredben
8:50: the upside down dip switch outputs 0 when set to "NO", perfectly logical :D
If "NO" = false = 0, then you have a programming language!
@@OrangeC7 Correction: Then you have PHP or Javascript.
@@ShenLong991 Also C/C++
(older versions especially, where boolean wasn't always a first-class citizen and could be implemented as an integer where 0 = false, and non-zero = true. In fact even with formal implementation of a boolean type that still holds)
@@KuraIthys I mean... that also how it works in python ( www.python.org/dev/peps/pep-0285/ ) and a lot of other languages
Well that works
When the world needed him The most HE CAME BACK
He'd come back a couple days early if you join his Patreon ^.^
@math your math so just add 1000000 to 0 and ull have $1000000
You know what's really underappreciated about your video's? The audio consistency between voiceover during a sped up part, and the normal video where you're just talking to the camera (assuming you do that, the consistency is so good I'm not even sure it's not all a voiceover).
I honestly sometimes have trouble figuring out if I'm watching a sped up video or not, because there literally isn't a change in audio that would indicate it. You're the only channel I've ever watched that accomplishes it, but IMO it makes for a much more comfortable video.
Are you also planning on doing a video on error correction for bigger data, or for multiple errors, if that's even possible?
It definitely sounds like VO to me, but I think he's talking as he doing everything because I think it would be harder to explain what you were doing after the fact vs while you were doing it
He sure is good at explaining the logics and how it all works as well as editing it all together in a really seemless fashion...
I believe he talks as he does things and records on a good setup with a good mic. When he needs to speed up some part, he explains what he is doing while doing the beggining; and speeds up the video portion to fit with the explanation. And then just cuts the rest of the audio. It's an awesome way of doing things, imo.
I noticed it too! Would love to hear from mr Eater how he does it
Check out reed solomon code, it uses finite fields and can correct lots (customly defined amounts) of errors. Very commonly used in server storage and military video transmission (AC130 AH64, and the drones)
I remember learning about Hamming codes when I was at University over 40 years ago. Needless to say, I'd forgotten everything. Very interesting.
Idk how up-to-date your profile picture is, but I’m surprised you’re 60+. You look 40 at most. Whatever you’re doing, keep it up
I really hope this channel keeps growing. It's honestly one of the few channels where I always watch the new video as soon as I can. You break everything down and explain it really well making it really easy to enjoy watching and learning.
As a programmer that dabbles in electronics I have to say I love seeing things done at such a low level. Everything I do is usually so far abstracted from the bare hardware that you really lose the appreciation of what is actually going into each line of code you write. It's kinda sad that most things now are just done in higher level languages when these hardware solutions are so elegant.
Same for me, i appriciate the comforts of high level programming language much more, it is so smart whats going on in the computer, way to smart for me ^^
This scheme can be expanded by adding more "parity bits" to allow correction of 2-bit or 3-bit errors, which brings me to something else which can be (and occasionally has been) done. Distributed storage.
By adding sufficient parity bits, in the right places, it's possible to divide a file into smaller "chunks" which collectively take up around 1.25 times the original storage space, but you only need to retrieve approximately 75% of the "chunks" and you can then calculate all of the missing data by using the parity bits stored with each "chunk" of data you did manage to retrieve.
This means that in a distributed storage scheme, data pertaining to a particular file can be stored across multiple servers and if one or two servers go out of service, you can still retrieve and reconstruct the original file from the remaining servers.
The down-side is that the reconstruction process takes a long time since the mathematics involved is complicated, and making hardware to perform the task would not be economically viable.
For those who didn't know, this is exactly how RAID 3, 4 and 5 disk arrays work, using 25% of the space to store parity data, and RAID 6 doubling the amount of parity data to be able to restore if two drives fail. Also, RAID 2 uses Hamming codes.
@@1loshvitalik oh my that's-a interesting to know
Well, that's cool.
That last sentence is exactly why ZFS is still only used on servers, the overhead is too high to use on a desktop system. Plenty of people have tried, usually with ZFSonLinux, but they usually have to add a lot more RAM, or disable most of the features which make ZFS useful (rendering the exercise pointless).
Are you talking about binary BCH codes?
Always excited when I see you have dropped a new video. I have spent most of my career involved with design and programming of standalone control systems which often were coded in assembly so i'm very familiar with what you are doing yet you still make it entertaining and fun for people who are very familiar and even those that are new to it all, a real talent.
Wow respect sir!!
Whats is your advice? I want to go electronics engineering but for some reason I'm also into compsci
Thank you so much for your videos. Can't explain enough how excited I am to go through every project/kit you've done!
The best way I remember how parity (or at least hamming 7,4) bits work is:
Parity 1 reads 1, skips 1, reads 1, e.t.c., including itself.
For each parity bit, double the amount of reads and skips.
The errored bit is either the total of each wrong parity minus one (for a 'human friendly' explanation) or the binary number represented by the parity bits.
Single bit error correction, multi-bit error detection.
I came here from 3B1B's video, watched this video, subscribed to this channel and binged watched an entire playlist of building a computer based on 65c02 microprocessor; At the end of the 9th video Ben said "You know how to make youtube tell you if I release another video for the continued work'"; and hearing that, I clicked the notifications icon on his channel.
Commendable job Ben! Extremely impressed by the patience you observe while explaining even the hardest concepts. I used to be a bit scared of understanding electronics, leave aside understanding a computer to its machine code. But today, I understood each and every topic of the 5 hours of content I saw. Thank you.
Excellent explanation and demonstration of Hamming codes! I really love the use of color, like different colors for the parity and data lines on the diagram, and different colors LEDs for data, parity and error! And the mask showing which LEDs were involved in each parity-check was simply amazing! Very inspiring teaching-style!
3b1b and ben eater both upload videos on hamming codes at exact same time. this cannot be a coincidence.
ben eater IS 3b1b ( Grant Sanderson )
Debunked, they've been seen together:
czcams.com/video/hKD-lBrZ_Gg/video.html
@@igNights77 ever heard of editing?
Ooo
This is honestly one of your best videos! I love the collaboration with my other favorite channel too! I had no idea you were friends, so cool!
Great to see you going into depth on the Parroty Bits shown in your VGA video. 🦜
I never understood this before today.
You make some amazing content, keep up the good work.
Glad to see you doing such great work with other youtubers!
Ben,
I really enjoy your amazing videos. They really help me in understanding computers and electronics. You can explain everything in such a way that anyone can understand it. Keep up the good work and before you know you are at a million subscribers. Thanks for you amazing contant.
i watched your CRC video a few weeks ago and then i got stand up maths video with the chessboard recommended so i watched that after that there was 3Blue1Brown coming full circle back to you
Thank you for your channel. You bring back fond memories and some cold sweats from the early 1980’s building 8080 computers from scratch that controlled motors and sensors.
Hats off to sir, You are sharing much powerful information about processor and controller in deeply......
Thank you so much for all sir.
Lots of Love from India 🇮🇳
😂😂 Friend, I think you mean: "Hats off". "Heads off" is something the french say to royalty as they lead them to the guillotine.
@@aaron41 TF lol
@@aaron41 😂😂 I fucking laughed so much
@@aaron41 _*led_ 😉
@@aaron41 oh ya, Thanks 😅👍..
it's my typo mistake ...
22:26 beautiful demonstration
I had some interest in your channel, but now that I'm taking computer organization, your channel is going to be one of the most solid resources out there for understanding this stuff. I just had to spend half an hour trying to understand hamming codes. Thanks!
Also please don't eat me.
I'm a veterinarian. I started out in computing, but it just got too heavy for me. Love watching your videos. Almost makes me wish I stuck with computing as my career instead.
wow, i remember thinking two years ago: let's dump everything and become a veterinarian, but i quickly realised that i would be bored to hell
Great video! Very good explanation of Hamming correction code. Actually building the circuit makes for great intuitive understanding.
Another great video! Thanks Ben.
Excellent work!!
In psychology, to get a knowledge that last for ever, you need a sensorial impact. The way you reproduce the logic of the error correction is clever an produce an impact. This concept will last for ever thanks to your way of explaining logic. You should receive the Spock golden prize...
It should also be noted that you can take two more XOR gates and wire them up to the three outputs (yellow LEDs) on the bottom board, and wire the output of that to another LED. That will then tell us if the problem is a parity-bit error (odd # of yellow LEDs on), or a data-bit error (even # of yellow LEDs on)! 😊
Can't hope for any better Collab from two of my favourite youtubers!
Very refreshing to see a full on hardware implementation of hamming. In school I was teached that hamming codes are coordinates in a bit hyperspace, and that if we get an undefined codeword on the line, we can just pick the "closest" word instead. If there is a tie, then we have detected but not corrected the error
Very cool! Cheers, Ben!
Is 'Be nEater' an instruction to your viewers? If it is, you're a good example.
@bob dash :-)
B-inator
@bob dash or be neater
Excellent Ben, thank you...
Very nice, informative, and clever implementation.
Thanks.
Very good explanation, concise. Thanks!
Nice! I did a bachelor’s thesis on FPGA implementation of Iwadari code codec - these are able to correct error bursts by basically introducing a checksum bit after every few bits. I had 16 bit burst correction (provided another error burst did not occur for another 1600). There was a 1 check bit every 4 data bits, and the checksum shift register was 267 bits long. Quite a monster. Did not get to the practical realization, but simulated pretty well.
my first impression of this channel: You seem like the kind of hardware geek (note that's a compliment coming from me) I might've been, or at least been very good friends with, had I not become paralyzed at the end of 2013.
While I was always more familiar with software, in all its abstracted-to-death safety (Love you anyway, Python!), I was uncertain enough about which way I want to go that I had recently invested in a breadboard and a few basic and/or/not logic gate ICs. so it's hard to say...
Oh Ben. I love your content. I've watched your entire series for building a computer on bread boards and your "hello world"too.
This is the first time I've not been able to enjoy one of your videos as I'm red\green colour blind :(
Please may I respectfully request you use either red\blue or blue\ green pairings, if using to differentiate
A bit late here, but can you not use video shaders? Android accessibility settings usually include color blind tone remapping as an OS feature. I'm fairly certain Windows 10 has this built in now too. Worst case, you might try downloading content you can't see correctly and watching it with VLC or any other video player that supports shaders.
@@dx3910 interesting. This is certainly worth trying, just got curiosity sake.
Tbh though, I watched the video anyway as even though the two colours look exactly the same to my eyes, the position and layout isn't effected by my visual impairment (it's the same reason I can deal with traffic lights in the daytime).
I was just about to say that this is a huge effort to make and would require nerd-level technical ability, before I remembered that I've consumed hours of Ben's content involving low-level, on-the-fly programming of microprocessors :)
It was more of a joking way of bringing the issue to light, so in the future it could be considered when choosing led colours for differentiating information
Simultaneous 3b1b and Ben Eater uploads!? I love this
Both about the same thing
this is cool
I wonder whether they did this on purpose or if it's just coincidence..🤔
Edit: Just watch 0:42 where he's talking about iz
I've received the three notifications at the same time, and was wondering if it was really coincidence.
NO WAY I was asking/pleading for this on your previous video!
Love your work mate.
I'm half-watching the video, pausing, thinking about how it could work, then building that in a logic simulator and then unpausing to see if i got it right
this was amazing and blew my mind
the circuit might become too big, but your current implementation is also able to detect errors in the parry bit. As long as it is just one error. Thanks for knowledge :). Keep up the good work!
Had to stop and comment. I know just barely enough to understand what is being explained. I’m blown away by how clever this logic is.
Not an engineer but have to admire brilliance when I see it.
Cool!! I'm actually subscribed to both channels.
This trilogy of videos was the brain food I so badly needed. Thank you.
Let's Just appreciate the hard work that Ben eater do for us , hats off for this great man 💪🏻👌🏻
What's funny about the timing of this video? I just had a block failure on my ssd and am currently running chkdsk /r to fix... First time I've ever had this before. Ben, your timing is impeccable.
A good review of Hamming (7,4) and (8,4). Great video!
Way cool. You rock Ben!
Another great video Ben. Not sure where it came from, but very informative. Does this mean the 6502 project is finished? Or are you still working on that? I was looking forward to adding inputs and making a game which I think you mentioned previously. Or will this tie into that series somehow since you mentioned a software imitation?
this was such a great explanation
Your videos are the real university for me
22:26 to 22:43
If that doesn't sell you on this scheme, i don't know what will. Absolutely beautiful!!
Thank god for PCBs, electronics would still take up entire rooms like they used to. These breadboards really put the size difference into perspective!
I always learn something from every video.
Ben Eater + 3blue1brown = super awesome 🔥🔥
well, now I know what ECC ram is so damn expensive
Ben Eater = 3blue1brown ????? hmmmm
Yeah and they have similar voice
His Asian doppelganger...
@Fred G. Oops. I was thinking of red pen blue pen...
xor(Ben Eater, 3blue1brown) == 0
Eli Eli Lema [xor(Ben Eater, 3blue1brown) == 0] => [Ben Eater == 3blue1brown] aha! I knew it!
You both are spoiling us!
Thank you Ben!
TRCI L2 GTR STHB university Algeria, when i first knew about error correction and message transfers.
Great video.
Love your videos👍
Been watching your vids for about two weeks and enjoyed them. Then i saw your face and thought i know this bloke sure i have met you. If I'm right i live just down the road from your brother (:
Just one word: brilliant!
6:50 - original hamming code
16:07 - error generating ckt
18:20 - computing correct parity bits
27:27 - building back corrected hamming code ckt
You're doing a great job.
Love these videos
I finally found someone who does this kinda stuff on CZcams!
We need more of these colabs
Love the use of a spudger as a pointer
I remember back when you discussed CRC, one of the ways you described it was as a generalization of parity checks. It kind of makes me wonder if you could apply similar logic as the Hamming codes to come up with a CRC-based error correction code that could correct some multi-bit errors.
what a great crossover
Your video are best to understand
No matter how beneath you are, he is Ben Eater
I just started a Digital Circuits class in college, so I get this a little more.
I too find the plastic pokey thing from the iFixit kit a useful tool for flipping DIP switches.
amazing! you're basically using parity groups to conduct a kind of binary search for the erronous bit!
Between this and your ALU’s addition/subtraction switch, you’d think there’d be a market for an XOR gate IC with all of one side internally connected. Call it an XOR buffer if you will. That way you’d be able to flip 8 bits simultaneously with a single DIP20. 8 in, 8 out, 2 power+gnd, 1 toggle, 1 tristate high-z enable or whatever.
Nice explanation, thanks! BTW they should really make a dual quad-input XOR gate as a general purpose 74xx IC!
26:52 I love this so much, that is extremely satisfying
I have a degree in Computer Engineering and this was literally one of the things that blew my mind when first introduced to me. I think it went something like "the odds of a transistor being flipped by random electrons is minuscule, but with billions of transistors, we need to design for minuscule" very cool design
Great video
Great video. Would love to know how you make the wires for your breadboard connections. Seems like it'd be time consuming
Wow i never expected this crossover
On the error corrector board there was an extra xor gate left over on each chip. Those could have been used as inverters by setting one of the inputs high, and you wouldn't have needed the hex inverter! :)
Excellent
thank you😊
excellent video... :)
Oh shit I might actually have the hardware for this! Fun!
Where were you 40 years ago when I was first learning this stuff. "Just Memorize It" was the mantra from my teachers.....
Ben great video, thank you.
Question. Your thoughts on how often digital communications suffer from errors of this sort?
Actually a lot higher than you would think, since the goal is to transmit as fast as possible. Numbers tend to look like bits per million for copper or less than 10e-14 for fiber - about 1 per 100 megabytes.
6:46 lickily a 2 bit error shuld cause the wrong combination of worn parity bits meaning that after the correction the parity check shuld fail a second time
Best crossover from 2020
How does this legend of a youtuber monetise his videos with all this swearing?
Double teaming Hamming codes with 3B1B has got me more excited than I probably should be. Time to make popcorn.
if you want to you can consolidate your BOM a bit by replacing the inverters with xors and one input being tied to vcc
You are my hero.
🤩wow I could learn more about this