I'm just glad to see someone finally tackling USB. For too long, everyone's basically assumed that the hobbyist tinkerer is permanently limited to only understanding VGA and PS/2, even if no one else actually uses either anymore.
The osdev wiki opens saying it's extremely complicated. But it doesn't look too bad if you just handle the easy paths. (Or maybe it's all xhci making it hard)
Its definitely possible for any hobbyist, just depends if they want to spend a month reading documents and datasheets. Or you can simply use the Avr LUFA libraries, or the Atmel ones for their arm mcus, then all you really need to learn is how to make your own report descriptors(for HID), which you can easily just change the examples. For non HID, you can pretty easily write vendor drivers with libusb, start with a simple driver that turns an LED on and off. Or if you feel brave, can get the WDK(windows driver kit) for VS community, and write a driver without libusb, which is next on my list of things to learn.
@@trissylegs If I look at this video, I wouldn't say it's "complicated". It may be a complex protocol, but it's all built up logically as a layered stack, making it fairly straight-forward if you understand the principles of each individual layer. Which is understandable given that it's supposed to be implemented by computers, not humans, and computers like stuff that is straight-forward.
@@LittleRainGames The biggest hurdle for hobbyists trying to implement USB, is getting a vendor and device descriptor. You are strictly not allowed to release devices which use "for development" descriptors or re-use other people's vendor/device ID's, and the only way to get a registered descriptor set is by paying a lot of cash to the USB consortium.
And that's what Windows means when it says "Failed to get device descriptior". Ben, you are to be congratulated on your entire series of videos. The clarity with which you explain some extremely complex concepts in computing, networking and communication protocols is remarkable. To make a complex process like this understandable in just over half an hour is some considerable achievement.
Ben is to be congratulated for sure, for the mind boggling detail which he's managed to get his head around. As for, "To make a complex process like this understandable ..." What? Seriously? ... you understood this? Honestly, I fell asleep after 5 minutes. When I awoke, pretty much the only thing I 'understood' was that this is WAY more 'complex' than it needs to be. It would seem that these specs were created, not by engineers, but by bureaucrats. And now Ben is teasing us with the 'n-key rollover' keyboard, which is even more complicated.
@@CZcamsBorkedMyOldHandle_why I understood it only because I've dabbled with the Arduino USB host library to get at the USB endpoints. The USB negotiation process is a little daunting but there's a good reason for all the to-ing and fro-ing between host and device. There's actually a lot more to it like the device declaring its power requirements along with the descriptor, vendor ID, product ID but that's as deep as I looked into it.
@@CZcamsBorkedMyOldHandle_why If you ever take a university degree you'll learn in Psychology 101 your statement, here above, is a clear declaration you realise don't understand complex concepts, even if and when they're described in simple, layman's terms. Meaning, you know you have a simple mind, you'll forever be stuck in a menial job, and you'll never achieve greatness. You'll forever be an internet troll because you're tunnel-vision-fixed on picking apart perfectly logical information simply because you don't have the requisite foundational background knowledge to appreciate what is being simply explained from a knowledgeable source. Enjoy flipping burgers, it's probably the only thing at which you'll ever excel.
23:26 There's a good reason for requiring explicit selection of the configuration even if there's only one: an unconfigured non-suspended* device is not allowed to draw more than 100 mA. So, if a bus-powered device requires more than 100 mA to function normally (this is called a "high-power bus-powered function"), it will have to keep part of its functionality switched off to limit its power consumption below 100 mA until the host has selected the configuration which declares this higher power requirement in its configuration descriptor. * A device which is suspended may draw only 0.5 mA, unless it is in a high-power configuration and remote wakeup is enabled in which case it may draw 2.5 mA.
Whenever I watch Ben's videos I appreciate just how complicated modern operating systems and hardware is. We're all sitting on the shoulders of giants.
Until USB C solved the problem, all USB cables are in a superposition and require being flipped over three times to get it in right. The fact he gets it in two shows a level of skill even physicists cannot demonstrate regularly. USB C is still in superposition, but it negotiates which pins are tx and rx internally, so we dont actually perceive them arguing on a quantum scale whether its right way around or not :P
thanks to another techchannel i now know the correct way for usb plugs is suppose to be: with the usb logo on top. To bad the thumbdrive has no usb logo...
@@MrBluemoon74 The correct way for a usb plug is always opposite of the first way you try to put it in. And I do mean always. Also many usb ports are mounted sideways, so there is no "on top".
What I find really interesting seeing this is how old and yet advanced the protocol is. I remember in 1997 when I got my P55T2P4 board with both PCI and USB and now 24 years later I realize how utterly complex the shift from PS2 to USB was on a protocol level and how many chip designers and firmware engineers that have had to deal with the protocol... Very nice video Ben (as always)
@@supernovahm1178 You're either trollin or have no idea about what you're talking. How can u think its simple? Using USB for attaching a device to your project is extemely complicated. If it's I²C or UART or 1-Wire or SPI, it's quite simple but with USB you'll have difficuly even attaching a simple numpad. Please enlighten us as compared to what other method do you find USB to be simple?
@@supernovahm1178 If you can easily and cheaply use USB instead of when you can use any of the other serial protocals i've mentioned, you're a lot smarter than any other person I've known in the last 20 years. I thought my question had the answer in it already but apparently you know better. You should make a video explaing why USB is a simpler choice than any of the other serial protocals I've mentioned. The question is not why it is complex, the question is why would anyone use USB when there are many simpler and cheaper serial protocols. If you have no experience with any of the other serial protocols, which obviously you do not, then you don't know what you're talking about. We are obviously not talking about attaching your usb keyboard to your windows machine :-)
@@supernovahm1178 USB is not even remotely simple when compared with PS/2 or RS232. It was quite complex for it's time. Just because you can #include a library doesn't make it simple. it means all the hard work is already done for you.
Showing the printout of the conversation: Ben: I won’t go through it in excruciating detail … Me: oh no, please, please do!! Ben continues to go through it in excruiciating detail Me (smiling and looking to the sky): Thank YOU! Excellent video, can hardly wait for the next part!
Hm. With a bit of external logic like shift latches I see no problem to implement a low speed USB on 6502. I guess the most challenging part is clock syncing. But if you are a USB host then I understand that it is you who defines the clock so that is no longer an issue.
While some of this might seem ridiculous and jumping through hoops, I can only assume it's to ensure compatibility. Why do you ask for 2 bytes just to get the length, then ask the same command again with a different length? Only because devices might have SO much or so little info that you can't keep it at a certain size. When it only has one configuration mode why does it still set the mode the device should use? Because everything is set up so it's crystal clear what mode it SHOULD use. When you think about just how long USB has been around, and how many devices use it, you can imagine just how many edge cases they've had to deal with. What if a device has multiple configuration modes, but some aren't compatible with the host? Well, then the host wants to make sure it's using the compatible mode, in case the device assumed otherwise. What if two devices are using slightly different versions of USB? You always want them to be backwards compatible, so anything you add in the future is put ON TOP of the current spec. Therefore, you get a lot of stuff that seems needlessly complicated and long-winded, when the reason they're doing it that way is because early USB devices STILL function on modern systems, and a modern USB system should have SOME functionality on an old system. The fact that works is honestly amazing, and it's because they're so specific in their communication, even if it means repeating what should seem obvious.
That's rather zany/clever that the same command (06 02 03 GET_DESCRIPTOR String N) works. Since all of the Descriptor structs always starts with 1 byte bLength and 1 byte bDescriptorType, GET_DESCRIPTOR len=2 always gives you the "head" of the structure. So it seems the standard way to get variable length structs is to get the head first, get the size, then get the N bytes of the struct. It's weird though that wLength is a uint16 but descriptors seem to have a max size of 255 (less 2 for the head), with no obvious way to get longer ones (the length isn't a bit-packed varint). GET/SET DESCRIPTOR is the only Setup request with variable length (all the rest are 0, 1, or 2), so nothing actually ever uses the wLength high byte. Maybe they wanted to leave the door open to bigger requests with entirely new bRequest types?
Yeah, a lot of it is to ensure compatibility. There's certain minimum-capability modes that are defined that are designed so that things like the system BIOS can work with USB keyboards and mice. I forget the specific term they use (or, more likely, repressed it), but it's all there in the spec.
It is not only way to do that. An I am acually wonder why they did not have a way for device to just send a blob with all the description info encoded in something like ASN.1 or Protobuffers. That would simplify devices also. It would only need, maybe, some special case handling only for large descriptors (say, >5Kb).
I doubt you can find all of them on one keyboard. The off-hook, ringing and mute LEDs are often found on the small "keyboards" inside the cable of an USB headset (that have volume up, volume down, mute and accept call buttons).
@@OrangeC7 I'm not so sure. Japanese qwerty keyboards include a special key to disable turning input into kana. I personally don't know but that makes more sense to me.
I never realised that I wanted to know in excruciating detail what goes on when a USB keyboard is plugged into a port. But somehow this is truly captivating. Ben really has a gift for taking complex technical details and breaking them down into easy to understand bits, while at the same time not dumbing anything down to point where the explanation is totally useless.
Whew!!! Brings back memories when I was helping develop specialized SCSI-2 devices. Lots of structures that hand-shake 'who are you?', 'What can you support?', 'What sort of speeds and data sizes do you support?' and more. And of course part of that is when the host 'learns' who you are, it goes off to find a suitable software driver that 'understands' all this.
This was absolutely fascinating. I was never a fan of USB when it came out, but man, this protocol/standard has legs! The people who designed it were thinking waaaaay ahead of what was in use at the time. Which is sadly lacking a lot of time in standards. And probably goes a long way to explaining why things like firewire and the like never really caught on. Flexibility is key in a "universal" interface. Looking forward to more breakdowns of this.
If they were thinking far enough ahead then maybe we wouldn't have umpteen different types of USB connector today. Firewire was more flexible in some ways, like it didn't assume that on one end of the wire was the host and on the other end was the device, a mistake that USB has had to backtrack on since.
@@gdclemo Well ultimately "device" and "host" are just labels, they dont really mean anything, as long as you know where the packets are being sent from. USB hubs just add on endpoints for all the devices, and route the individual packets without colliding, whether you can hook up two usb storage devices to a usb hub and tell a computer to transfer files between them without those files going to the computer's ram i have no idea, but ultimately it doesnt actually matter if you call one host and the rest devices.
I mean, considering we are still using USB protocol in USB C, which has more wires and more current options, so any USB C device can hook up to any usb device ever and still negotiate something, whether it be input from a keyboard or output of storage space, yeah, they really did a good job with the USB standards. I know that 2.0 and 3.0 added more than just speed, but thats the main difference most people think about, and considering that that is the biggest reason to increase the revision of USB, imagine if USB 1.0 had included space for increased data transfer speeds for future proofing. talk about universal, huh?
A video like this for the PCIe bus would be really cool! I have a couple of sabrelite boards connected across PCIe with jumper cable to decode PCIe (custom Linux driver). It works! I am studying the roles such as root complex, switch, direct memory transfers... would be good if a video existed on this topic similar to how you break down USB... food 4 thought. Really enjoy these videos!!!
I'd like to see similar videos about ISA, PCI, and AGP first. I feel like going straight to PCIe is jumping into the deep end. We did, after all, get one about the AT keyboard before this.
30:40 "This byte is reserved for OEM use. The BIOS should ignore this field if it is not used. Returning zeros in unused fields is recommended." - HID specification, Appendix B
@@gargaj Normally the device defines the report spec, however since full HID support is fairly complex there are also two predefined "boot device" protocols (one for keyboards, one for mice) that a HID device may support specifically for very limited host implementations such as a BIOS (which is why the sentence I quoted mentions BIOS). Such HID devices identify themselves through their interface descriptor (subclass=1 (boot device) and protocol = 1 (keyboard) or 2 (mouse)) and are required to use the corresponding report format defined in Appendix B until the host uses SET_PROTOCOL to switch from "boot protocol" (use predefined report format) to "report protocol" (i.e. use the report format defined by the HID descriptor). The BIOS will simply recognize compatible devices based on interface descriptor and simply assume the predefined report format is used, they never invoke SET_PROTOCOL nor query or parse the HID descriptor. Cheap keyboards don't bother implementing two report formats and instead just use the boot keyboard report format as their only report format.
Fwiw, D+ being pulled high indicates a _full_ speed (12MBits) device, which is distinct from _high_ speed (480Mbits). High speed devices initially register as full speed, then later negotiate a speed upgrade over the data channel
I can think of a meta-meta, where one uses a USB oscilloscope to analyze USB keyboard. And if you want the meta-meta-meta, run such oscilloscope on one PC, and via USB network transfer the captured file to analyzing PC ;)
Its explaining a high level concept in low level speak, but assumes you arent actually stupid, so ultimately its really high level explained to medium level.
This is how the rubber meets the road in computing (or how the angry pixies meet the thinking sand)... It’s something I’ve wondered about for a really long time before I found Ben’s videos, and it’s completely fascinating to me.
I'm in absolute awe and love with every video I've seen on your channel! I've always been mystified by a lot of what goes on with hardware and even started programming as a hobby to try to understand software at a higher level so I can't begin to describe how appreciative to have come across your channel and see the videos you put out - they're always incredibly interesting and informative! Again, thank you for all the great content you put out! I had no idea so much went into the USB spec and data transfers
Ben: Janet? Janet: *poof* Hi Ben! Ben: I need a black void, a ton of breadboards, infinite small wires, and every known chip with documentation Janet: Of course!
I don't know how you do it but you are such good teacher. I never ever thought about looking into the USB specs because i was a bit scared of it. You took my fear. Thanks ;)
Thanks Ben! I have wanted to learn this for decades. A UNIVERSAL Serial Bus obviously must have an elaborate negotiation protocol. I am also curious regarding hub protocols; adding a USB device must not disrupt in-progress communication and USB addressing must transit specific hub(s). Others are probably interested in USB power protocols.
I'd expect a hub to give a periodic report to the upstream host about each individual port -- whether or not there is a device connected, which address it has, the USB speed mode, and what power it consumes. So if a device is suddenly plugged in, the computer can detect that a previously unused port now has something attached with address 0, and start discovering it, while the other devices on that bus are unaffected. When the computer starts detecting devices, hub first and then downstream devices (which could also be hubs), it can start building up a tree of individual addresses, and each hub keeps track of which addresses are downstream of its ports, similar to the switching table in an Ethernet switch.
I’d love to know if it’s feasible to ‘hack’ a keyboard to change the device description to “Grandma’s Toaster” or “A Very Small Rock”. Would be a fun prank to pull on a borrowed keyboard :)
geez... my first thought is that this is way overcomplicated... but seeing the success of the USB protocol and how well things work, I must accept that there were good reasons for these complications. Thanks for the great explanation.
I love that you do all that tedious work just to show fx how the data looks and what it means. I mean, I knew that that data must be shared and some kind of descriptor is used, but i had no idea it was that complicated for a simple keyboard. Knowing is power, so thank you Ben!
I knew the video was 36 minutes long before starting to watch. But when you said you'd show the other keyboard in the next video I thought "NO... I want to see it now!!" and "no way this were 36 minutes!".
At 22:04, you say the keyboard is polled every 18ms, but it says 18 in hex, so shouldn't that be 24ms? Later in the video you do mention 24ms correctly though.
I've always been under the impression that the USB stack is incredibly complicated. But at least so far it looks benign. I do find it bizarre that the PC tells the keyboard how long a response it expects, though.
They obviously must agree on data lengths somehow. Since the computer (host) initiates all contacts, it is most convenient to specify the length at that time. You might also note that the device replies with data lengths for the requested information, which the computer uses in the next request.
@@byronwatkins2565 The usual way is to send all the data you need and just indicate the total size in the header. This also saves one round-trip and having to resend a portion again. I guess there might be a case for peripherals with very small memory, but seriously, it's not that much data.
I bet that if the PC tells the keyboard to send more keys than it's electronics allow, the rest of the data will just be blank. If, for example, a keyboard can only have 4 key rollover, then requesting 8 key rollover will not magically enable the keyboard to do that many rollovers. I'm betting the device driver sets a lot of these settings, because the manufacturer can set up what the device can support.
@@ChristieNel I think this let's the host have full control of scheduling. Imagine I have a real time device with strict timing requirements. If a keyboard can decide that it wants to give me 150 bytes this time, that variability may mean that I could not have it on the same USB bus. Or I might have to always leave space for its longest possible response. But if the protocol is designed where the host is in control, things can be much more rigidly controlled in systems which need it.
@@ChristieNel How many bytes in the header do you need for the total size? Keep in mind that there are TB drives that might be at the other end. "The usual way" is not "universal."
Just explain to your wife that someday when our Sun expands it is going to consume the entire Earth so it really doesn't matter what any of us do here.
Loving the USB Keyboard videos. And I think it would be really cool to have a video showing lots of devices hooked up to the oscilloscope showing how they work, eg. a mouse etc... Great work though Ben, keep it up 👍
USB protocol setup is performed by software, from the computer's perspective, so it can be viewed with any USB sniffer tool. Wireshark can do it. It will show all packets transmitted. Yes, it will not show stuff like the speed negotiation in the beginning, with setting the data line low or high, or any clock-related signals, which the oscilloscope displays. But for such large message exchange, it looks easier.
@@gideonmaxmerling204 the oscilloscope is tilted prior to this, if you're focusing on it for too long it looks like the screen is tilted in the opposite direction
@@QueueTeePies I suppose it could be BCD where each nibble (4bits) only represent 0..9 - now I got to go look up the USB spec - Although I'm assuming that the polling period would be obvious on the scope.
Thanks for breaking this down in your usual, thorough, yet easy to understand manner. I've been playing with Wireshark USB packet capture and WebUSB/WebHID and this has been a super informative series. Love your videos.
Man. There’s gotta be some exploit out there where you plug in a usb and it just takes over your computer with some malformed negotiation communication. Crazy.
I really appreciate this. As someone who's dealt with protocol specs before (though not USB), I really appreciate the time you've taken to not only clearly explain this one, but also to decode everything. Those tables can be monstrous and even when working with them every day trying to remember where everything is can be a challenge.
You do an amazing job of explaining these things, I really appreciate your videos and am actively trying to get my early teen nephew into them and will buy your kits. Thanks for passing on your knowledge in a manner that is very easy to follow and understand.
@BenEater Thank you so much for these amazing videos. I (and I am pretty sure many of your viewers) would love to see you interface a USB keyboard with the 6502 computer you built and show the text on a monitor (other than the small LCD Display).
thanks for sharing and explaining USB (HID) Keyboard negotiation. It's like opening black box of USB spec. Explaining using real USB communication and oscilloscope it (and) also decode it, all new level.
It would be interesting to see how much of that setup actually matters. I recently messed around with BLE and it turned out that most applications don't care and just send the data without checking what the device actually responds with. It may be that if Windows detects that it is a keyboard, it just assumes everything and leaves to the manufacturers to make it work
Very well done and explained! It took me much longer to figure all these things out, when writing the hidrd tool ☺ A few comments: it's just "Usage Minimum/Maximum", without "Local", although they're classified as "local" items. Constant inputs/outputs are normally used for padding. I would imagine you can also implement a keyboard without rollover by using one bit per key (like the modifiers are done in this one). Eagerly waiting for the next video, thanks a lot for your work 😁
Something I wonder is: if multiple devices are plugged in at once, how does the setup message not collide with all of the devices? Do the hubs forward the messages to each port in sequence?
You forgot step 0: the host provides 5V to the VBUS line. This is not merely to provide power, it is part of the connection negotiation. Even if a device is self-powered (i.e. it does not draw any power from VBUS), it is still required to wait until VBUS is provided before it is allowed to start signalling its presence by pulling up D+/D− [1], and if VBUS goes away the device must cease pulling up the data lines within 10 seconds [2]. This ensures the device will not (try to) pull up the data lines of an unpowered host for extended amounts of time. Also, a nitpick: at 1:35 you meant low speed (1.5 Mbps) or "full speed" (12 Mbps). High-speed (480 Mbps) devices initially identify as full-speed by pulling up D+ and then (try to) upgrade to high-speed using special signalling during the subsequent reset. [1] "The voltage source on the pull-up resistor must be derived from or controlled by the power supplied on the USB cable such that when VBUS is removed, the pull-up resistor does not supply current on the data line to which it is attached." - USB 2.0 specification, section 7.1.5.1 [2] "When VBUS is removed, the device must remove power from the D+/D− pull-up resistor within 10 seconds."- USB 2.0 specification, section 7.2.1
I hope you have a long term plans to dive deep in USB protocol and make somewhat full overview picture, so that us hobbyists had a chance to make use of it before it gets obsolete.
And there was I thinking it would be complicated (and apparently believing that it was grammatically correct to start a sentence with the word 'and' ). Awesome video, Ben.
If you don't need to inspect the analog waveforms, logic analyzers are just as good, have more channels and are far, far cheaper. I've had good luck with a DSLogic - the software is open source too
7:00 This IN transaction is not empty because wLength is zero but rather because any control write (host-to-device) transfer concludes with an IN transaction to poll whether the command has completed, to which the device responds with an empty DATA1 packet if the command completed successfuly, STALL if it failed, and NAK if it hasn't completed yet (in which case the host will retry the IN). In contrast, the ACK to the SETUP merely acknowledged that the device has successfully _received_ the command and will begin its execution. If wLength were non-zero there would be one or more OUT transactions between the SETUP transaction and the IN transaction. Similarly, every control read (device-to-host) transfer ends with an OUT transaction (after the SETUP transaction and one or more IN transactions) with empty DATA1 to which the device responds with ACK to indicate successful completion. This is probably mostly just done for symmetry, in practice any NAK or STALL response during a control read transfer will be in response to the IN transaction(s) rather than to the final OUT transaction since normally once all data of a control read command has been transferred there's nothing more for the device to do that could fail or require more time.
This series is awesome please continue. I also have a request after the gaming keyboard, please get a usb Bluetooth adapter and pair a keyboard and mouse to it. I'm curios if they communicate "usb" over Bluetooth.
Ben, since you explained the five constant input bits later on in the data packet as being for alignment, I'm guessing the earlier byte constant is the same, for those rare keyboards which might have more than eight modifier keys. They do exist: the legendary MIT space-cadet keyboard for the Lisp Machine has seven (SHIFT, CTRL, META, HYPER, SUPER, TOP, and GREEK/FRONT), doubled on each side of the keyboard for chording with actual characters. That already gets you upto 14 modifiers, so essentially if you had a USB space-cadet, you'd only have only two bits of stuffing there. What's probably throwing us off here is that with the now-common 2*4 modifiers, the modifier field at the beginning just happens to line up on a byte boundary, so that the need for padding towards the more extensive scenario is the easiest to define as a single contant byte field.
I have actually had to use some of the features of the USB setup. Knowing who made the device is of great help in finding a device driver, when the actual physical device either does not have the requisite markings, or they have been worn off. I've had to do this several times for a USB to serial adapter i have. Because whenever i get a new computer for work (which is more often than i'd like), i never remember the chip in the stupid cable, and have to go into the hardware configuration to try and figure out which chip my interface uses. Luckily, there are only 2, and the device drivers seems to be pretty manufacturer agnostic.
Awesome! When you first started discussing the keyboard USB protocol, I thought, "He's going to add keyboard support to the 8502, isn't he?" But now that you're mentioning arbitrary rollover sizes, I get the feeling you're going to be *building* your own USB keyboard 😂
Super interesting and informative... You answered many questions pertaining to USB that I had! Your videos are truly amazing. I just purchased some of the kits from your site and I'm far too excited for them to get here :) Also.... That scope is dream worthy. I finally upgraded to a second hand 4 channel 500mhz Tek scope that I bought from a friend of a friend that he used maybe a dozen times for his small business that has been sitting in his office room for over 7 years. Paid $600 and he wouldn't take a penny more! (Easily $2000-2500++ right now, but he did well in life and is a generous person. He'd probably have just given it to me, but I'm not poor and couldn't have ever accepted that) Even my new massive upgrade, as much as I love it to the point my fiancee gives me weird looks, doesn't even hold a candle to that massive scope of awesomeness you use here! Something to work towards!
Almost poetic to use oscilloscope to analyze and decode data transfer for usb keyboard, save the segments of data in a log on a usb drive, then read all the usb data saved on usb drive over usb on computer… lol. It really is so universal
Loved how you take the time to convert all this to human readable, really helps! Would be cool if one day you could have a video on interfaces or protocols between media (like hard drives) and pc. Hope you're all good!
Been meaning to dig into the USB HID spec for a while now as I've been wanting to do a set of custom USB input devices (8 button left handed trackball and scratch built fight stick being the big two), so this is a fascinating watch for me.
I wonder where you get all of those datasheets, specs, etc. Because the last time i searched "how does the usb protocol work", do you know what were my answers? They were - "if you plug in a usb cable into a usb port, the device turns on and you can use them, examples of such devices are - keyboard, mouse, webcam, joystick blah blah...".
Try searching for "usb protocol pdf" or "usb protocol datasheet" instead. You'll find not only datasheets but sites like beyondlogic with tons of USB implementation information.
I'm just glad to see someone finally tackling USB. For too long, everyone's basically assumed that the hobbyist tinkerer is permanently limited to only understanding VGA and PS/2, even if no one else actually uses either anymore.
The osdev wiki opens saying it's extremely complicated. But it doesn't look too bad if you just handle the easy paths. (Or maybe it's all xhci making it hard)
Its definitely possible for any hobbyist, just depends if they want to spend a month reading documents and datasheets.
Or you can simply use the Avr LUFA libraries, or the Atmel ones for their arm mcus, then all you really need to learn is how to make your own report descriptors(for HID), which you can easily just change the examples.
For non HID, you can pretty easily write vendor drivers with libusb, start with a simple driver that turns an LED on and off. Or if you feel brave, can get the WDK(windows driver kit) for VS community, and write a driver without libusb, which is next on my list of things to learn.
Microchip has also had USB code freely available for their USB capable PIC chips since before they bought Atmel.
@@trissylegs If I look at this video, I wouldn't say it's "complicated". It may be a complex protocol, but it's all built up logically as a layered stack, making it fairly straight-forward if you understand the principles of each individual layer. Which is understandable given that it's supposed to be implemented by computers, not humans, and computers like stuff that is straight-forward.
@@LittleRainGames The biggest hurdle for hobbyists trying to implement USB, is getting a vendor and device descriptor. You are strictly not allowed to release devices which use "for development" descriptors or re-use other people's vendor/device ID's, and the only way to get a registered descriptor set is by paying a lot of cash to the USB consortium.
And that's what Windows means when it says "Failed to get device descriptior".
Ben, you are to be congratulated on your entire series of videos. The clarity with which you explain some extremely complex concepts in computing, networking and communication protocols is remarkable. To make a complex process like this understandable in just over half an hour is some considerable achievement.
We typically tell people to reseat the device, try it in another pc and replace it if it still fails
Ben is to be congratulated for sure, for the mind boggling detail which he's managed to get his head around.
As for, "To make a complex process like this understandable ..." What? Seriously? ... you understood this? Honestly, I fell asleep after 5 minutes. When I awoke, pretty much the only thing I 'understood' was that this is WAY more 'complex' than it needs to be. It would seem that these specs were created, not by engineers, but by bureaucrats.
And now Ben is teasing us with the 'n-key rollover' keyboard, which is even more complicated.
@@CZcamsBorkedMyOldHandle_why I understood it only because I've dabbled with the Arduino USB host library to get at the USB endpoints. The USB negotiation process is a little daunting but there's a good reason for all the to-ing and fro-ing between host and device. There's actually a lot more to it like the device declaring its power requirements along with the descriptor, vendor ID, product ID but that's as deep as I looked into it.
@@CZcamsBorkedMyOldHandle_why If you ever take a university degree you'll learn in Psychology 101 your statement, here above, is a clear declaration you realise don't understand complex concepts, even if and when they're described in simple, layman's terms. Meaning, you know you have a simple mind, you'll forever be stuck in a menial job, and you'll never achieve greatness. You'll forever be an internet troll because you're tunnel-vision-fixed on picking apart perfectly logical information simply because you don't have the requisite foundational background knowledge to appreciate what is being simply explained from a knowledgeable source. Enjoy flipping burgers, it's probably the only thing at which you'll ever excel.
That's technically what it means, but unless you're programming the device, 9 times out of 10 it means the cable is broken.
23:26 There's a good reason for requiring explicit selection of the configuration even if there's only one: an unconfigured non-suspended* device is not allowed to draw more than 100 mA. So, if a bus-powered device requires more than 100 mA to function normally (this is called a "high-power bus-powered function"), it will have to keep part of its functionality switched off to limit its power consumption below 100 mA until the host has selected the configuration which declares this higher power requirement in its configuration descriptor.
* A device which is suspended may draw only 0.5 mA, unless it is in a high-power configuration and remote wakeup is enabled in which case it may draw 2.5 mA.
Whenever I watch Ben's videos I appreciate just how complicated modern operating systems and hardware is. We're all sitting on the shoulders of giants.
and unfortunately most of us take it for granted. to some of us, computers might as well be magic.
Just imagine 1000 years from now or one bilion
@@evanlohning "Sufficiently advanced technology is indistinguishable from magic" - Arthur C. Clarke
@@evanlohning Thanks to Docker Electron and VB before majority of programmers don't even know how computers works, this is computer illiteracy.
And all the credit goes to software developers who fix bugs 🙄
No offense to any software developers, obviously.
It is comforting that even Ben Eater first tries to insert the usb disk upside down.
Until USB C solved the problem, all USB cables are in a superposition and require being flipped over three times to get it in right. The fact he gets it in two shows a level of skill even physicists cannot demonstrate regularly.
USB C is still in superposition, but it negotiates which pins are tx and rx internally, so we dont actually perceive them arguing on a quantum scale whether its right way around or not :P
thanks to another techchannel i now know the correct way for usb plugs is suppose to be: with the usb logo on top.
To bad the thumbdrive has no usb logo...
@@MrBluemoon74 The correct way for a usb plug is always opposite of the first way you try to put it in. And I do mean always.
Also many usb ports are mounted sideways, so there is no "on top".
What I find really interesting seeing this is how old and yet advanced the protocol is. I remember in 1997 when I got my P55T2P4 board with both PCI and USB and now 24 years later I realize how utterly complex the shift from PS2 to USB was on a protocol level and how many chip designers and firmware engineers that have had to deal with the protocol... Very nice video Ben (as always)
@@supernovahm1178 see the vid going through the usb protocol
@@supernovahm1178 You're either trollin or have no idea about what you're talking. How can u think its simple? Using USB for attaching a device to your project is extemely complicated. If it's I²C or UART or 1-Wire or SPI, it's quite simple but with USB you'll have difficuly even attaching a simple numpad. Please enlighten us as compared to what other method do you find USB to be simple?
@@supernovahm1178 Ok, if it's simple explain what ben eater did in this video.
@@supernovahm1178 If you can easily and cheaply use USB instead of when you can use any of the other serial protocals i've mentioned, you're a lot smarter than any other person I've known in the last 20 years. I thought my question had the answer in it already but apparently you know better. You should make a video explaing why USB is a simpler choice than any of the other serial protocals I've mentioned. The question is not why it is complex, the question is why would anyone use USB when there are many simpler and cheaper serial protocols. If you have no experience with any of the other serial protocols, which obviously you do not, then you don't know what you're talking about. We are obviously not talking about attaching your usb keyboard to your windows machine :-)
@@supernovahm1178 USB is not even remotely simple when compared with PS/2 or RS232. It was quite complex for it's time. Just because you can #include a library doesn't make it simple. it means all the hard work is already done for you.
Showing the printout of the conversation:
Ben: I won’t go through it in excruciating detail …
Me: oh no, please, please do!!
Ben continues to go through it in excruiciating detail
Me (smiling and looking to the sky): Thank YOU!
Excellent video, can hardly wait for the next part!
Same
the word "excrutiating".. i don't think it means what you think it means...
I am so glad to know I am not alone in enjoying excruciating detail.
If this leads to usb compatibility on the 6502, I'll be stoked!
thats gonna take a lot of clock cycles.
Hm. With a bit of external logic like shift latches I see no problem to implement a low speed USB on 6502. I guess the most challenging part is clock syncing. But if you are a USB host then I understand that it is you who defines the clock so that is no longer an issue.
That's a lot of cycles
Maybe an external module
Is the 6502 worse than an AVR? I've seen some serious bitbanging USB done on the latter...
While some of this might seem ridiculous and jumping through hoops, I can only assume it's to ensure compatibility. Why do you ask for 2 bytes just to get the length, then ask the same command again with a different length? Only because devices might have SO much or so little info that you can't keep it at a certain size. When it only has one configuration mode why does it still set the mode the device should use? Because everything is set up so it's crystal clear what mode it SHOULD use. When you think about just how long USB has been around, and how many devices use it, you can imagine just how many edge cases they've had to deal with.
What if a device has multiple configuration modes, but some aren't compatible with the host? Well, then the host wants to make sure it's using the compatible mode, in case the device assumed otherwise. What if two devices are using slightly different versions of USB? You always want them to be backwards compatible, so anything you add in the future is put ON TOP of the current spec. Therefore, you get a lot of stuff that seems needlessly complicated and long-winded, when the reason they're doing it that way is because early USB devices STILL function on modern systems, and a modern USB system should have SOME functionality on an old system. The fact that works is honestly amazing, and it's because they're so specific in their communication, even if it means repeating what should seem obvious.
That's rather zany/clever that the same command (06 02 03 GET_DESCRIPTOR String N) works. Since all of the Descriptor structs always starts with 1 byte bLength and 1 byte bDescriptorType, GET_DESCRIPTOR len=2 always gives you the "head" of the structure. So it seems the standard way to get variable length structs is to get the head first, get the size, then get the N bytes of the struct.
It's weird though that wLength is a uint16 but descriptors seem to have a max size of 255 (less 2 for the head), with no obvious way to get longer ones (the length isn't a bit-packed varint). GET/SET DESCRIPTOR is the only Setup request with variable length (all the rest are 0, 1, or 2), so nothing actually ever uses the wLength high byte. Maybe they wanted to leave the door open to bigger requests with entirely new bRequest types?
Yeah, a lot of it is to ensure compatibility. There's certain minimum-capability modes that are defined that are designed so that things like the system BIOS can work with USB keyboards and mice. I forget the specific term they use (or, more likely, repressed it), but it's all there in the spec.
@@jaysonl You could see that in the device descriptor as the "Boot feature" of the keyboard.
@@Stoney3K That sounds about right. Now I just wish there was a "boot feature" for Bluetooth keyboards :)
It is not only way to do that. An I am acually wonder why they did not have a way for device to just send a blob with all the description info encoded in something like ASN.1 or Protobuffers. That would simplify devices also. It would only need, maybe, some special case handling only for large descriptors (say, >5Kb).
Now I want a keyboard with all those possible LEDs put on it - the usual three and Compose Key, Kana, etc.
I don't know, I'd much prefer a Compose LED to a Scroll Lock LED.
The only thing I use my Scroll Lock button for is sending commands to my KVM switch
The Kana light is probably for Japanese keyboards, specifically ones which support typing directly in Japanese
I doubt you can find all of them on one keyboard. The off-hook, ringing and mute LEDs are often found on the small "keyboards" inside the cable of an USB headset (that have volume up, volume down, mute and accept call buttons).
It would look like nuclear plant control room wall lol
@@OrangeC7 I'm not so sure. Japanese qwerty keyboards include a special key to disable turning input into kana. I personally don't know but that makes more sense to me.
"HID Device is a Human Interface Device... Device" that second device is the attention to detail I love. Keep up the great work!
21:24
Imagine the clever guys that designed and developed the Scope with all that intelligence and options.
that's why they cost as much as they do lol
@@kaukospots lol. How we laughed. People who say lol are brain dead.
@@custardtart1312 nice life
@@custardtart1312 lol
I recently gave up on a low-level USB client implementation because the USB spec was so complicated. This video is gold to me rn
I never realised that I wanted to know in excruciating detail what goes on when a USB keyboard is plugged into a port. But somehow this is truly captivating. Ben really has a gift for taking complex technical details and breaking them down into easy to understand bits, while at the same time not dumbing anything down to point where the explanation is totally useless.
You don't know how much this helped me with a project i'm making. Thank. You. So. Much.
Whew!!! Brings back memories when I was helping develop specialized SCSI-2 devices. Lots of structures that hand-shake 'who are you?', 'What can you support?', 'What sort of speeds and data sizes do you support?' and more. And of course part of that is when the host 'learns' who you are, it goes off to find a suitable software driver that 'understands' all this.
This was absolutely fascinating. I was never a fan of USB when it came out, but man, this protocol/standard has legs! The people who designed it were thinking waaaaay ahead of what was in use at the time. Which is sadly lacking a lot of time in standards. And probably goes a long way to explaining why things like firewire and the like never really caught on. Flexibility is key in a "universal" interface.
Looking forward to more breakdowns of this.
If they were thinking far enough ahead then maybe we wouldn't have umpteen different types of USB connector today. Firewire was more flexible in some ways, like it didn't assume that on one end of the wire was the host and on the other end was the device, a mistake that USB has had to backtrack on since.
@@gdclemo Well ultimately "device" and "host" are just labels, they dont really mean anything, as long as you know where the packets are being sent from. USB hubs just add on endpoints for all the devices, and route the individual packets without colliding, whether you can hook up two usb storage devices to a usb hub and tell a computer to transfer files between them without those files going to the computer's ram i have no idea, but ultimately it doesnt actually matter if you call one host and the rest devices.
I mean, considering we are still using USB protocol in USB C, which has more wires and more current options, so any USB C device can hook up to any usb device ever and still negotiate something, whether it be input from a keyboard or output of storage space, yeah, they really did a good job with the USB standards.
I know that 2.0 and 3.0 added more than just speed, but thats the main difference most people think about, and considering that that is the biggest reason to increase the revision of USB, imagine if USB 1.0 had included space for increased data transfer speeds for future proofing.
talk about universal, huh?
As has been said, "One of the great things about standards is . . . there are so many to choose from."
A video like this for the PCIe bus would be really cool! I have a couple of sabrelite boards connected across PCIe with jumper cable to decode PCIe (custom Linux driver). It works! I am studying the roles such as root complex, switch, direct memory transfers... would be good if a video existed on this topic similar to how you break down USB... food 4 thought. Really enjoy these videos!!!
I'd like to see similar videos about ISA, PCI, and AGP first. I feel like going straight to PCIe is jumping into the deep end. We did, after all, get one about the AT keyboard before this.
Ben's treatment of the subject matter makes a person feel like tackling this new subject with confidence. Thanks Ben!
30:40 "This byte is reserved for OEM use. The BIOS should ignore this field if it is not used. Returning zeros in unused fields is recommended." - HID specification, Appendix B
But if the device defines the report spec, could it just not send that byte?
@@gargaj Normally the device defines the report spec, however since full HID support is fairly complex there are also two predefined "boot device" protocols (one for keyboards, one for mice) that a HID device may support specifically for very limited host implementations such as a BIOS (which is why the sentence I quoted mentions BIOS).
Such HID devices identify themselves through their interface descriptor (subclass=1 (boot device) and protocol = 1 (keyboard) or 2 (mouse)) and are required to use the corresponding report format defined in Appendix B until the host uses SET_PROTOCOL to switch from "boot protocol" (use predefined report format) to "report protocol" (i.e. use the report format defined by the HID descriptor). The BIOS will simply recognize compatible devices based on interface descriptor and simply assume the predefined report format is used, they never invoke SET_PROTOCOL nor query or parse the HID descriptor.
Cheap keyboards don't bother implementing two report formats and instead just use the boot keyboard report format as their only report format.
@@gargaj Out of curiosity I checked whether UEFI (specifically tianocore edk2), bloated as it is, implements full HID... nope, boot protocol only.
Ben Eater: **exists*
a kid named "Ben": Oh no I'm in danger
This dude is just pumping out quality content like a maniac. One of my favorite channels.
Fwiw, D+ being pulled high indicates a _full_ speed (12MBits) device, which is distinct from _high_ speed (480Mbits).
High speed devices initially register as full speed, then later negotiate a speed upgrade over the data channel
I m very glad to see u again
12:35 So we are using a USB flash drive to list the USB keyboard setup sequence. Meta.
I can think of a meta-meta, where one uses a USB oscilloscope to analyze USB keyboard.
And if you want the meta-meta-meta, run such oscilloscope on one PC, and via USB network transfer the captured file to analyzing PC ;)
@@ikocheratcr ok what the fuck
@@ikocheratcr for levels of meta not thought possible: use a USB rubber ducky to automate all this while it's being captured.
I'd love to see a similar series on CAN bus protocol.
Watch greatscott's channel
Just spent 300$ on your 8-bit kit!
Love your videos keep it up! :)
This is both high level and low level at the same time...
that's why it's called differential pairs :)
Like Glasgow Central station.
It's Ben's eye view
Its explaining a high level concept in low level speak, but assumes you arent actually stupid, so ultimately its really high level explained to medium level.
This is how the rubber meets the road in computing (or how the angry pixies meet the thinking sand)... It’s something I’ve wondered about for a really long time before I found Ben’s videos, and it’s completely fascinating to me.
The effort Ben has gone into this video is phenomenal. Please Ben; don't stop your style, enthusiasm and production values! Your videos are A+
This Channel twists my brain out by it's simplisity in electronics. Thank you. I hope that it will not end.
I'm in absolute awe and love with every video I've seen on your channel! I've always been mystified by a lot of what goes on with hardware and even started programming as a hobby to try to understand software at a higher level so I can't begin to describe how appreciative to have come across your channel and see the videos you put out - they're always incredibly interesting and informative! Again, thank you for all the great content you put out! I had no idea so much went into the USB spec and data transfers
Amazing breakdown of the process! I'm jealous of that oscilloscope.
Ben: Janet?
Janet: *poof* Hi Ben!
Ben: I need a black void, a ton of breadboards, infinite small wires, and every known chip with documentation
Janet: Of course!
Who is Janet?
@@gideonmaxmerling204 from the TV show "The Good Place"
That’s my girl!
Janet: Not a girl.
@@nickwallette6201
I don't know how you do it but you are such good teacher. I never ever thought about looking into the USB specs because i was a bit scared of it. You took my fear. Thanks ;)
Thanks Ben! I have wanted to learn this for decades. A UNIVERSAL Serial Bus obviously must have an elaborate negotiation protocol. I am also curious regarding hub protocols; adding a USB device must not disrupt in-progress communication and USB addressing must transit specific hub(s). Others are probably interested in USB power protocols.
I'd expect a hub to give a periodic report to the upstream host about each individual port -- whether or not there is a device connected, which address it has, the USB speed mode, and what power it consumes. So if a device is suddenly plugged in, the computer can detect that a previously unused port now has something attached with address 0, and start discovering it, while the other devices on that bus are unaffected.
When the computer starts detecting devices, hub first and then downstream devices (which could also be hubs), it can start building up a tree of individual addresses, and each hub keeps track of which addresses are downstream of its ports, similar to the switching table in an Ethernet switch.
the world needs more of your videos!
I’d love to know if it’s feasible to ‘hack’ a keyboard to change the device description to “Grandma’s Toaster” or “A Very Small Rock”.
Would be a fun prank to pull on a borrowed keyboard :)
I guess you'd have to hack a firmware update.
Windows: "A very small rock is having problems communicating to Windows."
geez... my first thought is that this is way overcomplicated... but seeing the success of the USB protocol and how well things work, I must accept that there were good reasons for these complications. Thanks for the great explanation.
I feel like Ben Eater is secretly the god of computing
Yes really, his explanation is very satisfying..
I love that you do all that tedious work just to show fx how the data looks and what it means.
I mean, I knew that that data must be shared and some kind of descriptor is used, but i had no idea it was that complicated for a simple keyboard.
Knowing is power, so thank you Ben!
I knew the video was 36 minutes long before starting to watch. But when you said you'd show the other keyboard in the next video I thought "NO... I want to see it now!!" and "no way this were 36 minutes!".
Ben didn't sleep since the last video, he read the entire USB spec...
I'm always impressed by your tidy (and informative) notes. Keep up the great work!
At 22:04, you say the keyboard is polled every 18ms, but it says 18 in hex, so shouldn't that be 24ms? Later in the video you do mention 24ms correctly though.
That scope is amazing with all the USB features. Also I had no idea USB was so complex! Great videos.
I've always been under the impression that the USB stack is incredibly complicated. But at least so far it looks benign. I do find it bizarre that the PC tells the keyboard how long a response it expects, though.
They obviously must agree on data lengths somehow. Since the computer (host) initiates all contacts, it is most convenient to specify the length at that time. You might also note that the device replies with data lengths for the requested information, which the computer uses in the next request.
@@byronwatkins2565 The usual way is to send all the data you need and just indicate the total size in the header. This also saves one round-trip and having to resend a portion again. I guess there might be a case for peripherals with very small memory, but seriously, it's not that much data.
I bet that if the PC tells the keyboard to send more keys than it's electronics allow, the rest of the data will just be blank. If, for example, a keyboard can only have 4 key rollover, then requesting 8 key rollover will not magically enable the keyboard to do that many rollovers.
I'm betting the device driver sets a lot of these settings, because the manufacturer can set up what the device can support.
@@ChristieNel I think this let's the host have full control of scheduling. Imagine I have a real time device with strict timing requirements. If a keyboard can decide that it wants to give me 150 bytes this time, that variability may mean that I could not have it on the same USB bus. Or I might have to always leave space for its longest possible response. But if the protocol is designed where the host is in control, things can be much more rigidly controlled in systems which need it.
@@ChristieNel How many bytes in the header do you need for the total size? Keep in mind that there are TB drives that might be at the other end. "The usual way" is not "universal."
Ben, are you going to explain to my wife why I'm not getting anything done this morning? :)
If anyone can explain it Ben can!
Just explain to your wife that someday when our Sun expands it is going to consume the entire Earth so it really doesn't matter what any of us do here.
Your mum is not your wife, basement dweller.
Loving the USB Keyboard videos. And I think it would be really cool to have a video showing lots of devices hooked up to the oscilloscope showing how they work, eg. a mouse etc... Great work though Ben, keep it up 👍
USB protocol setup is performed by software, from the computer's perspective, so it can be viewed with any USB sniffer tool. Wireshark can do it. It will show all packets transmitted. Yes, it will not show stuff like the speed negotiation in the beginning, with setting the data line low or high, or any clock-related signals, which the oscilloscope displays. But for such large message exchange, it looks easier.
Another half an hour well spent, as expected. Thanks Ben.
12:58 took me a while to realize the screen wasn't tilted
why?
@@gideonmaxmerling204 the oscilloscope is tilted prior to this, if you're focusing on it for too long it looks like the screen is tilted in the opposite direction
22:00 "Polled every 18ms" - yet every other number is hex thus shouldn't the 18 indicate polling every 24ms?
Maybe they represent individual digit like 0x1 for 10ms and 0x08 for 8ms.
@@QueueTeePies ive got a whole 0x300000 ping
It looks like it should be 24ms to me as well.
@@QueueTeePies I suppose it could be BCD where each nibble (4bits) only represent 0..9 - now I got to go look up the USB spec - Although I'm assuming that the polling period would be obvious on the scope.
34:30 A-HA! in the end the host tells the keyboard to IDLE for 24ms
Thanks for breaking this down in your usual, thorough, yet easy to understand manner. I've been playing with Wireshark USB packet capture and WebUSB/WebHID and this has been a super informative series. Love your videos.
CZcams Channels like you are valuable, valuable, wonderful and useful
I am keen for the next video - I have that same Das Keyboard (typing on it now)
Man. There’s gotta be some exploit out there where you plug in a usb and it just takes over your computer with some malformed negotiation communication. Crazy.
Well, a rubber ducky attack might be what you are searching for
Thank you for this wonderful description .
Seems simple enough... *brain explodes*
(Also - when writing Braille it is not impossible to reach the key rollover...)
I really appreciate this. As someone who's dealt with protocol specs before (though not USB), I really appreciate the time you've taken to not only clearly explain this one, but also to decode everything. Those tables can be monstrous and even when working with them every day trying to remember where everything is can be a challenge.
thanks for making this kind of videos, i learnt a lot from you than college probably.
Yesterday I’ve seen the usb protocol video (awesome video btw) and now with another interesting video
You do an amazing job of explaining these things, I really appreciate your videos and am actively trying to get my early teen nephew into them and will buy your kits. Thanks for passing on your knowledge in a manner that is very easy to follow and understand.
This is a fantastic video! Only thing is that I have to turn my volume all the way up just to hear it. :P
@BenEater Thank you so much for these amazing videos.
I (and I am pretty sure many of your viewers) would love to see you interface a USB keyboard with the 6502 computer you built and show the text on a monitor (other than the small LCD Display).
thanks for sharing and explaining USB (HID) Keyboard negotiation. It's like opening black box of USB spec. Explaining using real USB communication and oscilloscope it (and) also decode it, all new level.
Has there already been a nod to Jan Axelson’s great series of USB books? If not, there should be. Even the oldest editions are very good reads.
It would be interesting to see how much of that setup actually matters. I recently messed around with BLE and it turned out that most applications don't care and just send the data without checking what the device actually responds with. It may be that if Windows detects that it is a keyboard, it just assumes everything and leaves to the manufacturers to make it work
Very well done and explained! It took me much longer to figure all these things out, when writing the hidrd tool ☺ A few comments: it's just "Usage Minimum/Maximum", without "Local", although they're classified as "local" items. Constant inputs/outputs are normally used for padding. I would imagine you can also implement a keyboard without rollover by using one bit per key (like the modifiers are done in this one). Eagerly waiting for the next video, thanks a lot for your work 😁
Glad to see a new vid, very informative and inspiring! keep it up!
Something I wonder is: if multiple devices are plugged in at once, how does the setup message not collide with all of the devices? Do the hubs forward the messages to each port in sequence?
You forgot step 0: the host provides 5V to the VBUS line. This is not merely to provide power, it is part of the connection negotiation. Even if a device is self-powered (i.e. it does not draw any power from VBUS), it is still required to wait until VBUS is provided before it is allowed to start signalling its presence by pulling up D+/D− [1], and if VBUS goes away the device must cease pulling up the data lines within 10 seconds [2]. This ensures the device will not (try to) pull up the data lines of an unpowered host for extended amounts of time.
Also, a nitpick: at 1:35 you meant low speed (1.5 Mbps) or "full speed" (12 Mbps). High-speed (480 Mbps) devices initially identify as full-speed by pulling up D+ and then (try to) upgrade to high-speed using special signalling during the subsequent reset.
[1] "The voltage source on the pull-up resistor must be derived from or controlled by the power supplied on the USB cable such that when VBUS is removed, the pull-up resistor does not supply current on the data line to which it is attached." - USB 2.0 specification, section 7.1.5.1
[2] "When VBUS is removed, the device must remove power from the D+/D− pull-up resistor within 10 seconds."- USB 2.0 specification, section 7.2.1
I have that exact same keyboard (but the ultimate model). Really looking forward to the next video.
Loving this recent dive into USB - I've implemented a handful of USB devices for hobby projects and still learned a lot from this!
I hope you have a long term plans to dive deep in USB protocol and make somewhat full overview picture, so that us hobbyists had a chance to make use of it before it gets obsolete.
And there was I thinking it would be complicated (and apparently believing that it was grammatically correct to start a sentence with the word 'and' ). Awesome video, Ben.
another epic video with detailed information in one understandable way that few people can achieve! Congrats!
I start to think the whole channel is a well disguised oscilloscope commercial. Damn! I want to buy one right now.
If you don't need to inspect the analog waveforms, logic analyzers are just as good, have more channels and are far, far cheaper. I've had good luck with a DSLogic - the software is open source too
I love these videos so much! It's wonderful to get to see behind the curtain at how devices work.
7:00 This IN transaction is not empty because wLength is zero but rather because any control write (host-to-device) transfer concludes with an IN transaction to poll whether the command has completed, to which the device responds with an empty DATA1 packet if the command completed successfuly, STALL if it failed, and NAK if it hasn't completed yet (in which case the host will retry the IN). In contrast, the ACK to the SETUP merely acknowledged that the device has successfully _received_ the command and will begin its execution. If wLength were non-zero there would be one or more OUT transactions between the SETUP transaction and the IN transaction.
Similarly, every control read (device-to-host) transfer ends with an OUT transaction (after the SETUP transaction and one or more IN transactions) with empty DATA1 to which the device responds with ACK to indicate successful completion. This is probably mostly just done for symmetry, in practice any NAK or STALL response during a control read transfer will be in response to the IN transaction(s) rather than to the final OUT transaction since normally once all data of a control read command has been transferred there's nothing more for the device to do that could fail or require more time.
This series is awesome please continue.
I also have a request after the gaming keyboard, please get a usb Bluetooth adapter and pair a keyboard and mouse to it.
I'm curios if they communicate "usb" over Bluetooth.
I've read other specification for file formats like wav for implementation and this is like it. I'm glad I had experience.
Thanks to your videos I started to enjoy low level computer science, which used to be my nightmare on university. Well done sir, keep it up!
Ben, since you explained the five constant input bits later on in the data packet as being for alignment, I'm guessing the earlier byte constant is the same, for those rare keyboards which might have more than eight modifier keys.
They do exist: the legendary MIT space-cadet keyboard for the Lisp Machine has seven (SHIFT, CTRL, META, HYPER, SUPER, TOP, and GREEK/FRONT), doubled on each side of the keyboard for chording with actual characters. That already gets you upto 14 modifiers, so essentially if you had a USB space-cadet, you'd only have only two bits of stuffing there.
What's probably throwing us off here is that with the now-common 2*4 modifiers, the modifier field at the beginning just happens to line up on a byte boundary, so that the need for padding towards the more extensive scenario is the easiest to define as a single contant byte field.
Your content is ridiculously good. Thanks for all the work!
Whoa, I requested this on your other USB video yesterday, but I thought it might be too high level!
I have actually had to use some of the features of the USB setup. Knowing who made the device is of great help in finding a device driver, when the actual physical device either does not have the requisite markings, or they have been worn off. I've had to do this several times for a USB to serial adapter i have. Because whenever i get a new computer for work (which is more often than i'd like), i never remember the chip in the stupid cable, and have to go into the hardware configuration to try and figure out which chip my interface uses. Luckily, there are only 2, and the device drivers seems to be pretty manufacturer agnostic.
I'm more impressed with the crazy oscilloscope you're using. I haven't done computer engineering for over a decade lol.
Just looked it up and it is 20000$ so there's that...
Thank you for pointing out how little I know about my supposed area of expertise!
Awesome! When you first started discussing the keyboard USB protocol, I thought, "He's going to add keyboard support to the 8502, isn't he?" But now that you're mentioning arbitrary rollover sizes, I get the feeling you're going to be *building* your own USB keyboard 😂
Super interesting and informative... You answered many questions pertaining to USB that I had!
Your videos are truly amazing. I just purchased some of the kits from your site and I'm far too excited for them to get here :)
Also....
That scope is dream worthy. I finally upgraded to a second hand 4 channel 500mhz Tek scope that I bought from a friend of a friend that he used maybe a dozen times for his small business that has been sitting in his office room for over 7 years. Paid $600 and he wouldn't take a penny more! (Easily $2000-2500++ right now, but he did well in life and is a generous person. He'd probably have just given it to me, but I'm not poor and couldn't have ever accepted that) Even my new massive upgrade, as much as I love it to the point my fiancee gives me weird looks, doesn't even hold a candle to that massive scope of awesomeness you use here! Something to work towards!
great video! thats where NKRO keyboard kick in, using one for past 10 years
Almost poetic to use oscilloscope to analyze and decode data transfer for usb keyboard, save the segments of data in a log on a usb drive, then read all the usb data saved on usb drive over usb on computer… lol. It really is so universal
I can't stop watching this.
Loved how you take the time to convert all this to human readable, really helps! Would be cool if one day you could have a video on interfaces or protocols between media (like hard drives) and pc. Hope you're all good!
Been meaning to dig into the USB HID spec for a while now as I've been wanting to do a set of custom USB input devices (8 button left handed trackball and scratch built fight stick being the big two), so this is a fascinating watch for me.
Very clear. Usb host-device protocol is intricate, and everybody documents the device side, but the host perspective is neglected.
21:23, the first 9 B of the response are the same, but they're not, they differ in the 3rd byte.
Man, this is the best youtube eletronic channel
I wonder where you get all of those datasheets, specs, etc. Because the last time i searched "how does the usb protocol work", do you know what were my answers? They were - "if you plug in a usb cable into a usb port, the device turns on and you can use them, examples of such devices are - keyboard, mouse, webcam, joystick blah blah...".
Try searching for "usb protocol pdf" or "usb protocol datasheet" instead. You'll find not only datasheets but sites like beyondlogic with tons of USB implementation information.
Absolutely brilliant video, thank you for all your hard work making it