everyone should test their code this way
Vložit
- čas přidán 14. 10. 2023
- Did you know you can just SCREAM at your code to find bugs? Yeah seriously it's that easy. In this video we'll talk about libfuzzer, which is a simple to use tool to write code that finds bugs in your code. In this video we write some code, find a bug, and patch it.
🏫 COURSES 🏫 Learn to code in C at lowlevel.academy
📰 NEWSLETTER 📰 Sign up for our newsletter at mailchi.mp/lowlevel/the-low-down
🙌 SUPPORT THE CHANNEL 🙌 Become a Low Level Associate and support the channel at / lowlevellearning
Why Are Switch Statements so FAST? • why are switch stateme...
Why Do Header Files Exist? • why do header files ev...
How Does Return Work? • do you know how "retur...
🔥🔥🔥 SOCIALS 🔥🔥🔥
Low Level Merch!: lowlevel.store/
Follow me on Twitter: / lowleveltweets
Follow me on Twitch: / lowlevellearning
Join me on Discord!: / discord - Věda a technologie
I already yell around 5-10 times a day at my computer
Why is fuzzing better than boundary tests?...after watching I withdraw my question.
I'm on the HDL/Hardware side where something like this is called Constrained Random Verification. Of course we do checks on boundary conditions in directed specific tests, but these devices have 30+ interfaces, so complex interactions can occur. Boundaries cover the 3 cases of too low, too high, or just right data inputs. But what if, say, if condition A AND Condition B And Condition C occur within x milliseconds to error out? A, B, and C are all within bounds, but this specific combination is deadly.
For example, if on a server client 1 is somehow allowed to delete files in use by client 2 via an unsafe delete(file f) function, unless you know exactly how this exploit works you won't make a test for it. Two fake clients banging on a virtual keyboard, however, might find the right inputs over time to crash.
@@adissentingopinion848 I'm sold. I commented too early and using the Int vs unit is what did it, something that could be missed with the range. I have not used a tool like this so is it full path coverage or random? I wonder if the expense for full path doesn't become too high in terms of time....
@@millax-ev6yzIt's probably not going to get full code coverage UNLESS you explicitly get into a specific state for operation first. That "harness" mentioned for interfacing with the code can be very large and very customized. Simulations for hardware are terribly slow, but purely software testing ought to be rather fast up to a point.
In hardware at least, you can set assertions that cover functional requirements such as message format. That way you don't have to error out, just capture the incorrect functionality from the harness itself.
Just for fun though, there's a footgun hidden in the example code, too. As the recv buffer has a hardcoded length limit of 1024 bytes, directly casting the input buffer into a struct that contains a user-controlled length field is not really a good idea. If somehow the codebase got updated in a certain way and the memcpy destination was a heap allocation, it may lead to information leak. E.g. ask the server to echo a 65535-byte data chunk from a 1024-byte input.
Seems like nearly every video I'm warning about magic numbers. He really needs to tighten up his examples.
Well, the quicker way to ensure no crash here is just sanitize the input data. Then probably add a unit test for some edge cases. However - with much more complex example maybe using a fuzzer would be simpler, IDK. But THIS is probably the simplest explanation on how to use a fuzzer to begin with.
Its always these hardcoded buffers that blow up in your face.
@@anon_y_mousse It's on purpose to get engagement from these comments.
@@MrAsddasdasda You may be right because I leave a comment every time just to say something about it.
"like literally yelling at the code" proceeds not to yell at the code
I love this type of videos where you show a useful tool and an example using this tool, and what's even cooler is the fact that using it you were able to detect a bug that wasn't intentional
Use -fsanitize=fuzzer,address and you should be able to find another bug in the parse code. If the input is less than the size of the struct you would read outside the memory. Does not always cause crash without address sanitizer. However not a bug in the program due to the receiving buffer size.
That's why I used to use unsigned everywhere by default, until negative values are explicitly required by design.
And yes, using e.g. -1 magic value to represent things like a non-existent index is a bad design. Don't do it.
what could I do instead for non-existent index??
@@joaquinnapan3237 In rust you would do Option don't know about other languages.
Error-as-types. Like Rust
@@joaquinnapan3237
Rust → Option
C++ → std::optional
C# → Nullable
...
For languages with no option-like concept out of the box, you certainly can come up with something.
E.g. in C you can utilize out parameter for the actual value and return the error code, or vice versa.
Or return something like
struct optional_uint32 { bool has_value; uint32_t value; }
@@gigachad8810in C?
I publish fuzzers. Applied to tech roles for nineteen months without success. Hiring teams are ass.
Shoot me an email
And I wrote a cryptographic library that was "fuzzed" with Python's Hypothesis library. Do hiring teams understand it? Of course not.
Satisfied customer here, been doing this for the last 10 years
10/10 - my code has feared me ever since
I already do this every day
Ah, there's a name for it. I do this regularly the manual way in my own projects, though granted those are all smaller projects where my scope of potential issues is "is there some way a user can force invalid data down this thing's throat". Useful to know if I ever manage to get a real job, lol(being a dev without a college degree is the dark souls of job hunting, I swear)
It would be really funny if he said "there's no more bugs in this code" and libfuzzer just crashed.
Segmentation fault (Core dumped)
why did i think we might actually be yelling at code?
the most reasonable action in the world of C programming
I yell at my code, but it doesn't usually fix any bugs lol
Because you're a fan of slamming desks.
I didn't quite catch why 7:45 is an issue. Would anyone mind please clarifying?
overflow probably, would be my first guess.
It checks if len>64, to prevent writing more than the allocated buffer. But negative numbers are also smaller than 64, so they also pass the check.
The program then crashes in the memcp again, because it tries to copy a negative number of bytes.
@@turun_ambartanen thanks so much!
For an signed number we're using the two's complement to represent negative and positive numbers. Here the MSB decides whether the number is interpreted as an positive or negative number, where 0 = positive and 1 = negative. Looking at 7:45 for example, a hex value of 0xFF is represented in binary with 0b1111_1111. When assigned to a signed variable, this is actually a -1 in decimal. Since we use this variable "len" to access entries in an array, this will result in an error as it doesn't have negative entries to point at.
@@louispetrick thanks so much for the thorough explanation 😁
Amazing brother, you have the gift of communicate complex concepts into simple terms. Thanks! Glad to find your channel! ;)
Well, because I am so good at messing up function calls by using function pointers and structs/unions, I need no help. The code would yell either way nevertheless.
Also, I am obsessed with keeping the memory usage low, so it's likely that I am gonna use a goofy assembly or stuff for my personal performance-intensive stuff. Especially on microcontrollers, but those don't count.
at first we code safely by yelling
in time elaborate rituals involving chanting, holy oils and incense is necessary to please the machine spirit and banish demonic bugs
Interesting, I have no idea this type of testing exists. Thanks man
id love a video of you describing your linux setup. i use wsl and customize very few things but would love more insight into your setup for vim and tmux/whatever multi shell youre using
As always chef's kiss!
“Port 1337” that took me a second. Very funny
Hi, sorry of the OT but I have a Rust/C question nobody was able to point me in the right direction.
With redhook (unmaintained lib) I used LD_PRELOAD to override getenv which worked fine in NodeJs but Rust did not care about it at all. Do you know what is different or what should I read to understand how this all work? Thank you so much
2:38 in, I expect your issue is that you didn't check the length argument in your payload. This should pop up with many static analyzers. But I get it, it's just an example. Fuzzing is more for discovering weird edge cases and undefined behaviors as I understand it. Or I'm totally wrong and length was not the issue :D
I was hoping for Torvalds kind of screaming at someone else code, but I guess this is fine.
0:05 should have been the end lmao
This was awesome.
Fuzz all the things.
Instructions unclear
I’ve been yelling at code this whole time
This is so cool, does something like this also exist in the Java world?
To test for what?
Jazzer does exactly that and is based on this.
While this tool is awesome as is, is there any way to get it into an IDE? I think productivity would go up a lot of you can just select a function and some extension can do all the work for you returning only the result. Maybe I'm overlooking something that'd make you not want to use an extension like that but I think it'd be cool
Flag '-g' makes stack traces of gdb or any sanitizer look pretty. Use it.
Does it statically analyze the wrapped function to deduce how to do the fuzzing? I’m struck by how it got the magic word immediately.
I'm partial to American Fuzzy Lop, which compiles C++ code so that it knows which branches were taken. Can Rust code be fuzzed the same way, and is there a way to fuzz Haskell code that does something similar?
I yell at code all day.
Also related but not the same: Property-based testing, those who haven't tried it will be amazed at it's usefulness.
Those if statements are not very readable, but that is the prefered way, implementations details rather than intensions or requirements, if that is what people do then there is no alternative.
Para pensar, señores.
It reminds me some OOM error bug that got in project that was caused by using msgpack library (Java). The msgpack library deserializes byte stream into some objects - it was deserializing a base64 string to object. Apparently the library supports read a big array of bytes. Msgpack reads the message in sequence - does not know what data comes next - when the byte with flag for huge byte arrays comes in it pre-allocates array of 2^32-1 byte-elements. Found it because we had a malformed string that was not object we wanted to deserialize but rather random string.
Later to confirm to architects that any idiot with msgpack documentation, paper and pencil can do it - prepared a base64 string on paper that mimic the good object to deserialize and then put the bytes of memory doom. They wanted to do some happy checking of first few bytes - after short demonstration - they changed their minds. With some java like fuzzer I would do that automatically (and probably the error could be found earlier), but fun of playing with bytes was awesome.
I really like the terminal environment you're using, how can I get my setup to look like that?
Vim, prolly
Neovim *
i3wm
This is cool af
how do i remove the path stuff inside my exe.. i see it exposes my directory in the exe.
How does this compare to concolic testing?
pretty cool.
6:41 shell users everywhere are screaming at you there's no need to use cat, just use the
I thought you were doing like me and really cursing while programming, well that will prevent me from cursing
This is how that belt makes your child stronger
simple good video
Fuzzing is how Zenbleed was found!
1:09 I already can guess r will be less than REQ_SIZE because recv doesn't have WAIT_ALL flag.
Even with WAIT_ALL maliciously crafted input could cause errors or DoS
can you share the code with the bug ? thanks
At 2:33 i see the bug. He copies data based on the user inputed length on a buffer that ia limited to 64 bytes. I will watch more to see if this is what the fuzzer finds
I presume this fuzzer actually looks at the soruce code of the program, to predict how to best gain different outputs? It is not just random text generator?
It doesn't exactly look at the source code. Instead, it memorizes which random inputs caused which if-branches to be taken, and randomly mutates those inputs to "cover" as many routes through the program as possible. They call it "coverage-guided fuzzing".
@@user-qm4ev6jb7d thanks for the explanation, it's pretty cool
This seems great. I expect those eagle eyed developers saw the h- >len value, and thought to themselves about how user input is always evil :p but hey, the unsigned one did surprise me too! Luckily i like writing u32 u64 et al.
Even faster with a switch statement? You are already using a switch statement!
I could see the bug even before the first test iteration...
My favorite part of testing is "cat /dev/urandom | ./a.out" But that's specifically for testing proper error handling.
Hi ! I don’t understand the unsigned problem. Could someone explain?
Signed numbers include negative numbers which the program had no way to handle so they caused a crash. By making it unsigned, it forces all values to be positive integer values 0-255 which the program could easily check.
What’s that you say? I’m not retarded I’m just left handed. This video just made me literally cry 😭
Limit the stack size to zero.😂
Wow, sharp transitions should be smoothed out, otherwise this is an ultra-useful video
Although slightly off-topic, I was wondering if you could make a video explaining how cheat codes function in games like GTA San Andreas or Vice City. How they interact with the memory and what processes occur behind the scenes. I'd really appreciate a deep dive into this. Thank you!
They're just series of inputs that the game checks for and does something in response. Its not really complicated.
Yeah, you're probably thinking of Game Genie. Which I would like to see a video about how it works!
This guy: "Make your code safer by yelling at it. That's right, LITERALLY yelling at your code, in a very literal sense, can make your code, literally, safer. Legit stretch those vocal chords, open your mouth all the way, and just let out the biggest scream at the very top of your lungs, at your code, to make it safer!"
This guy 20 seconds later: "So this process involves feeding random data into your program and..."
clickbate my beloved
i = 4;
cout
C++ and C languages have very interesting thing: "Undefined behavior".
This doesn't mean that behavior would be randomly chosen from "a set of possible behaviors".
This means that behavior would be completely undefined. It can run into segfault or start erasing data on your PC. Anything is possible. Nothing is guaranteed.
And for this case:
In "a+b" expression, computation of a and b is not sequenced. They can happen in any order.
Side effects of unsequenced operations cause Undefined behavior.
Once it happened - nothing is guaranteed.
@@sudo-gera thanks
go fuzz 🎉
I'll use Zig
2:30 not validated user input
I love Rust
goat
Rust is good, but confusing for me
You weren't yelling at the code wth
*Pauses video 29 seconds in*
You can't say LITERALLY yelling at your code if you don't mean to actually YELL at it vocally. That's the opposite of what LITERALLY means. :/
8:27 Wrong. switch statements are not faster than switch statements.
You need a pump gun to fix your code.😂
Rust is silly.
a
23h ago
at 0:24 you said tha it's about "literally yelling at your code", but i didnt hear any yelling, though. Literally yelling means moving your face muscles to produce loud noise, yet during all your vide you were very calm. Why did you lie about this technique?
0:24 two incorrect uses of "literally" in less than half a minute. Congrats
third
First..!!!!😁🤩😍💯💥✨💫🔥👍🏻👏🏻✊🏻🤜🏻🤛🏻🙌🏻🫶🏻🙏🏻👌🏻
first
This is too powerful... people should just stick to Python