The Magic of Property Testing
Vložit
- čas přidán 16. 07. 2024
- I like well-tested code. But I don't like writing and maintaining hundreds of unit tests. So here's a quick walkthrough of one of my favourite testing techniques - property testing. Under the right circumstances, property tests let you test hundreds of possible situations for nearly zero programmer effort. They are incredible value for money. 🤑
So in a footnote to our usual interviews, here's some coding demonstrating unit tests in action, and why I like them so much. And along the way I get to show you one of my favourite languages - PureScript - and one of my favourite editors - Neovim. 😎
--
Source code: github.com/krisajenkins/prope...
Kris on Mastodon: mastodon.social/@krisajenkins
Kris on LinkedIn: / krisjenkins
Kris on Twitter: / krisajenkins
--
0:00 Why Property Testing Is Great
1:31 Let's Set up a Build
3:04 It's Set Up - Let's Code
8:04 Low Cost Changes
8:36 Complex, Nested Types
10:15 One More Example
11:03 Conclusion
I like this format! It's nice having a demo on the topic with actual code.
seems weird to have a demo of a testing tool without any failing test
You’re a breath of fresh air. I love your style, your approach, etc. I hope to be as respectable and competent one day. That you for being a mentor!
Just found your channel, interesting stuff. Never heard of property testing before. Watched the other vid and yes, TDD ends up with a lot of tests to maintain.
I like your easy going style, subscribed and waiting for more demos.
Very cool! I'd say it's basically like having a test that fuzzes a single function 100 times
I agree with the other commenters, I thought this was really interesting to witness and would definitely appreciate further videos like this alongside your usual high-quality podcast interviews :)
One reason to back up my opinion is that while it's always fun to hear about tools, languages or new approaches towards them, watching a demo video like this takes it from being an abstract concept into something concrete that we can immediately see the benefit of. Even though I don't do much JS programming myself, seeing something like this in action is very inspiring and certainly leaves me hungering for more!
All in all this is just a long-winded way of saying I really appreciate what you're doing on this channel and look forward to seeing more of it :D
Great demo! :)
Thanks!
Yes, you should do more!
spagbowl is aussie for spagetti, but it's even more aussie to further contract that into spago. Recursive contraction is the pure essence of being Australian. Also, regardless of your Realpolitik, defining gender as a complex number was a stroke of genius, haven't seen that before. Hats off, moar!!
Spagbol, surely?
@@adamcetinkent yeah I'll pay that, spagbol is most likely correct.
3:43 I laughed so hard! Thank you!
Do. more. Demos. :)
Aye aye sir. 😁
Awesome demo. Is there value to fixing some kind of seed? I just wonder if the tests could be slightly flaky when the data is so random - but maybe that's actually a benefit, and not a down-side? Overall though I'm impressed with property testing - I haven't seen it in practice like this before.
Most property testing libraries will allow both. They run with a random seed, but if they fail they report the failure _and_ the seed that was used. Then you can repeat the run with the same seed as-needed.
very interesting
Do you know of any good tooling for this for Python?
A friend had mentioned Hypothesis, which _seems_ similar on the surface, but I haven't got around to trying it yet.
I don't, sadly, but having looked around Hypothesis is definitely where I'd start. It looks good. 👍
I would like this for other languges
what is the difference between property testing and fuzzing test ?
The difference is the assertion of a known property of your system, that is automatically checked. Fuzzing is just throwing random input to a system an then manually analyze unexpected exceptions or crashes. You are just checking general robustness and not any specific feature.
I really like your color scheme. Which one is it?
It's github.com/tomasr/molokai 👍
@@DeveloperVoices Thank you!
Neovim btw
This seems doable when metaprogramming is available and you can put in random values that cover the entire breadth of types, but how would you adapt property testing to scenarios where that isn't true? Defining valid and invalid ranges for different values, or scenarios where fields are dependent on each other in more complicated ways (e.g. you have two pairs of (int min, int max), but pair1.max must always be less than pair2.min, or replacing that with (int min, int middle, int max) or what-have-you where it must be true that min
Depends on the library but you can often pass in a function that generates the inputs you want. Limiting ranges is trivial with that. Fields dependent on others may be a bit harder but should be doable in any decent library.
The cool part is that you can often just write what kinds of inputs are _invalid_, so you can keep the breadth prop testing provides, only disallowing inputs you explicitly forbid. (Prop testing is supposed to catch inputs you didn't think of, so it would be silly to only test inputs you think of).
Note: what i said is just pushing the work to the library. I think it would just keep trying to generate from the whole breadth, so not efficient. There are probably builtin implementations for narrowing common patterns like ranges of numbers. Or, since you are in control of the generating function, you could try to make it efficient yourself.
Usually there is a re-usable generators/shrinkers library (in Haskell instances of Arbitrary type-class). Having metaprograming is just way to get it with indirect implementation via nicier syntax. E.g. use Java reflection to get method signature in order to understand what values it should accept.
Those gnerators can be implemented explicitly. Including for simple data wrappers to trigger more complex logic. E.g. you generate consistent pairs of values.
You can also play with how numbers are generated. E.g. (Int a, Positive deltaB, Positive deltaC) ⇒ run(pair(a, a + deltaB), pair(a + deltaB, a + deltaB + deltaC))
You can also use implication to "filter out" invalid ones. E.g. checkConsistent(pair1, pair2) → test(pair1, pair2)
But what do u test here? It seems you tested, that prebuilt features work, which isn't helpful when testing ur application. As far as I see it, you got actually zero return from those tests. If u want to test serialization to call a rest API for instance, then you would need to test against the specs of that API. It all could serialize into a QR code and back and your tests would be happy but calling the rest endpoints would all fail.
My first thought was exactly that. It is nice to have automatic generation of data cases, but what it does is testing if the serialization was done properly. And that should be already tested by the library.
Yeah, the tricky thing for me to judge with a vide like this is what do I put in, what do I leave out? I couldn't see a way to include custom serialization code without adding in a lot of extra explanation about PureScript's language features. And that would have muddied the main point.
FWIW, here's an example of the custom serialization code. The testing side would remain the same as in the video: github.com/krisajenkins/FallFolks/blob/main/src/Common/Types.purs#L165-L172
Now I'm always going to think of gender as complex. Spoken as an 8-3.2i.
(I think this means I'm an arborgender who produces pollen instead of eggs or sperm so I identify as a tree. It's nice to find the numeric representation!)
😁