Do We Still Need Dataclasses? // PYDANTIC Tutorial
Vložit
- čas přidán 5. 08. 2024
- Pydantic is a very useful package that makes dealing with data much easier, similar to what Python's built-in dataclasses do. Where Pydantic shines is in dealing with validating and sanitizing data. In this tutorial video, I show you how it works, and when to choose Pydantic over dataclasses in your code projects.
The code that I worked on in this video is available here: github.com/ArjanCodes/2021-py....
💡 Here's my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Designer Mindset Team Packages: www.arjancodes.com/sas
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
A few interesting links to articles and books:
- Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides: amzn.to/3jllgyH
- Principles of Package Design: Creating Reusable Software Components by Matthias Noback: amzn.to/2NETK3l
- Clean Code: A Handbook of Agile Software Craftsmanship by Robert Martin: amzn.to/3qVZgNs
- The original Design Principles and Design Patterns article by Robert Martin: fi.ort.edu.uy/innovaportal/fi...
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
💬 Join my Discord server here: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
👀 Channel code reviewer board:
- 0xUnicorn
- Ryan Laursen
- Sybren A. Stüvel
🔖 Chapters:
0:00 Intro
0:52 Explaining the example
1:54 Creating a Pydantic Basemodel
4:42 Adding field validation
9:47 Adding model-level validation
12:54 More possibilities by adding a Config class
14:03 Converting the model instances to a dictionary
14:38 Other things that Pydantic can do
15:38 So, are dataclasses obsolete?
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
In CZcams there are lot of python contents. But 99% are for beginners. You are the only guy who teach python advance concepts very clearly. You are doing a such a amazing job sir. Keep it up. Good Luck! 🙌
Thank you - glad you like the videos!
Check out mCoding
How you feel about Python CZcams videos is EXACTLY how I feel about all MATH videos. I specialize in differential algebra. There exist literally ZERO videos about differential Galois theory, for example! Or differential rings.
I know less about Python now in January 2023 than 8 years ago when I began studying it. In fact, I know less about programming now than I did in Summer 1981 when I took my first computer coding class: Fortran at local community college.
Yes, there SHOULD be MANY more ADVANCED TOPIC videos for non-beginners.
However, 99% of all Python creators/coders are beginners. Extreme beginners.
That is why 99% of the videos are for beginners.
You gave him at least one new subscription for your comment
I was just looking up fast api, with tortoise and this. You are literally reading my brain like an open book. Thanks for uploading as always.
Thank you, glad to be service again ;)
After watching and rewatching your content for just 2 weeks I can already let you know that you have made me a better Software -Engineer.
Your creative work matters to many!
Thank you a lot!
Great to hear! I'm happy that the videos are helpful to you.
me need to learn automatic doc creation :D and be better developer
In my opinion Pydantic serves a very specific purpose, which is parsing and validating data. If you know what data you are working with and are sure what type of data you have, there is just no reason to use pydantic as the instance creation time is higher for a pydantic model compared to a dataclass
Well, that's not really an opinion, data validation is something you must do when you can't trust the input data. If you 100% know that it's well formed and conforming data validation is redundant and thus, not efficient. That's exactly why you have booth option with Python dataclasses and Pydantic dataclasses, they're not substitutes but complements
Thank you very much for choosing topics that are not only for beginners but for more advanced users. This kind of topics are really enjoyable.
Thank you very much!
Great video. Things to note about the difference vs dataclasses. By doing runtime (i.e. on every object of your class creation and every change to validated field) you are taking quite a considerable performance hit. So don't use pydantic on classes you don't need to. If your data comes from the outside (user input, file, HTTP request and so on) you should absolutely validate it but don't do that for data-focused class. Also pydantic offers a module called `pydantic.dataclasses` that lets you use dataclasses with validation. Note that BaseModel provides another functionality as well but if you don't need it and want some validation for your datalcasses you can use this with minial changes to your dataclass-using code.
Oh Man, I'm from Brazil and for a long time I've been looking for this kind of content. With this whole technology bubble I've just seen superficial content about technology "how to create a crud in Django" and it has its own value but when you decide to go deep into the technology that you use most part of time we needed to get a book and try to extract as much as possible from it, but now we have you buddy, it's so nice to hear this deep content about python and it's like you've said "if you want to become a better software developer" you should probably go with arjan codes hahaha. Thank you for your content.
Thank you so much! I’m happy you’re enjoying the videos and that they help you. Ultimately, having a proper background in software design principles and patterns, knowing them through and through is one of the most important skills to have as a software developer. And that’s why I’m doing these kinds of videos instead of highly specific recipes. Feel free to spread the word to anyone who you think might be interested in this as well!
3:21 Python 3.9 deprecated typing.List in favor of built-in type list (or dict, set. tuple, etc), which is much nicer because you don't need to import anything. (Additionally, there were many other deprecations from the typing module in favor of their collections.abc counterparts, so make sure to check that out too). Really these days, I only go to typing for "type theory" kind of stuff, like Any and Optional. Great video nonetheless! I'm a big user of attrs, which fills a a similar, but different, void, and didn't really know how they compared until I watched this video. Thanks!
Thank Tim! I learn something new every day, thanks for pointing that out. I’ll look at it and switch over to the built-in types in my videos.
True that you dont need to import anything, but with typing.List you can also specify whats inside the list.
@@Kevin-jm1go That also works with the built-ins. For example, `foo: list[str] = ['hi', 'there']` works as you'd expect. Try it out!
And now Optional[something] can be replaced with something | None
if you want your code to run on multiple version of python it's still useful to use the import
I really needed your videos to get on next level no other channel covers what your channel covers.
thankyou so much
One of my favourite things with Pydantic is that it supports nested objects i.e an object as an attribute for another object. Really helpfull when dealing with related objects.
Thank you so much for this very interesting Pydantic example with both validation levels Arjan! 🙏😀
I'm hoping you'll blend in more Pydantic in your tutorials in the future.
FYI: for the price field you can use Decimal or condecimal with max_digits and decimal_places to more accurately represent money.
Really helpful video! Even just finding out dataclass is the better fit for my project and seeing some good clean code is always a chance to learn. I really like that you put emphasis on the small things of nice code like adding docstrings.
Do you have a video on what class methods are?
videos are getting better and better. Thanks a lot for your time and effort, much appreciated.
Thank you so much!
your content is so good, i learn a lot and it made me better.
Thank you a lot!
wow, i was just searching for a tutorial on pydantic. As a newbie, i can say that your videos have single handedly upgraded my knowledge of the field. thank you very much and please keep up the good work 😊
Thank you Zara! I’m happy that the videos are helpful!
Just fell in love with pydantic. I was poking around the code and noticed that pydantic solves the problem for nested dataclasses, for this matter I modified the title and subtitle to support mult lang (each field was now a list of text and lang pair), i created an extra class called MultLangStr and modified the json, pydantic identified the nested information and used the proper class whilst built-in dataclass ignored it and keept the dict format.
Awesome!
Wow! Beautiful coding... And Pydantic, very powerful! love that dict(exclude/include feature), remembers me a NoSql DB style query.
Thank you - glad you liked it!
As always, excellent video thanks for the time, i learned something now to implement it
Excellent tutorial on pydantic ! :) Thank you! :) 🙌
Looking at this guy coding makes me feel noob in python again
Really awesome video, I feel like I'm learning a lot about best practices really quickly watching these.
Glad you enjoyed it!
Great introduction. I didn't know about Pydantic before but now introduced this in one of my own open source projects. You have a new subscriber. ❤️
Welcome onboard ;).
This looks really nice, I'm interested in seeing how it will integrate with typing.Annotated coming with python 3.10
Best Pydantic tutorial on the youtube
I want to thank you from my bottom of my heart! You have made me a better Data professional!
I'm glad my content has been helpful! :)
You are one of the best if not the best Python Channels that I know of. I am learning a lot from you 😊 Keep up the interesting topics and good work 👍👍👍
Thanks, will do!
Great video Arjan. Subscribed.
Thank you, glad you enjoyed the video!
@ArjanCodes, You are a maestro!
Thank you so much!
Thanks! Good stuff.
Very interesting stuff. Continue the great work!!
Thank you!
Thank you for the clear explanation!
Glad it was helpful!
so helpful and clear thank you so much!
Glad it was helpful, Kevin!
Thanks Arjan for awesome video on pydantic and its uses. I was wondering if we need to add '@classmethod' after validator and root_validator while defining a class, as according to documentation both validator and root_validator are classmethods themselves.
Excellent tutorial. More of these please, thank you! :-)
Thank you , will do!
My go to for anything related finishing project. Even as hobby programmer i almost always use your tutorials to make proper code, and in end i remove my bugs from code.
Glad to hear they’re helpful!
Great video and interesting topic.
Subscribed. Thanks for the video!
You’re welcome. And welcome onboard 😉
Dataclasses wouldn't ignore any additional attribute in the dataset used to initialize an instance of the class, it will always raise an Exception, another beautiful thing about pydantic, pydantic will just ignore any extras. Which comes in very handy when using them to model third-party response for syntax highlighting rather then just relying on dictionaries.
great video - thanks!
You bet!
Very useful and interesting. Than you Arjan. Small correction to custom exception, there should be inheritance from ValueError. Otherwise FastAPI generates status 500 without Pedantic validation message.
Great tutorial, you earned a new sub.
Thanks Machinima, happy you’re enjoying the content! and Thanks for subscribing!!
Good video!
TBH pandas could have been used here as well.
As someone who often works with immutable data, what I would love to see from you would be an OOP vs FP in python and when to use which, as thats a question that gets raised often :)
Great suggestion, thank you!
this is helpful, thanks for sharing.
Glad you enjoyed the content, Lawrence!
You are a treasure in innumerable universes
Thank you - you’re so kind!
oh thank you arjan
Great video!!!
Glad you liked it, Chris!
Very nice, I didn't know about that. Is it good for filtering data too? For example: return all books that have title starting with xxx (but without for-cycle, more like JSONPath?
Hi Arian
Love your videos 👍🏽
I wanted to ask if you have any videos explaining how and when to use the __init__ function and the super() function as well as the **kwargs?
I’ve seen you use them in your videos
Are these explained in your course on your website?
Just found your channel. Awesome stuff :)
Thanks man!
Hi Arjan, great video. I often watch your content and I love it. I have a simple question though: does your ISBN10FormatError really need its “value” attribute?
Thanks, Michael! It can be useful sometimes to have extra information in an Error object that you can do something with. For example, if you want to parse the value and then do something else with it to respond to the error.
It kinda annoys me that after about 15 years of writing declarative class with python Traits package (and more recently using the Atom package) offering validation, notification and visualisation, people are talking about data classes as if they are new and exciting (but only offer validation?). Pydantic still only offers a few of the features in Traits.
Interesting 🤔
excuse me , I have a many-to-many sqlalchemy models but how can I create nested ones in Pydantic in order to implement crud?
At this point, I don't care about the topics of your videos anymore. I just watch it for all the best practice and explanaition while programming.
Glad to hear you like the videos!
Awesome stuff.
Glad you enjoyed it
Hi, can you please tell us which VSCode color scheme you are using? Thanks!
Another great video. I thought pedantic is just for validation but I know more now. It is great tool to deal with json object like book = [(**list) for list in json]
Glad it was helpful!
Also check out my original dataclasses video if you haven't seen that yet: czcams.com/video/vRVVyl9uaZc/video.html.
Can we just take a moment to appreciate Arjan's typing skills? I'm impressed haha great work on the video as always!
the video speed is increased while typing, you can see it whith the head mouvements
would you be able to reveal your setup (hardware etc. mouse keyboard) for your work as well?
Was just randomly poking around... Trying to get smart, Eating popcorn when oh bam, this exactly solves today's problem! Thanks!
You're most welcome :).
There exist many MANY ideas I would love to code, but I know they are not worth it: no matter in what language I code them.
They are all math ideas. A lot (not all) are coding the solutions to PDEs and ODEs. Some are for papers I wish to publish.
Python is the only language I know, and just barely. Used to know C++, a little Java, and Fortran.
I realize now that the mathematical ideas are simply too complex to be coded.
For example in 2020 I attempted to code computing all infinitely many solutions for a trinomial w=C*z-z^A
where w, C, A are complex numbers. Got close, but there was no way to get roots, z, that are within error limits when plugged back into the trinomial. Had hoped to publish it. But I had to leave it unfinished.
Definitely a HUGE lesson about NOT coding anything deep or complicated:
OTHER than the simplest of calculations, say, with a for loop. In other words, Python is my extension of a handheld calculator.
When defining the root_validator check (10:00 - 13:00) why do you go to the trouble of adding the "title" field to "check_isbn10_or_isbn13" when it doesn't appear in the output message/error (12:59) ?? Seems like this was a minor bug glossed over ? I think it would be helpful to know which record was incorrect, as referenced by the title field ? Was that the intention ??
amazing!
Thank you - glad you liked it!
Thanks for the explanation, it helps me a lot .
what is weighted sum? why we are subtracting 10 - index ?
Hi @ArjabCodes, Thank you for your videos. They are very helpful. Once problem which I'm trying to solve. During the running pydantic model a lot of validation happen. Do you know the way how we can display/print/report these validations? I understand that if data is not valid we will get the error, but I want to see what was validated during the run time. Do you know how to do this?
Thank you
One little advantage of pydantic is that mutable default arguments are easier to use. I know dataclasses have the field possibility, but that has its limits and is a little ugly.
Also the validation implementation is a lot nicer in the newer versions :)
can you set an order for the objects in pydantic as for dataclasses?
@ArjanCodes I'd love to see an updated version using the new Pydantic v2, your thoughts on it, and how it sits now in the pydanticV2 vs dataclasses p3.11 vs attrs
Thanks a million Arjan for great video. I had a newbie question.
When I changed the order of "@classmethod" and "@pydantic.validator("isbn_10")" the way you suggested(minute 9 video), I did not get the Error. I know the importance of placing decorators in order but I do not understand how come the validation does not take place if I put the "@pydantic.validator("isbn_10")" immediately above the method declaration? it gets overridden?
what is the cls in the isbn_10_valid function?
Great video! Thanks for video and link of your Git repository !
Glad it was helpful!
I’m glade that my tutor recommended your channel to me wish you a good luck and more video)
Thanks!
Thanks and glad you liked the video!
At 11:15, is writing the condition like this better?
if not ("isbn_10" in values or "isbn_13" in values):
Btw Love your content. I was thinking of pydantic when watching your dataclasses video, and then found this. I used pydantic in FastAPI and it's absolutely awesome!
can this be used in dataclasses for validation as well?
hi, into the documentation where explain about how pydantic type more than 2 dates like return example () -> int, str .....
Thank you! I learn some useful stuff from this. Can you define a custom parser for one of the fields? For example if your data is a list of songs with that have a title datafield that is actually both artist and song title separated by ' = '. For example "Metalica - Nothing Else Matters'. Can you define a custom parser that splits this into two separate fields in your model?
I think the root_validation method returns the value to be set in the object, so you can modify the value that you receive there - haven’t tried it though.
@@ArjanCodes Thanks for the tip!!
it's really better for parsing like you what you have done, However if this data is internally, i mean inside my application, i will lose a lot of resources within dynamic checking, although i might use attr class or dataclass for that task...
thanks for sharing your ideas with us...
this video just kinda changed my life))
Hopefully in a good way 😊
at 9:30, where does the value go?
Can you post the link for your code that you stated in this video on Github? I can't seem it in the description. Thank you!
I somehow forgot to add that. I put the link in the description.
Is it possible to use validator to check if either isbn_10 or isbn_13 is provided? Since we check two fields, is root_validator necessary?
As a class method it looks like the validator has access to only the single attribute value. Or did you mean combining the two attributes into one?
00:03 Pydantic offers data validation, conversion, and sanitizing features.
01:44 Using Pydantic to work with data, add validation and sanitation
03:46 Pydantic is helpful for unpacking data items into keyword arguments and easily printing data.
05:27 The video discusses the usage of dataclasses and the validation of ISBN 10 field.
07:53 Using Pydantic to add validation functions to ensure data cleanliness and conformance to defined rules.
10:01 Adding a root validator to check if a book has either an ISBN 10 or an ISBN 13
12:39 Using Pydantic to create immutable objects and perform data processing.
14:46 Pydantic is a good solution for importing and validating data.
Crafted by Merlin AI.
Was this timestamped TOC created by AI?
Hey Arjan, quick tip. If you are in Python 3.9+, you no longer have to import the List to do type hints with lists.... so in your example, you can just do list[book], and kill the List import.
I guess I don't know if Pydantic provides support for this or not yet...
Also, really getting into your videos. Super well done.
Thanks Joseph! Yeah, I noticed that too. Same for dict actually, so I'll stick to those from now on.
The text in the background
When not write a Rest API, why not using attrs instead of pydantic? Is faster and not so resource hungry.
Why not simply define an array of validation methods for a dataclass (und treat dataclass like simple c-structs, without additional methods). This way all our components are neatly separated and we don't need to overpopulate our "Book" data type with a million methods. This also would make modification simpler, because then we simply add or remove validation callbacks as needed and have no need to modify the Book class. This, imho, would be way more SOLID (and thus maintainable), than what was presented in this video.
or use a separate library, such as cerberus. You will then get a clean, fast code and separation of concerns. I don’t understand all this hype about pydantic. You get hard to extend classes bloated with unnecessary methods that do everything but nothing of good quality.
Dear Arjan, I'm a Python developer and I found this channel by accident. I see it has a ton of high quality gems, but my problem is that I don't know 'how to ArjanCodes channel' . :'D
When you are typing or coding can you please zoom in because most of us watch your videos on the phone screens. Love your content. Thanks
Thanks! In my more recent videos, I've zoomed in more on the code!
I know pydantics better now, thanks. :-)
Glad to hear the video was helpful!
In the custom exception class for "ISBN not provided", what's the benefit of doing self.title = title? It seems awesome for debugging but I didn't see it in the traceback
Also, awesome video!! I consider myself a bit of a python noob, so I'm always learning new stuff from your videos 😍. Just started a small project using MVC architecture and working with classes. It's been awesome to implement some stuff that I've seen here
Thank you Nick! At the moment, that title field is indeed not doing much, but I do think this is really useful, because it allows you to pass better information in the Exception, which you can then later use for debugging, or taking other actions in you application depending on those values. You could add a repr method to the Exception class to print a better message, or it might be possible to turn the ISBN exception into a dataclass, which adds that automatically.
@@ArjanCodes Is it considered bad form to store a reference to the object in the Exception?
Is there any course available for purchase
Thanks for asking. Not yet, but I am working on something.
Can you do a video on Type hinting in python.
Great idea, thank you.
In python3.9, you no longer need typing.List, do you ?
Correct! I’ve recently switched to using the built-in list in my videos.
If you add "from ___future___ import annotations" you can use the built-in classes (with some exceptions) in Python 3.7+ ------ Note: I am not able to find the specification of the CZcams markup to prevent turning "future" between underscores into " _future_ " (italics).
any chance you want to update the code for V2?
I’m going to revisit Pydantic soon!
Decent tutorial. However, the title is misleading: I was expecting an in-depth comparison and pros cons with respect to dataclasses in code.
Bro what is your keyboard? It's so soothng to hear and type i guess?
Keychron K3 :)
@@ArjanCodes cherry mx silver? or red?
I wonder what is the runtime cost for all these extra options...
Negligible if any.
@@jl_woodworks unless you create many instances in a tight loop...
@@jl_woodworks compared to attrs and dataclass, it is very slow, especially when serializing many objects. I don't recommend using it where speed is important. stefan.sofa-rockers.org/2020/05/29/attrs-dataclasses-pydantic/
@@rozarioagro5532 Thanks for the link to that article. Pretty insightful. And it shows that you should not take claims about speed made by the project itself unquestingly.
Of course, in the real world it's annoying that programs dealing with books insist there has to be an ISBN. I have quite a few books from before the introduction of ISBNs (the oldest is from 1711), and I rejected quite a few cataloguing systems because they wouldn't accept some of my books.
Will we see Pydantic vs Marshmallow?