New Features You Need To Know In Python 3.12
Vložit
- čas přidán 21. 07. 2024
- Python 3.12 is just around the corner, and I couldn't be more excited! 🎉🐍 In this video, I'll not only dive into the thrilling new features and improvements coming our way in Python 3.12 but also discuss some elements that will be removed in the upcoming release.
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
💻 ArjanCodes Blog: www.arjancodes.com/blog
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/jlpmsd
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
Social channels:
💬 Discord: [discord.arjan.codes](discord.arjan.codes/)
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
0:51 Better error messages
2:06 Performance improvements
2:28 Comprehension inlining
3:26 Smaller object sizes
3:41 Immortal objects
4:39 Per-interpreter GIL (PEP684)
5:47 F-strings
6:36 TypedDict kwargs
7:32 Override decorator
8:26 Generic types
10:24 Other
11:37 Outro
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
We make use of kwargs in our library for passing user arguments to functions from other libraries (i.e in our wrapper functions)
If I remember correctly, packages with graphing capability like pandas and seabourn use this to enable pass-through args to the underlying matplotlib graph object.
One thing I'd love to see is improvements to the documentation of kwargs, it can get very frustrating, even though it'd very powerful.
kwargs are very powerful, especially when used with args, but the price you pay for the flexibility is obfuscation, if we could have all the benefits without the obfuscation, then that would be amazing
@@PixelThornyou might like typing.ParamSpec, helps a lot for this kind of thing
Such practice requieres you to know implementation details of the function, I dislike that in general.
@@sebastianrodriguezcolina634Like most things in software engineering, there is a trade off between power/expressiveness and clarity/safety. Python already leans toward power/expressiveness in most aspects (dynamic typing, etc.), so using kwargs is at least pretty consistent with other conventions available in Python.
I have two use cases for kwargs:
1. when creating a wrapper function for another function.
2. when passing around parameters for ML and data science projects. Typically I store these as json files, so it’s nice to use dictionaries to pass around those parameters.
Do you have an example I could learn from?
@@kelkka7no
@@kelkka7 Check how to write decorators for functions with arguments.
This. I do this as well. Nice and clean.
Also when making an abstract or parent class, different child classes might need different arguments in their constructors and other functions. As you can't control how future developers will extend the class, you need to keep some functions generic. This is similar to the wrapper use case, except in this instance you're wrapping functions that might not exist yet.
Hey @ArjanCodes, if you want function annotations, then create a Protocol class and define the __call__ method and then add the type anotations for the parameters in that __call__ method. When you will use the the Protocol to type a function it will describe it with the type of the __call__ method.
Kwargs are a really great feature when you are writing larger repositories. When passing arguments to other repeatedly called functions, this makes the code much cleaner. One example if with spark dataframes. I have one function that does all of the filtering with arguments per column. Most other functions call this function at the beginning with kwargs being passed directly to it.
I have not used kwargs too much myself, but I have seen some really good use cases. One of them is collecting a bunch of optional (kw) arguments at top level, and then reducing the kwargs dict as you move down the tree of functions or methods, as each of them have explicit named arguments set by some of the kwargs provided by their "parent" function (the caller) and then collecting the remainder as a smaller kwargs dict passed to their children.
Super terrible idea! Do this and after a month you won't have any idea what inputs your method require. When you feel the need for kwargs use a class.
Thanks for again for your excellent content. I would love to see a comprehensive tutorial for typing. All of existing ones go through the basics, but someone needs to explain how I can get VSCode and Black to pass on real code with a "strict" setting.
🎯 Key Takeaways for quick navigation:
00:56 🐍 Python 3.12 introduces improved error messages, including suggestions for missing imports and common coding errors, enhancing the coding experience.
02:24 🚀 Python 3.12 brings performance improvements, particularly in comprehension inlining, resulting in up to 2x speedup in some cases and an overall 11% performance boost.
03:51 🧊 Python 3.12 introduces "immortal objects," objects with a fixed reference count, reducing memory usage and simplifying code optimization.
04:49 🌐 Python 3.12 lays the foundation for per-interpreter Global Interpreter Lock (GIL), enhancing multi-core utilization in future versions.
05:48 📜 F-strings in Python 3.12 become less restrictive, allowing nested double quotes for improved string formatting.
06:44 🎯 Python 3.12 simplifies type annotations, allowing for easier definition of keyword argument types and introduces the "override" keyword for explicit method overriding.
08:40 🧬 Python 3.12 introduces a new syntax for type parameters and generic classes/functions, simplifying the handling of generics.
10:32 📂 Pathlib now includes a "walk" method, making it easier to traverse directory trees, and CPython 3.12 supports instrumentation for monitoring calls, returns, lines, and exceptions, improving debugging and coverage tools.
11:28 ❌ Python 3.12 removes deprecated modules like asyncore and asynchat, encourages the use of async.io, and deprecates old method names in the unit test package.
Made with HARPA AI
I usually use *args and **kwargs when creating custom decorators. I like to create decorators for code reusability and custom tools for my projects. Also a custom paters is the composition and some time you can create some custom wrappers utilities that receive an object instance with some arbitrary parameters and you perform some kind of logic.
Using kwargs with argparse for passing command line args to the program entry point is one of my favorite Python patterns. Nothing more satisfying than writing main(**vars(args)) instead of passing everything to main manually.
I would love to see more content in the form of series on
1. threading vs multiprocessing
2. distributed computing in python using ray framework or its equivalent
Yes i use kwargs when I have a wrapper function over a core function and I still want to provide all the functionality of the core function without having to update the params in two places.
Double quotes in f-strings is life-changing. How did I ever life without this?
override decorator is a nice addition. Before every method was virtual. Now you have a bit more control over it.
I have a query builder that passes user arguments (optional and required) along to a DB client as a formatted query string. Good for when I have to maintain a lot of different etl and dbt pipelines and want to use the same basic development pattern for them. In general kwarg unpacking is really invaluable or any kind of client wrapper function or class.
Using kwargs in Django view helpers. Have a few pdf reports that require generation and using pdfkit underneath. So kwargs is being used to pass context for pdf report generation and then also for configuring pdfkit
the TypedDict kwargs feature will be useful for cases where we have function params that are being passed around or through as a group
5:48 (f-strings), very much needed and appreciated!
10:06 I agree with you, but sadly the PEP for it is rejected. See PEP 677.
Nested quotes in f-strings is a useful feature when refactoring, and I think a good choice to allow (there was significant debate on this one). However, it also looks bad and is harder to quickly parse visually, thus I hope code formatters will reformat it when possible. This would make for a nice compromise, because if I refactor my code to place a string inside of an f-strings, I could get the ease of readability of alternating quote types with a simple call to my formatter.
It’s be pretty funny to see a Python 3.14 version nicknamed Pi-thon with a bunch of circle based updates
The place that i use kwargs is when passing arguments to celery tasks in flask applications
Mainly a yawn except for better comprehension speed and better GIL lock optimization which really isn't surfaced. Type annotation stuff leaves me quite cold because if I wanted a typed language I would use something else in the first place.
The override decorator is my favorite new feature.
7:23 - yeah, all the time. i come from languages where this is necessary so it's more of a habit. but i enjoy reading and refactoring code that has type annotations, it speeds up my work tremendously
The enablement of having nested quotes in f-string is great when you need to access some dictionary field inside the f-string.
@Arjan,You are the best python programmer. Waiting for your architecture course!
The new generic style and the new type alias function is what I'm more excited about.
Same, plus Unpack and TypedDict for type annotating kwargs is quite useful
@@sohangchopra6478 yea I can't wait to get to use them.
I only use kwargs for input cases, and even then it's rarely used as there aren't too many cases where you can't write an explicit alternative
One of the common cases for kwargs is when you need to pass-through args to os.execv. For example for cli tools that wrap something.
I usually try to avoid kwargs as well, but it is useful when making a wrapper around something that either has *a lot* of parameters or itself has a kwargs that you need to supply.
kwargs are suitable for metaprogramming, especially when making libraries. I use them for making general functions for tests.
One thing I was playing around with was using a combination of inspect and vars() to loop through each function, extract the arguments it needs and select the appropriate variable(s) from vars() to pass in. This is nice when variable names match the function argument names so it's not perfect but useful in some cases. I don't have to worry about figuring out exactly what a function needs. I'm sure there are negatives to this as well but it's one way I like.
I tried using kwargs in that pattern that you keep passing it down and each layer gets its parameters, but I think it gets really messy after some time. But I do use it in the signature in some situations where I , for example, load a config and want to initialize an object with it. I'd call it like **config. The object gets what it wants from the config, but I don't pass kwargs further down. I find it especially useful when the code is changing quickly.
Thanks for review! Which theme do you use for VS Code?
Thank you for your video. I've never used Generic function or class. Wouldn't it be interesting if you do a video on it?
A video about Python 3.12 that has a duration of 12.03. Nice!
Nice, the typevar hustle was one of the reasons I switched to Julia
Keyword arguments make sense when you are writing code where you don't know your inputs. For example, I had written a Python PowerShell wrapper function which would take a unknown number of arguments but the logic to execute the code was mainly the same. Very special use case but I agree that I can't see a reason to use it much.
We're using kwargs mostly in forward method for pytorch models. It allows for versatile forward implementation like diffusion models that take not only input but also timestamp, as well as other tailor made models that require more than just input. This way we don't have to break our model interface or split it
At the cost of not telling your fellow developers what could go into kwargs…
I use kwargs mostly in the inner functions of decorators; here I want a very generic function interface to allow any function to be decorated. But other than that I agree, kwargs are kind of fuzzy. I saw a really bad use case for kwargs in a multiple inheritance example once, where kwargs where used to 'bubble up' arguments through an inheritance chain lol.
6:30 I think is important that new feature, I recently build an API for NLP, and the user can input strings, and if the string had " character, the pydantic validator returned 422 status. I think that now could be fixed, right?
I'd love to see a beginner guide on how to upgrade from 3.1x to 3.12.. Including getting all the globally installed modules carried over and working.
That depends on how outdated your dependencies are. We just finished upgrading a big system from 3.6 to 3.11. It was extremely painful! From 3.1x to 3.12 it should be a piece of cake.
There really won't be any price to pay for upgrading from a recent version of Python. It should be seamless except for certain packages which will need to release wheels for that build. You should be fine.
I used keywords arguments for web handlers so that I can convert responses to an object very quickly and don't have to worry about excess properties coming from the frontend.
Thanks for the update.
Keyword args are fairly important when writing data pipelines in Apache Airflow
Amazing! Almost all of these changes are things that have personally annoyed me many times. Python has been killing it recently with these updates.
Your sense of humor is on point today!
0:10 In the past Python had their last version be a mathematical constant.
1.6.1 aka φ (Golden ratio)
2.7.18 aka e (Euler's number)
3.14.1? aka π
Coincidence? I think not
Does this mean Python 4 is coming soon? Probably not
Tried to subscribe to your newsletter but did not get a confirmation email. Is your service down?
I usually need keyword arguments when writing a function A that wraps another function B and I need to simply forward the arguments from A to B. But that is a very specific use case.
8:59 so default type vars specified with the new syntax are now covariant instead of invariant?
EDIT: Answer: No apparently type checkers can infer it now.
As a data scientist, I use kwargs a lot because a lot of the code I work with needs to be run in many configurations during experimentation.For example, model training loops can be way easier to control with a function that uses kwargs.
most languages don't rely on variadic arguments outside of string formatting/collection creation, and nobody complains.
python's variadic arguments are a prefect example of induced demand. most languages would define structs/data classes like LineStyle or ModelConfig, but python does kwargs.
importerror:DLL load failed while importing _cext:The specified module could not be found, what is this error please give me solution, this occur when I import matplotlib, in python 12.00,please😭😭😭
for the beginner these new error messages are amazing and so more useful!
What's that keyboard, and what switches? Thanks
Hi! I have error from the designguide email when open: PR_END_OF_FILE_ERROR
I use lot of keywords arguments and type notations,❤
I create lots of functions and sometimes i forget which type of argument it needs to pass, so i have to go back to function and read or read doc its pretty complicated so i prefer to use keywords args, notation lots, and it give a impression look 👀 code, my manager always gwt impressed with my code 😂❤
Wonderful! We get amazing new interpreters. But to access them, the API is the same as exec and eval's.
The only place I can think of where kwargs are appropriate is when writing function decorators (or any other kind of delegates)
Anyone knows the PR for f-stringa, i have mixed reactions. This is going to invoke some chaos compared to f-strings.
I love python, fingers crossed that nogil implementation goes smoothly
Hi Arjan, I tried to sign up for your 7-step guide but your website would not accept my email address. It has a "+" in it and this tends to stymie many websites. Maybe you can fix this?
Edit: It seems that, despite throwing an error, it sent me your guide link, anyway. Thank you!
I have used kwargs only for decorator functions.
the design guide link is broken
Are immortal object immutable as implied or are the effectively a const ref. Different use-cases?
I am pretty sure that he's just making stuff up. Immortal objects aren't necessarily immutable. They arguably aren't even "const ref", but more like "intentionally leaked" or "static lifetime" objects.
I don't use keywords arguments, generally, trying to pass a single pydantic models between functions
kwargs are good when using class inheritance. For instance, a constructor for a subclass is only required to declare input variables that differ from the super class. further instantiating the base class is as simple as super().__init__(**kwargs). in this way, you don't need to declare all input variables in the subclass AND the base class. Most IDEs are aware of this pattern making systems like intellisense recommend input variables for the subclass, even though they are only in the base class.
Hi, please could you create a Video about Mojo Programming Language with Python
I also have some hopes for a fully functional cython at some point. A python syntax language but compiled and making use of some more C features.
i like cython more than mojo actually. Cython is the best choice for me. Unfortunally, does not implement 100% python, which is sad.
Would you cover how to dockerize a Python app?
By the way, the new error messages with suggestions are so useful.
I tried to run python -m ttkbootstrap and had an error.
Fallowed the suggestion I changed Image.CUBIC to Image.BICUBIC somewhere in the library and that fixed it.
Come to think of it, I rarely find myself using **kwargs in plain function parameters. I'll often unpack a dictionary to provide the specified arguments though.
how many versions of python do you guys keep on your machines?
😂👌 exactly!
I never use kwargs, I use a dict if I need flexible parameter.
kwargs unpacking will be useful for a big and complicated things like libraries and frameworks
@10:00 I don't understand, why do you want an infix arrow notation for lambdas? What's wrong with `lambda`?
What I mean is that I’d like to see more readable function type annotations in Python. Instead of ‘Callable[[int], bool]’ I’d like to write something like ‘(int) => bool’.
@@ArjanCodes That could be done if you could define your own custom operators, like “=>”.
Thank you for the video
You're welcome! :)
Is this a non-beta release? I was kidding around wirh the alpha release weeks ago
Nested double quotes!
Imo this deserves the nobel prize for programming!
it's already that time again?
I remember last time there was actually a live stream of the publishing, which was quite interesting, because the different core developers for various projects talked about what they did.
I always like the feature the most where it breaks all your dependencies 🥰
Did you test Mojo too?
after using typescript for years I just hoping there will be a better typing then this was python has actually. :D
Has anyone created a class to pass arguments around ? Would you consider it ? Why and why not ?
Nice video! Sorry to tell you but there is some kind of issue with the sound, like an echo from the wall.
I use kwargs as an argument for my GraphQL resolvers!
GIL is still there?
The @override decorator makes Python a little bit closer to C++. That's a great sign, because the more sophisticated C++ OOP we adopt, the more programmers we need to operate Python
finally there is unpack and better generics
I like typings in general, but a lot of people hate being specific.
immortal doesn't mean "isn't going to change" it means "isn't going to die" it won't get cleaned up. It can still change
thx, i was wondering about that, now it seems useful.
I initially learned python about 20 years ago. A great feature of python 2, compared to C/C++ for instance, is that it didn't care much about types. It cared about interface instead, i.e it doesn't matter the type of the object, as long as it has the method foo() that the code is expecting. That made python inherently polymorphic. Trying to updating myself to python 3, I see that types are back big time nowadays. Not sure where the need for this came from.
there are multiple studies (eg. monitoring projects on github) that show evidences that typing reduce runtime errors, so python but also javascript (with typescript) and php tried to introduce without enforcing this concept. So personally speaking i never had issues with lack of typing in python 2, but i guess it's very different when you have to work over large code bases that you don't really own.
@@giacomo83m thank you
My birthday is 3rd of December and I approve this message
I'd love to have a sane code import (why do I have to care about circular imports in 2023?!), namespacing mechanism and packaging
I cant wait to remove the `if TYPE_CHECKING` and value: 'str' stuff.
Good video 👍
Thanks 👍
I'm not using **kwargs in my functions because it seems as a lack of control. You never know what it contains. Explicity is the way :)
I use kwargs in mixins.
Don't use key arguments, don't use difficult to read comprehensions, don't use ugly (syntax) f"strings".
That intro is really funny lmao
Gives me 2014 vibes