__new__ vs __init__ in Python
Vložit
- čas přidán 7. 06. 2024
- What's _new_ and what's the difference vs. __init__?
Try Hostinger: hostinger.com/mcoding
Use coupon code MCODING at checkout for up to 91% off all yearly hosting plans!
What's the difference between the "__new__" and "__init__" magic methods in Python? New is for object creation, and init is for object initialization. It is rare that a programmer would actually need to override new, so let's see some examples of how it can be used.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
Python new docs: docs.python.org/3/reference/d...
tp_new in CPython: github.com/python/cpython/blo...
C extension example: cython.readthedocs.io/en/late...
Singleton pattern: en.wikipedia.org/wiki/Singlet...
SUPPORT ME ⭐
---------------------------------------------------
Patreon: / mcoding
Paypal: www.paypal.com/donate/?hosted...
Other donations: mcoding.io/donate
Top patrons and donors: Laura M, Jameson, John Martin, Dragos C, Vahnekie, Pieter G, Sigmanificient, Casey G
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding
CHAPTERS
---------------------------------------------------
0:00 Intro
0:50 New vs init
2:51 Main purpose, subclassing immutable types
5:03 Singleton example
6:35 Client caching example
7:58 Encrypted file example
10:33 Metaclasses - Věda a technologie
This chanel has a very high python value. It's nothing like the others, recycling docummentation or from one another. You can easilly see this on the file examples, where there are codecs and itertools correctly used.
I'm happy that my viewers can see and appreciate everything I put into the videos! Thanks for being an awesome viewer!
Yes I like this channel for quick examples of certain features and learning why things are the way they are and what you can do with those features. For software architecture and design patterns I like Arjan Codes. Especially the innovative code roasts series
Absolutely, you are quite literally the only person I have clicked the notification bell for. Top notch content, keep it up!
Yeah, everybody else for this language has one of a few gimmicks running. Selling entry level courses, covering some thing with a crummy library under the clickbait premise that its a smart to thing to learn because it uses “so few lines of code”, or, well, intermediate level courses.
Almost no explaining builtins and patterns.
Quite useful when I want to know something. Straight, to the point. I still think it’s funny that I went: time to learn python to make a quadratic cracker despite having no experience in non block coding, and after a summer, I am now ok at python!
Decrypted message zoom in was priceless.
As usual, excellent video.
Thank you very much, glad you enjoyed it :)
@@mCoding Love your videos. Finally subscribed, that zoom in was well done.
@@mCoding yea, may I ask why you chose those TERRIBLE encryption algorithms
@@rogervanbommel1086 I'm assuming so that people wouldn't try to use his code for actual encryption and blame him if things went wrong?
@@rogervanbommel1086 what do you mean? he used the only mathematically proven unbreakable encryption algorithm
I didn't even know __new__ was a thing until seeing this video. I've always used __init__ and that's what I've seen other people always do as well.
Completely understandable, you can go your whole life and never actually need to override new, but init obviously is needed all the time.
I really like how you get straight to the point and give clear and concise explanations. This channel is one of the best if not the the best resource for understanding more difficult concepts in python.
Man your videos are awesome. I like that they are so much on point without artifical fluff to make them unnecessarily longer. Every video has one topic and sticks to it similar to "a function should do one job only" :). Great job!
Another banger, as we've come to expect at this point :)
I remember reading the documentation on this but the only thing that stuck was the fact new creates the class and runs before init - what that actually meant or what I could do with it I had no idea, but changed this! Great video
Nice video! That last point really highlighted where I might personally use __new__ in my own projects, really insightful video!
Great to hear!
Quality content, yet again. Idiomatic Python, concise explanations, nothing less. 11/10
thanks shrek
I spent hours on internet, this is the only video/blog explain clearly about __new__ and __ini__ !! Especially the part about when we need to modify __new__(). Thank you so much!!
Dude you're amazing, I've never seen anyone cover python at the level you do. Amazing detailing and explanations. Also , quick and easy to watch! Great stuff! 👍
Hi, I really like your videos and they always help me understand something i didn't know in python! Thank you!
Happy to hear that!
The value of a singleton is when you need an abstraction to something where there is an actual constraint that "there can be only one". The classic example is hardware access. Suppose you open a connection to a device that only supports a single connection (e.g. a COM port), and a set of objects that can access that device. Making the device abstraction a singleton is a great design pattern. Any object can instantiate the device object. If it's the first one, a new object is instantiated. If it's a subsequent construction, the object gets a reference to the already instantiated device.
Brilliant video. You are one of the very few programming channels which are presenting stuff which helps you move from a beginner software developer, to an intermediate one.
oh man - the awkward pause to subscribe was GOLD.
these vids are great - I didn't know about the __new__ method - I'm intrigued!
Sincerely,
-new subscriber
This is very useful for helping people learn about what happens when an object is being created. Nice examples of when to specify your own new method! Great material!! 😁
Man I'm just so happy that such high quality content is for free. Greetings from Germany!
Really loving this channel, always finding new ways to do things. Thank you!
I'm always finding init ways to do things!
Excellent, just as always! The Python level you teach is just another level. Love your videos.
Can you do a tutorial on super function as well? I would really appreciate that.
Yes, that would be good, especially in the case of multiple inheritance. Actually, the do's-and-don't's of multiple inheritance in general would be great
I am always learning new stuff here, you are awesome! thank you for these contents. They really help us a lot.
Happy to hear that!
I love those videos where you look at one single issue and explain it in-depth! while all of those videos may already exist on yt, most are just not as well made. yours are a joy to watch!
So glad to hear your kind words! Thanks!
Insanely great! Thanks for your work!
After many years of daily use, I thought I new everything about python. Your videos never fail to prove otherwise. Thanks for the excellent content!
Great one! This is really high quality content! Can't wait for the video on metaclasses
Excellent video! I have never used __new__ myself, but would consider it now- very well explained, thanks!
Thank you James, great explanation!
I learned something new! (__init_subclass__)
Thanks!
This is a great channel!
Nice video! The examples are really appreciated.
Glad it was helpful!
Great as always. I'm really looking forward to the meta class video, I never really got those.
Had I not seen this last year, today I would have fumbled around with __init__, not understanding what I was doing, and it would have cost me hours. Cheers for another top shelf lesson in the finer points of Python
This channel is the only one on CZcams that actually showcases unique and interesting stuff and doesn't just read out the iter-tools docs.
Many thanks for the kind words. I love reading itertools docs :)
Again a very high quality video! Thanks a lot
Today I learnt about ```__new__```. But I also learnt about ```print(f'{EXPRESSION=}')``` That '=' at the end is rly neat
He covered that in another video. He has great content!
The last example seems like it would be done more simply with a factory pattern too
The comment I made about Factory pattern was meant to apply to all the creational patterns in this video! The main intent of overriding new is subclassing builtin (or C extension) immutable types.
Ive never learn so much with just one python video
You are awesomeeeee, the múltiple examples reaaaally make things way easier to understand and the performace tips are very intresting and usefull
Finally someone explained this, so many people call __init__ a constructor.
init is constructor of the object
new is constructor of the bases :/
@@NoNameAtAll2 I would disagree , considering definition of the class constructor.
I always wondered why the heck we use 'self' in the __init__ method. Now I know! Thanks for this informative video, it helped alot!
I needed this, thank you!
In the Singleton example, do you not have the problem that __init__() will be called multiple times? If __init__() does anything non-trivial, this could be a huge problem. Or does overriding __new__() mean that __init__() might not get called? But that's not what it looks like at 1:20
The code at 1:20 gets called every time someone does Singleton(), so indeed every time you call this, the init will be called, and hence if new is returning a single instance the init will be called on the same object as many times as Singleton() is written. Therefore, if you use the singleton pattern this way, your init should either be empty or at least not modify the object. Most builtin types either have any empty new or an empty init.
I always learn something intimidatingly new here thanks for the video.
I tend to never subscribe to a channel as a response to being asked... But this was both sneaky and fun so now there's an exception to that rule. Also thanks for the clear and concise videos!
So good, glad I found your channel
This might be the only programming channel that I'm seriously considering binging the content of
After this video.. I still have lot of things to learn in Python. This is such a quality video. Thank you.
This channel is so good for me who bury them self into python
a lot of details about how python work and pythonic technique
Love!
Holy shit, although I work everyday with python on an advanced level your videos make me feel like I didn't know anything. I really like your videos since it seems like they are the only resource constantly delivering pro level python tutorials.
You're welcome! Keep watching and you'll learn all the stuff too!
@@mCoding One question: Why did you call object.new in your last example instead of super().new like the examples before. Does this make any difference? As far as I undestand it would make a difference if your Base class iwould inherit something from another class, right?
I'd like to recommend the channel AnthonyWritesCode. The guy behind it maintains Tox, Flake8, and DeadSnakes.
I've been trying to find a good definition of what subclassing entails in python for the last couple of months. I now understand the distinction vs e.g. subclassing a window's render hooks in COM. Thanks!
Subscribing right after the zoom in :P
Hands down, best Python channel out there.
Great content as always. Going to use the EncryptedFile example instead of a proxy class in my code.
9:20 This is actually perfect for a mini-feature I wanted to implement previously (generating a complete list of classes that all derive from the same base class). I still have to manually import all these classes, but at least I can now dynamically generate the list based on imports (I presume) instead of manually editing a list.
You're one of the most if not the most underrated programming channel on CZcams.
Caching clients is a super practical example. I've used this technique numoirous times in production code.
Amazing content. Learning so much 😌
Mind = Blown. Amazing stuff
Please make more videos about design patterns and the context in which they are best used.
Great video!
But I did not understand the drawback of a singleton? Isn't your client example a Singleton too?
Saving this for later. I think I can use that init subclass pattern at work
It's useful because init subclass will be inherited across multiple levels so you don't need to recursively track children, they just automatically call init subclass anyway!
Another great video, James! You're the only one in CZcams that I found that covers advanced Python topics in a simple and clear manner!
Could I request a topic for a video? I never quite understood how can I bundle C code in a Python package like Numpy does, for example. Could you cover that topic in a future video?
Thank you for sharing your knowledge!
Thanks! C extensions are on my list don't worry!
You always makes me feel like I know nothing in python😄😄. I really like this channel.
Exactly!!
While watching @mcoding videos, I feel like I need atleast 2 years of experience in Python, but when I look at the general knowledge about the understanding of Python in companies and market, there is a big difference 😂
@@ashishjain518 😄😄
Nice content 👍 , Thanks for sharing 🙏
The EncryptedFile example reminded me of an Abstract class implementation. I'm not sure if python supports abstract classes the way other languages do, but this reminded me of it.
python supports abstract classes in a kinda unusual way. You need to use an abc module and do something like this:
from abc import ABC, abstractmethod
class Abstract(ABC):
@abstractmethod
def someMethod():
pass
This essentially makes it so you can't create an instance of an abstract class and any class inheriting from it has to define all methods marked as abstract or else it'll cause a TypeError when you try to create an instance of a derived class.
channel is underrated!
Thanks for the content!! Do we consider the Encryption sample as strategy pattern? And I think my question is that, what is the benefit implementing in this way comparing with passing different encryption method into a third class?
i need a complete course from this channel.
I loved the look at the end saying "take the hint" xD
Hello. You mentioned the terms "Factory pattern", what does it mean? You said it was a better suited option to solve the client example you took.
Also, at 9:20, you refer to the dunder method __init_subclass__ as a "hook", could you explain what a hook means? If it is a generic programming term, I'd appreciate if you linked a website or resource explaining it in simple terms. Thanks.
I just want to let you know that this channel is basically single-highhandedly teaching me all of the 'advanced' python techniques/knowledge I know. Obviously I do my own research as well, but there are a lot of things that you just can't look up if you don't know they exist.
Great to hear :) glad you enjoyed my videos!
Nice video with good and concise explanations! I'd only add some thread safety to the singleton example, the current example would potentially lead to race conditions if used in production code.
Thanks! The normal singleton design pattern is not thread-safe, but indeed if your singleton is mutable and you access it from multiple threads then you would need to add some kind of synchronization.
u a monster man, love your videos
Thanks for the love, I appreciate it!
Excellent video!
These videos are all those questions you Google and find answers to on stack overflow.
if you implement some sort of plugin pattern with factory class instances. Using the singelton pattern to solve that is a very good solution. There are also probably more situations where singletons are viable as a solution. Especially if the concrete class that is going to be made is not know for the code that needs to make new instances of an object.
Thumbs up for how clearly you explain these things, however I do feel as there could have been more examples on subclassing immutable data types and restricting their domain. Say if you want to make a class for natural numbers that inherits from int, or a string that's strictly an email address. Doing validation at the type level encourages the "parse, don't validate" design pattern, that I personally think we could use more of.
Quite Fascinating, how adding a = to a print statement would calculate the value as well.
Can you do more content about design patterns in Python?
As someone who isn't really familiar with factory pattern, it'd be cool to see how that last example would be better implemented using factory pattern instead
I've been using Python for a decade and I never ___new___ this!
is there a formal design pattern name for the pattern used in the "Encrypted File Example"? Is that an implicit "factory"?
ah, nvm, just saw your reply to an earlier comment about factory patterns in this vid. thanks!
Great video, bump for algorithm
Much appreciated!
Great stuff
I create classes with static and class methods, then create dunder new to make the class behave like a function. My colleagues don't like it, but I find it quite useful for splitting large functions.
How do you mean this? What are the class methods for if your dunder new just acts like a normal function?
@@comedyclub333 For example: `def _ _ new _ _(cls, a, b): return cls._f2(cls._f1(a), b)`
@@senyai Oh okay so you are basically splitting you function up and encapsulate everything inside this modified class to only expose the first function?
> My colleagues don't like it
I wonder why 🤔
at 9:30, what does prefix="rot13" do? Are you extending from two classes EncryptedFiles and prefix? Not sure what prefix is. Thanks.
These are called metaclass keyword arguments. www.python.org/dev/peps/pep-3115/#specification
It passes the keyword arguments to many functions that the metaclass calls when instantiating the class.
Cool video!
The uppercase tuple is great candidate for a coding challenge. I'll use it on my colleagues.
Roast them then make sure to share the vid with them afterwards :)
What about the prefix argument in the inheritence statement? Can we pass such arguments that way? :o
What is the pattern for the EncryptedFile subclassing?
Thank you for another great video.
The comment mentioning that a singleton has the same issue as a global variable misses the point of a singleton. A singleton is only used when, by design, only one instance is desired. Usually this is a poor design choice, but not always. There are some cases where only one item of a given type should exist.
I agree about using the Factory patter. The book, "Design Patterns" changed the way I design code. I knew some of the contexts. That book provided a way to think and communicate design ideas.
Who were the authors of the book?
Awesome video, thank you.
Quick Q.
At 1:43 you demonstrated that init will not be called if the is instance(x, A) does not identify x as an object of type A.
However later Encrypted file example you are returning subclass objects. Why will this not fail as is instance will identify them as a subclass type, not the type of the class itself, thus not running init?
Hope that question makes sense.
A subclass IS and instance of the super class, so the instance check will pass. Hope this helps!
an annoying issue with the singleton/caching use case is it will call __init__ on your pre-existing object every time, essentially making __init__ unusable, which can be a rather unfortunate gotcha. In these cases it's almost always more clear to use a function/classmethod instead, and maybe use a different language feature to prevent __init__ from being called nakedly.
But if you don’t put anything in your init method, then it will do nothing. Put all the object setup in the new method instead.
Hello James, do you think you could explain one day the entire process of object creation? Because I suppose, that there must be something like type.__call__() function in the very beginning of the object creation process. I tried to do my research and even implement some stuff, but haven´t succeeded getting past some kind of "super().__call__()" in my metaclass - didn´t know, what to put as parameters etc.
Also, one more thing. When I inspect the frame one step back of the __new__ , it shows the scope of where the cls(args, kwargs) was called, which I find surprising, since I would expect something like '.__call__' or something. Something cursed is going on here.
I didn't even know __new__ existed, seems really niche, but potentially useful
Hi James,
You mentioned (5:20) that the singleton is a way to do a global configuration object. You said personally that you can't recommend it. Do you have a better way in any of your videos?
Thanks
Hi Darian! Global configuration is one of those things that I've yet to find "the one true answer" for, unfortunately. Due to the nature of it being global it often times does make sense to have it as some kind of global object, though the singleton pattern specifically is particularly confusing for many developers and it also makes testing difficult. I would say that just having a single global instance is often times a decent solution. Note that a single global instance is different from a singleton because you could theoretically make another instance of the global config, you just choose not to, whereas with a singleton you would have to do some class surgery to make another instance (typically necessary during testing code).
@@mCoding Thanks James. I've been using a class and loading up all the class variables for a global config for things like credentials and app options. Then using an instance of the class to encrypt/decrypt passwords etc.. but the singleton sounds like it will do the same job. From my googling, everyone seems to agree that there is no good solution either. Maybe a topic for a video?
Thanks for showing me the existence of ___init_subclass___
Very welcome!
The main topic of this video was definitely interesting, but at 6:30, I learned that you can make an f string print the expression as well as the evaluation by adding an = inside the curly braces.
Now I'll need to actually read the documentation on f strings to see if there are any other special bits like that. I just thought it looked at the content of braces, evaluated them, concatenated the string, and then printed it, but apparently it does a lot more than that, since {x is y =} should give a syntax error.
EDIT: and now I see that you have a video on that very topic. Watching now.
I have a feeling you will enjoy a lot of my videos :)
took me a while due to mistake, but it works thanks
Really cool! The singleton pattern was the first thing that came to mind for me when I learned about the __new__() method. However, there is a small problem that other languages don't have.
When you return the an instance in __new__(), __init__() will ALWAYS be called (if the type returned is correct). This means that in the singleton pattern, whenever you use the constructor to get the existing instance, it will be reinitialized, unless you explicitly add a check at the start of your implementation so it is only initialized once. Here is an example of a complete implementation of the singleton pattern circumventing that issue:
```
class Singleton:
__instance = None
__initialized = False
def __new__(cls):
if cls.__instance is None:
cls.__instance = super(Test, cls).__new__(cls)
return cls.__instance
def __init__(self):
# Check if the class has been initialized
if self.__initialized:
return
# Mark the class as initialized, so that the initialization code
# doesn't run more than once when the class is instantiated again
self.__initialized = True
# Actual initialization code
self.attr = 1
```
You could also override the dunder call on the metaclass so avoid this issues.
Thanks
May I ask for a video to develop functions around numpy and pandas? It's a bit of a pain to check inputs, decide on what to return (do I return always an array or do I try to return a series/dataframe when I receive one of those?), etc.
It depends on what you're doing, but maybe it's better to return the DF in case you need it and use .to_array() (or something like that, I forgot) if you need a numpy array
So, generally speaking __new__ main use is too make the code execute faster, right? I've gotta check what other parts of Python allow me to do that.
Most often, __new__ is used to inherit from a builtin immutable type. There are some very niche cases (e.g. defining Cython extension classes) where it can make things faster, but this is not usually its purpose.
Singletons are also super hard to test because of tight coupling and difficulty to mock in tests.