Understanding __new__ and __init__ magic methods in Python

The purpose of this post is to discuss __new__ and __init__ methods in Python. The __new__ and __init__ methods behave differently between themselves and between the old-style versus new-style python class definitions.

Understanding the difference between __new__ and __init__

The major difference between these two methods is that __new__ handles object creation and __init__ handles object initialization. During instantiation, these two have slight differences in how they work, when defined differently.

A bit about classes in python

Classes in python are classified as new-style and old-style. Old-style classes are the ones that are prior to Python 3 and defined without inheriting from the base class object, which in turn is inherited from type by default.

Old-style class in python 2.x :

class A:  # -> inherits from 'type'
    pass

New-style class in python 2.x:

class A(object):  # -> clearly inherits from 'object'
    pass

In python 3 there aren’t new or old styles of classes and they inherit directly from ‘object’ so there is no need to specify it as a base anymore.  The object base class brings methods/properties that are common to all new-style classes.
Throughout the rest of the article, we will examine the __new__ and __init__ methods in both cases to determine how they behave and how we can use them.

Treating different cases

Before diving into the actual implementations you need to know that __new__ accepts cls as its first parameter and __init__ accepts self, because when calling __new__ you actually don’t have an instance yet, therefore no self exists at that moment, whereas __init__ is called after __new__ and the instance is in place, so you can use self with it.

__new__ and __init__ for old style classes

Old-style classes don’t actually have a __new__ method because for them __init__ is the constructor, so basically if we would have:

class A:
    def __new__(cls):
        print ("A.__new__ is called")  # -> this is never called

A()

the body of __new__ will never be executed in this case because it is not the purpose for old-style classes.

If we were to override __init__ instead:

class A:

    def __init__(self):
        print ("A.__init__ called")

A()

the output would be:

A.__init__ called

Now let’s try returning something from __init__:

class A:
    def __init__(self):
        return 29

A()

this would yield:

TypeError: __init__() should return None​

so that means we can’t actually control what to return when instantiating the old-style class.

__new__ and __init__ for new-style classes

The new-style classes let the developer override both __new__ and __init__ and they have distinct purposes, __new__ (the constructor) is solely for creating the object and __init__ (the initializer) for initializing it.

Let’s see the order of their execution:

class A(object):  # -> don't forget the object specified as base

    def __new__(cls):
        print ("A.__new__ called")
        return super(A, cls).__new__(cls)

    def __init__(self):
        print ("A.__init__ called")

A()

the output will be:

A.__new__ called
A.__init__ called

You may be wondering from where is __init__ or __new__ called and what I have for you is that __new__ is called automatically when calling the class name (when instantiating), whereas __init__ is called every time an instance of the class is returned by __new__ passing the returned instance to __init__ as the self parameter, therefore even if you were to save the instance somewhere globally/statically and return it every time from __new__, then __init__ will be called every time you do just that.
Knowing this means that if we were to omit to call super for __new__ then __init__ won’t be executed. Let’s see if that’s the case:

class A(object):

    def __new__(cls):
        print ("A.__new__ called")

    def __init__(self):
        print ("A.__init__ called")  # -> is actually never called

print A()

whose output is:

A.__new__ called
None

Obviously, the instantiation is evaluated to None since we don’t return anything from the constructor.

Wondering what might happen if we were to return something from __new__? 

class A(object):

    def __new__(cls):
        print ("A.__new__ called")
        return 29

print A()

guess what the output is:

A.__new__ called
29

Let’s see what happens when we try to return from __init__:

class A(object):

    def __init__(self):
        print ("A.__init__ called")
            return 33  # -> TypeError: __init__ should return None

A()

which yields:

TypeError: __init__ should return None

This is mainly because the handler that calls __init__ raises the TypeError exception and it wouldn’t even make sense to return anything from __init__ since its purpose is just to alter the fresh state of the newly created instance.

It seems the new-style classes have much to offer in terms of flexibility, allowing us to do any PRE/POST operations at both creation and initialization levels and leaving us control over what we return at instantiation time.
Considering that, let’s try and return an instance of a different class. First, we define our alternative class:

class Sample(object):
    def __str__(self):
        return "SAMPLE"

And then we define our __new__ overriding class:

class A(object):
    def __new__(cls):
        return Sample()

which could also be written as:

class A(object):
    def __new__(cls):
        return super(A, cls).__new__(Sample)

and then call:

print A()

that would output:

SAMPLE

References

https://docs.python.org/2/library/functions.html#objecthttp://www.cafepy.com/article/python_types_and_objects/python_types_and_objects.html#the-object-withinhttps://wiki.python.org/moin/NewClassVsClassicClass

https://spyhce.com/blog/understanding-new-and-init