Python's Magic Methods: Leverage Their Power in Your Classes

Python's Magic Methods: Leverage Their Power in Your Classes

by Leodanis Pozo Ramos 55m advanced python

Table of Contents

Remove ads

Recommended Course

Python's Magic Methods in Classes (1h 38m)

As a Python developer who wants to harness the power of object-oriented programming, you’ll love to learn how to customize your classes using special methods, also known as magic methods or dunder methods. A special method is a method whose name starts and ends with a double underscore. These methods have special meanings for Python.

Python automatically calls magic methods as a response to certain operations, such as instantiation, sequence indexing, attribute managing, and much more. Magic methods support core object-oriented features in Python, so learning about them is fundamental for you as a Python programmer.

In this tutorial, you’ll:

  • Learn what Python’s special or magic methods are
  • Understand the magic behind magic methods in Python
  • Customize different behaviors of your custom classes with special methods

To get the most out of this tutorial, you should be familiar with general Python programming. More importantly, you should know the basics of object-oriented programming and classes in Python.

Take the Quiz: Test your knowledge with our interactive “Python's Magic Methods: Leverage Their Power in Your Classes” quiz. You’ll receive a score upon completion to help you track your learning progress:

Python's Magic Methods: Leverage Their Power in Your Classes

Interactive Quiz

Python's Magic Methods: Leverage Their Power in Your Classes

In this quiz, you'll test your understanding of Python's magic methods. These special methods are fundamental to object-oriented programming in Python, allowing you to customize the behavior of your classes.

Getting to Know Python’s Magic or Special Methods

In Python, special methods are also called magic methods, or dunder methods. This latter terminology, dunder, refers to a particular naming convention that Python uses to name its special methods and attributes. The convention is to use double leading and trailing underscores in the name at hand, so it looks like .__method__().

The double underscores flag these methods as core to some Python features. They help avoid name collisions with your own methods and attributes. Some popular and well-known magic methods include the following:

Special Method Description
.__init__() Provides an initializer in Python classes
.__str__() and .__repr__() Provide string representations for objects
.__call__() Makes the instances of a class callable
.__len__() Supports the len() function

This is just a tiny sample of all the special methods that Python has. All these methods support specific features that are core to Python and its object-oriented infrastructure.

Here’s how the Python documentation defines the term special methods:

A method that is called implicitly by Python to execute a certain operation on a type, such as addition. Such methods have names starting and ending with double underscores. (Source)

There’s an important detail to highlight in this definition. Python implicitly calls special methods to execute certain operations in your code. For example, when you run the addition 5 + 2 in a REPL session, Python internally runs the following code under the hood:

Python 
>>> (5).__add__(2)
7

The .__add__() special method of integer numbers supports the addition that you typically run as 5 + 2.

Reading between the lines, you’ll realize that even though you can directly call special methods, they’re not intended for direct use. You shouldn’t call them directly in your code. Instead, you should rely on Python to call them automatically in response to a given operation.

Magic methods exist for many purposes. All the available magic methods support built-in features and play specific roles in the language. For example, built-in types such as lists, strings, and dictionaries implement most of their core functionality using magic methods. In your custom classes, you can use magic methods to make callable objects, define how objects are compared, tweak how you create objects, and more.

Note that because magic methods have special meaning for Python itself, you should avoid naming custom methods using leading and trailing double underscores. Your custom method won’t trigger any Python action if its name doesn’t match any official special method names, but it’ll certainly confuse other programmers. New dunder names may also be introduced in future versions of Python.

Magic methods are core to Python’s data model and are a fundamental part of object-oriented programming in Python. In the following sections, you’ll learn about some of the most commonly used special methods. They’ll help you write better object-oriented code in your day-to-day programming adventure.

Controlling the Object Creation Process

When creating custom classes in Python, probably the first and most common method that you implement is .__init__(). This method works as an initializer because it allows you to provide initial values to any instance attributes that you define in your classes.

The .__new__() special method also has a role in the class instantiation process. This method takes care of creating a new instance of a given class when you call the class constructor. The .__new__() method is less commonly implemented in practice, though.

In the following sections, you’ll learn the basics of how these two methods work and how you can use them to customize the instantiation of your classes.

Initializing Objects With .__init__()

Python calls the .__init__() method whenever you call the constructor of a given class. The goal of .__init__() is to initialize any instance attribute that you have in your class. Consider the following Point class:

Python 
>>> class Point:
...     def __init__(self, x, y):
...         self.x = x
...         self.y = y
...

>>> point = Point(21, 42)
>>> point.x
21
>>> point.y
42

When you create the point instance by calling the class constructor, Point(), Python automatically calls .__init__() under the hood using the same arguments that you’ve passed to the constructor. You don’t have to call .__init__() by yourself. You just rely on Python’s implicit behavior. Note how the .x and .y attributes hold the values that you pass in when you call the constructor.

Creating Objects With .__new__()

When you call a class constructor to create a new instance of a class, Python implicitly calls the .__new__() method as the first step in the instantiation process. This method is responsible for creating and returning a new empty object of the underlying class. Python then passes the just-created object to .__init__() for initialization.

The default implementation of .__new__() is enough for most practical use cases. So, you probably won’t need to write a custom implementation of .__new__() in most cases.

However, you’ll find that this method is useful in some advanced use cases. For example, you can use .__new__() to create subclasses of immutable types, such as int, float, tuple, and str. Consider the following example of a class that inherits from the built-in float data type:

Python 
>>> class Storage(float):
...     def __new__(cls, value, unit):
...         instance = super().__new__(cls, value)
...         instance.unit = unit
...         return instance
...

In this example, you’ll note that .__new__() is a class method because it gets the current class (cls) rather than the current instance (self) as an argument.

Then, you run three steps. First, you create a new instance of the current class, cls, by calling .__new__() on the float class through the built-in super() function. This call creates a new instance of float and initializes it using value as an argument.

Then, you customize the new instance by dynamically attaching a .unit attribute to it. Finally, you return the new instance to meet the default behavior of .__new__().

Here’s how this class works in practice:

Python 
>>> disk = Storage(1024, "GB")

>>> disk
1024.0
>>> disk.unit
'GB'

>>> isinstance(disk, float)
True

Your Storage class works as expected, allowing you to use an instance attribute to store the unit in which you’re measuring the storage space. The .unit attribute is mutable, so you can change its value anytime you like. However, you can’t change the numeric value of disk because it’s immutable like its parent type, float.

Representing Objects as Strings

A common task that you’ll perform in your Python code is to display data or produce output. In some cases, your programs may need to display useful information to the user. In other cases, you may need to show information to other programmers using your code.

These two types of output can be pretty different because their respective target audiences can have different needs. The information for the user may not need to highlight implementation details. It may just need to be user-friendly and clear.

In contrast, the information for an audience of programmers may need to be technically sound, providing details of your code’s implementation.

Python has you covered in both scenarios. If you want to provide user-friendly output, then you can use the .__str__() method. On the other hand, when you need to provide developer-friendly output, then you can use the .__repr__() method. These methods support two different string representations for Python objects.

Along with .__init__(), the .__str__() and .__repr__() methods are arguably the most commonly used special methods in custom classes. In the following sections, you’ll learn about these two useful methods.

User-Friendly String Representations With .__str__()

The .__str__() special method returns a human-readable string representation of the object at hand. Python calls this method when you call the built-in str() function, passing an instance of the class as an argument.

Python also calls this method when you use the instance as an argument to the print() and format() functions. The method is meant to provide a string that’s understandable by the end user of the application.

To illustrate how to use the .__str__() method, consider the following Person class:

Python person.py 
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def __str__(self):
        return f"I'm {self.name}, and I'm {self.age} years old."

This class has two attributes, the person’s name and age. In the .__str__() method, you return a user-friendly string representation of a person. This representation includes the name and the age in a descriptive message.

Here’s an example of how your class works:

Python 
>>> from person import Person

>>> jane = Person("Jane Doe", 25)

>>> str(jane)
"I'm Jane Doe, and I'm 25 years old."

>>> print(jane)
I'm Jane Doe, and I'm 25 years old.

When you use an instance of Person as an argument to str() or print(), you get a string representation of the object on your screen. This representation is specially crafted for the user of your application or code.

Developer-Friendly String Representations With .__repr__()

The .__repr__() method returns a string representation of an object that’s targeted at the developer. Ideally, the string returned should be crafted in such a way that you can construct an identical instance of the object from the string.

To dive into how .__repr__() works, go back to the Person class and update it as in the code snippet below:

Python person.py 
class Person:
    # ...

    def __repr__(self):
        return f"{type(self).__name__}(name='{self.name}', age={self.age})"

In .__repr__(), you use an f-string to build the object’s string representation. The first part of this representation is the class’s name. Then, you use a pair of parentheses to wrap the constructor’s arguments and their corresponding values.

Now restart your REPL session and run the following code:

Python 
>>> from person import Person

>>> john = Person("John Doe", 35)

>>> john
Person(name='John Doe', age=35)

>>> repr(john)
"Person(name='John Doe', age=35)"

In this example, you create an instance of Person. Then, you access the instance directly through the REPL. The standard REPL automatically uses the .__repr__() method to display immediate feedback about an object.

In the last example, you use the built-in repr() function, which falls down to calling .__repr__() on the object that you pass in as an argument.

Note that you can copy and paste the resulting string representation, without the quotations, to reproduce the current object. This string representation is pretty handy for other developers using your code. From this representation, they can obtain critical information about how the current object is constructed.

Supporting Operator Overloading in Custom Classes

Python has multiple types of operators, which are special symbols, combinations of symbols, or keywords that designate some type of computation. Internally, Python supports operators with special methods. For example, the .__add__() special method supports the plus operator (+) as you already saw.

In practice, you’ll take advantage of these methods that are behind operators for something that’s known as operator overloading.

Operator overloading means providing additional functionality to the operators. You can do this with most built-in types and their specific supported operators. However, that’s not all you can do with the special methods that support Python operators. You can also use these methods to support some operators in your custom classes.

In the following sections, you’ll learn about the specific special methods that support Python operators, including arithmetic, comparison, membership, bitwise, and augmented operators. To kick things off, you’ll start with arithmetic operators, which are arguably the most commonly used operators.

Arithmetic Operators

Arithmetic operators are those that you use to perform arithmetic operations on numeric values. In most cases, they come from math, and therefore, Python represents them with the usual math signs.

In the following table, you’ll find a list of the arithmetic operators in Python and their supporting magic methods:

Operator Supporting Method
+ .__add__(self, other)
- .__sub__(self, other)
* .__mul__(self, other)
/ .__truediv__(self, other)
// .__floordiv__(self, other)
% .__mod__(self, other)
** .__pow__(self, other[, modulo])

Note that all these methods take a second argument called other. In most cases, this argument should be the same type as self or a compatible type. If that’s not the case, then you can get an error.

To illustrate how you can overload some of these operators, get back to the Storage class. Say that you want to make sure that when you add or subtract two instances of this class, both operands have the same unit:

Python storage.py 
class Storage(float):
    def __new__(cls, value, unit):
        instance = super().__new__(cls, value)
        instance.unit = unit
        return instance

    def __add__(self, other):
        if not isinstance(other, type(self)):
            raise TypeError(
                "unsupported operand for +: "
                f"'{type(self).__name__}' and '{type(other).__name__}'"
            )
        if not self.unit == other.unit:
            raise TypeError(
                f"incompatible units: '{self.unit}' and '{other.unit}'"
            )

        return type(self)(super().__add__(other), self.unit)

In the .__add__() method, you first check if other is also an instance of Storage. If not, then you raise a TypeError exception with an appropriate error message. Next, you check if both objects have the same unit. If not, then you raise a TypeError again. If both checks pass, then you return a new instance of Storage that you create by adding the two values and attaching the unit.

Here’s how this class works:

Python 
>>> from storage import Storage

>>> disk_1 = Storage(500, "GB")
>>> disk_2 = Storage(1000, "GB")
>>> disk_3 = Storage(1, "TB")

>>> disk_1 + disk_2
1500.0

>>> disk_2 + disk_3
Traceback (most recent call last):
    ...
TypeError: incompatible units: 'GB' and 'TB'

>>> disk_1 + 100
Traceback (most recent call last):
    ...
TypeError: unsupported operand for +: 'Storage' and 'int'

In this example, when you add two objects that have the same unit, you get the correct value. If you add two objects with different units, then you get a TypeError telling you that the units are incompatible. Finally, when you try to add a Storage instance with a built-in numeric value, you get an error as well because the built-in type isn’t a Storage instance.

As an exercise, would you like to implement the .__sub__() method in your Storage class? Check the collapsible section below for a possible implementation:

Here’s a possible implementation of .__sub__() in the Storage class:

Python storage.py 
class Storage(float):
    def __new__(cls, value, unit):
        instance = super().__new__(cls, value)
        instance.unit = unit
        return instance

    def __add__(self, other):
        if not isinstance(other, type(self)):
            raise TypeError(
                "unsupported operand for +: "
                f"'{type(self).__name__}' and '{type(other).__name__}'"
            )
        if not self.unit == other.unit:
            raise TypeError(
                f"incompatible units: '{self.unit}' and '{other.unit}'"
            )

        return type(self)(super().__add__(other), self.unit)

    def __sub__(self, other):
        if not isinstance(other, type(self)):
            raise TypeError(
                "unsupported operand for -: "
                f"'{type(self).__name__}' and '{type(other).__name__}'"
            )
        if not self.unit == other.unit:
            raise TypeError(
                f"incompatible units: '{self.unit}' and '{other.unit}'"
            )

        return type(self)(super().__sub__(other), self.unit)

Note that .__add__() and .__sub__() have several lines in common, which makes for repetitive code. You can fix this by using a helper method to extract the common logic.

In the above example, you overloaded the addition operator (+) on the built-in float type. You can also use the .__add__() method and any other operator method to support the corresponding operator in a custom class.

Consider the following example of a Distance class:

Python distance.py 
class Distance:
    _multiples = {
        "mm": 0.001,
        "cm": 0.01,
        "m": 1,
        "km": 1_000,
    }

    def __init__(self, value, unit="m"):
        self.value = value
        self.unit = unit.lower()

    def to_meter(self):
        return self.value * type(self)._multiples[self.unit]

    def __add__(self, other):
        return self._compute(other, "+")

    def __sub__(self, other):
        return self._compute(other, "-")

    def _compute(self, other, operator):
        operation = eval(f"{self.to_meter()} {operator} {other.to_meter()}")
        cls = type(self)
        return cls(operation / cls._multiples[self.unit], self.unit