Python Descriptors: An Introduction

Python Descriptors: An Introduction

by Davide Mastromatteo 21m intermediate python

Table of Contents

Remove ads

Recommended Course

Python Descriptors (1h 3m)

Descriptors are a specific Python feature that power a lot of the magic hidden under the language’s hood. If you’ve ever thought that Python descriptors are an advanced topic with few practical applications, then this tutorial is the perfect tool to help you understand this powerful feature. You’ll come to understand why Python descriptors are such an interesting topic, and what kind of use cases you can apply them to.

By the end of this tutorial, you’ll know:

  • What Python descriptors are
  • Where they’re used in Python’s internals
  • How to implement your own descriptors
  • When to use Python descriptors

This tutorial is intended for intermediate to advanced Python developers as it concerns Python internals. However, if you’re not at this level yet, then just keep reading! You’ll find useful information about Python and the lookup chain.

Take the Quiz: Test your knowledge with our interactive “Python Descriptors: An Introduction” quiz. You’ll receive a score upon completion to help you track your learning progress:

Python Descriptors: An Introduction

Interactive Quiz

Python Descriptors: An Introduction

Learn how Python descriptors control attribute access, use special methods for getting and setting values, and implement read-only attributes.

What Are Python Descriptors?

Descriptors are Python objects that implement a method of the descriptor protocol, which gives you the ability to create objects that have special behavior when they’re accessed as attributes of other objects. Here you can see the correct definition of the descriptor protocol:

Python 
__get__(self, obj, type=None) -> object
__set__(self, obj, value) -> None
__delete__(self, obj) -> None
__set_name__(self, owner, name)

If your descriptor implements just .__get__(), then it’s said to be a non-data descriptor. If it implements .__set__() or .__delete__(), then it’s said to be a data descriptor. Note that this difference is not just about the name, but it’s also a difference in behavior. That’s because data descriptors have precedence during the lookup process, as you’ll see later on.

Take a look at the following example, which defines a descriptor that logs something on the console when it’s accessed:

Python 
# descriptors.py
class Verbose_attribute():
    def __get__(self, obj, type=None) -> object:
        print("accessing the attribute to get the value")
        return 42
    def __set__(self, obj, value) -> None:
        print("accessing the attribute to set the value")
        raise AttributeError("Cannot change the value")

class Foo():
    attribute1 = Verbose_attribute()

my_foo_object = Foo()
x = my_foo_object.attribute1
print(x)

In the example above, Verbose_attribute() implements the descriptor protocol. Once it’s instantiated as an attribute of Foo, it can be considered a descriptor.

As a descriptor, it has binding behavior when it’s accessed using dot notation. In this case, the descriptor logs a message on the console every time it’s accessed to get or set a value:

  • When it’s accessed to .__get__() the value, it always returns the value 42.
  • When it’s accessed to .__set__() a specific value, it raises an AttributeError exception, which is the recommended way to implement read-only descriptors.

Now, run the example above and you’ll see the descriptor log the access to the console before returning the constant value:

Shell 
$ python descriptors.py
accessing the attribute to get the value
42

Here, when you try to access attribute1, the descriptor logs this access to the console, as defined in .__get__().

How Descriptors Work in Python’s Internals

If you have experience as an object-oriented Python developer, then you may think that the previous example’s approach is a bit of overkill. You could achieve the same result by using properties. While this is true, you may be surprised to know that properties in Python are just… descriptors! You’ll see later on that properties are not the only feature that make use of Python descriptors.

Python Descriptors in Properties

If you want to get the same result as the previous example without explicitly using a Python descriptor, then the most straightforward approach is to use a property. The following example uses a property that logs a message to the console when it’s accessed:

Python 
# property_decorator.py
class Foo():
    @property
    def attribute1(self) -> object:
        print("accessing the attribute to get the value")
        return 42

    @attribute1.setter
    def attribute1(self, value) -> None:
        print("accessing the attribute to set the value")
        raise AttributeError("Cannot change the value")

my_foo_object = Foo()
x = my_foo_object.attribute1
print(x)

The example above makes use of decorators to define a property with attached getter and setter methods. But as you may know, decorators are just syntactic sugar. The example before, in fact, can be written as follows:

Python 
# property_function.py
class Foo():
    def getter(self) -> object:
        print("accessing the attribute to get the value")
        return 42

    def setter(self, value) -> None:
        print("accessing the attribute to set the value")
        raise AttributeError("Cannot change the value")

    attribute1 = property(getter, setter)

my_foo_object = Foo()
x = my_foo_object.attribute1
print(x)

Now you can see that the property has been created by using property(). The signature of this function is as follows:

Python 
property(fget=None, fset=None, fdel=None, doc=None) -> object

property() returns a property object that implements the descriptor protocol. It uses the parameters fget, fset and fdel for the actual implementation of the three methods of the protocol.

Python Descriptors in Methods and Functions

If you’ve ever written an object-oriented program in Python, then you’ve certainly used methods. These are regular functions that have the first argument reserved for the object instance. When you access a method using dot notation, you’re calling the corresponding function and passing the object instance as the first parameter.

The magic that transforms your obj.method(*args) call into method(obj, *args) is inside a .__get__() implementation of the function object that is, in fact, a non-data descriptor. In particular, the function object implements .__get__() so that it returns a bound method when you access it with dot notation. The (*args) that follow invoke the functions by passing all the extra arguments needed.

To get an idea for how it works, take a look at this pure Python example from the official docs:

Python 
import types

class Function(object):
    ...
    def __get__(self, obj, objtype=None):
        "Simulate func_descr_get() in Objects/funcobject.c"
        if obj is None:
            return self
        return types.MethodType(self, obj)

In the example above, when the function is accessed with dot notation, .__get__() is called and a bound method is returned.

This works for regular instance methods just like it does for class methods or static methods. So, if you call a static method with obj.method(*args), then it’s automatically transformed into method(*args). Similarly, if you call a class method with obj.method(type(obj), *args), then it’s automatically transformed into method(type(obj), *args).

In the official docs, you can find some examples of how static methods and class methods would be implemented if they were written in pure Python instead of the actual C implementation. For instance, a possible static method implementation could be this:

Python 
class StaticMethod(object):
    "Emulate PyStaticMethod_Type() in Objects/funcobject.c"
    def __init__(self, f):
        self.f = f

    def __get__(self, obj, objtype=None):
        return self.f

Likewise, this could be a possible class method implementation:

Python 
class ClassMethod(object):
    "Emulate PyClassMethod_Type() in Objects/funcobject.c"
    def __init__(self, f):
        self.f = f

    def __get__(self, obj, klass=None):
        if klass is None:
            klass = type(obj)
        def newfunc(*args):
            return self.f(klass, *args)
        return newfunc

Note that, in Python, a class method is just a static method that takes the class reference as the first argument of the argument list.

How Attributes Are Accessed With the Lookup Chain

To understand a little more about Python descriptors and Python internals, you need to understand what happens in Python when an attribute is accessed. In Python, every object has a built-in .__dict__ attribute. This is a dictionary that contains all the attributes defined in the object itself. To see this in action, consider the following example:

Python 
class Vehicle():
    can_fly = False
    number_of_wheels = 0

class Car(Vehicle):
    number_of_wheels = 4

    def __init__(self, color):
        self.color = color

my_car = Car("red")
print(my_car.__dict__)
print(type(my_car).__dict__)

This code creates a new object and prints the contents of the .__dict__ attribute for both the object and the class. Now, run the script and analyze the output to see the .__dict__ attributes set:

Shell 
{'color': 'red'}
{'__module__': '__main__', 'number_of_wheels': 4, '__init__': <function Car.__init__ at 0x10fdeaea0>, '__doc__': None}