PEP 771 – Default Extras for Python Software Packages

PEP 771 – Default Extras for Python Software Packages

Author:

Thomas Robitaille <thomas.robitaille at gmail.com>, Jonathan Dekhtiar <jonathan at dekhtiar.com>

Sponsor:

Pradyun Gedam <pradyunsg at gmail.com>

Discussions-To:

Discourse thread

Status:

Draft

Type:

Standards Track

Topic:

Packaging

Created:

13-Jan-2025

Post-History:

15-Jan-2025, 06-Feb-2025, 09-Jun-2025

---

Table of Contents

• Abstract
• Motivation
  • Recommended but not required dependencies
  • Packages supporting multiple backends or frontends
• Rationale
• Specification
  • Default-Extra Metadata Field
  • New key in [project] metadata table
  • Overriding default extras
  • Examples
    • Recommended dependencies and minimal installations
    • Packages requiring at least one backend or frontend
    • Supporting extras that should not remove default extras
    • Packages with multiple kinds of defaults
• Backward Compatibility
  • Packages not using default extras
  • Packages using default extras
  • Packaging-related tools
• Security Implications
• How to teach this
  • Package end users
  • Package authors
    • Supporting older versions of package installers
    • Avoiding the addition of many default dependencies
    • Inheriting from default extras
    • Incompatible extras
    • Circular dependencies
    • Documenting packages with default extras
  • Packaging repository maintainers
• Reference Implementation
• Rejected Ideas
  • Using a meta-package for recommended installations
    • Mismatch between package and module name
    • Multiple repositories or monorepos
    • Depending on the minimal package
    • Uninstallation
    • Package distributions
    • Synchronizing metadata
    • Summary
  • Syntax for deselecting extras
  • Adding a special entry in extras_require
  • Relying on tooling to deselect any default extras
• Copyright

Abstract

PEP 508 specifies a mini-language for declaring package dependencies. One feature of this language is the ability to specify extras, which are optional components of a distribution that, when used, install additional dependencies. This PEP proposes a mechanism to allow one or more extras to be installed by default if none are provided explicitly.

Motivation

Various use cases for default extras and possible solutions in this PEP were discussed extensively on this DPO thread. These fall into two broad cases that provide the motivation for the present PEP.

Recommended but not required dependencies

Package maintainers often use extras to declare optional dependencies that extend the functionality or performance of a package. In some cases, it can be difficult to determine which dependencies should be required and which should be categorized as extras. A balance must be struck between the needs of typical users, who may prefer most features to be available by default, and users who want minimal installations without large, optional dependencies. One solution with existing Python packaging infrastructure is for package maintainers to define an extra called, for example, recommended, which includes all non-essential but suggested dependencies. Users are then instructed to install the package using package[recommended], while those who prefer more control can use package. However, in practice, many users are unaware of the [recommended] syntax, placing the burden on them to know this for a typical installation. Having a way to have recommended dependencies be installed by default while providing a way for users to request a more minimal installation would satisfy this use case, and this PEP describes a solution to this.

Examples of packages that demonstrate this pattern by encouraging users to include extra dependencies by default include:

• astropy: astropy[recommended]
• sunpy: sunpy[all]
• fastapi: fastapi[standard]

Packages supporting multiple backends or frontends

Another common use case for using extras is to define different backends or frontends and dependencies that need to be installed for each backend or frontend. A package might need at least one backend or frontend to be installed in order to be functional, but may be flexible on which backend or frontend this is. Concrete examples of such frontends or backends include:

• The Qt frontend library, which can be provided by PyQt5, PyQt6, PySide2, or PySide6
• Backend databases, such as MySQL, PostgreSQL, SQLite

With current packaging standards, maintainers have to either require one of the backends or frontends or require users to always specify extras, e.g., package[backend], and therefore risk users having an unusable installation if they only install package. Having a way to specify one or more default backend or frontend and providing a way to override these defaults would provide a much better experience for users, and the approach described in this PEP will allow this.

Note that this PEP does not aim to address the issue of disallowing conflicting or incompatible extras, for example if a package requires exactly one frontend or backend package. There is currently no mechanism in Python packaging infrastructure to disallow conflicting or incompatible extras to be installed, and this PEP does not change that.

Examples of packages that require at least one backend or frontend to work and recommend a default extra to install a backend or frontend include:

• kivy: kivy[base]
• napari: napari[all]
• glueviz: glueviz[qt]

In all three cases, installing the package without any extras results in a broken installation, and this is a commonly reported support issue for some of these packages.

Rationale

A number of possible solutions have been discussed extensively by the community for several years, including in this DPO thread as well as in numerous issues and pull requests. The solution that is presented below:

• is an opt-in solution which means that package maintainers can choose whether or not to use it
• is flexible enough to accommodate both of the major use cases described in Motivation
• re-uses the syntax from PEP 508

It is the only solution, out of all those discussed, that meets all three criteria.

Specification

Default-Extra Metadata Field

A new multiple-use metadata field, Default-Extra, will be added to the core package metadata. For this field, each entry must be a string specifying an extra that will be automatically included when the package is installed without any extras being specified explicitly.

Only entries already specified in a Provides-Extra entry can be used in a Default-Extra entry.

Examples:

Default-Extra: recommended
Default-Extra: backend1
Default-Extra: backend2
Default-Extra: backend3

Since this introduces a new field in the core package metadata, this will require Metadata-Version to be bumped to the next minor version (2.5 at the time of writing).

New key in [project] metadata table

A new key will be added to the [project] table in project metadata as originally defined in PEP 621 and now defined in the PyPA specifications. This key will be named default-optional-dependency-keys with the following description:

• TOML type: Array of strings
• Corresponding core metadata field: Default-Extra

Each string in