Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 725 – Specifying external dependencies in pyproject.toml

Author:
Pradyun Gedam <pradyunsg at gmail.com>, Jaime Rodríguez-Guerra <jaime.rogue at gmail.com>, Ralf Gommers <ralf.gommers at gmail.com>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Topic:
Packaging
Created:
17-Aug-2023
Post-History:
18-Aug-2023, 22-Sep-2025
Table of Contents

Abstract

This PEP specifies how to write a project’s external, or non-PyPI, build and runtime dependencies in a pyproject.toml file for packaging-related tools to consume.

This PEP proposes to add an [external] table to pyproject.toml with seven keys. “build-requires”, “host-requires” and “dependencies” are for specifying three types of required dependencies:

  1. build-requires, build tools to run on the build machine
  2. host-requires, build dependencies needed for the host machine but also needed at build time.
  3. dependencies, needed at runtime on the host machine but not needed at build time.

These three keys also have their optional external counterparts (optional-build-requires, optional-host-requires, optional-dependencies), which have the same role that project.optional-dependencies plays for project.dependencies. Finally, dependency-groups offers the same functionality as PEP 735 but for external dependencies.

Cross compilation is taken into account by distinguishing between build and host dependencies. Optional build-time and runtime dependencies are supported too, in a manner analogous to how that is supported in the [project] table.

Motivation

Python packages may have dependencies on build tools, libraries, command-line tools, or other software that is not present on PyPI. Currently there is no way to express those dependencies in standardized metadata [1], [2]. Key motivators for this PEP are to:

  • Enable tools to automatically map external dependencies to packages in other packaging repositories,
  • Make it possible to include needed dependencies in error messages emitted by Python package installers and build frontends,
  • Provide a canonical place for package authors to record this dependency information.

Packaging ecosystems like Linux distros, conda, Homebrew, Spack, and Nix need full sets of dependencies for Python packages, and have tools like pyp2spec (Fedora), Grayskull (conda), and dh_python (Debian) which attempt to automatically generate dependency metadata for their own package managers from the metadata in upstream Python packages. External dependencies are currently handled manually, because there is no metadata for this in pyproject.toml or any other standard location. Other tools resort to extracting dependencies from extension modules and shared libraries inside Python packages, like elfdeps (Fedora). Enabling automating this type of conversion by only using explicitly annotated metadata is a key benefit of this PEP, making packaging Python packages for distros easier and more reliable. In addition, the authors envision other types of tools making use of this information, e.g., dependency analysis tools like Repology, Dependabot and libraries.io.

Software bill of materials (SBOM) generation tools may also be able to use this information, e.g. for flagging that external dependencies listed in pyproject.toml but not contained in wheel metadata are likely vendored within the wheel. PEP 770, which standardizes how SBOMs are included in wheels, contains an instructive section on how that PEP differs from this one.

Packages with external dependencies are typically hard to build from source, and error messages from build failures tend to be hard to decipher for end users. Missing external dependencies on the end user’s system are the most likely cause of build failures. If installers can show the required external dependencies as part of their error message, this may save users a lot of time.

At the moment, information on external dependencies is only captured in installation documentation of individual packages. It is hard to maintain for package authors and tends to go out of date. It’s also hard for users and distro packagers to find it. Having a canonical place to record this dependency information will improve this situation.

This PEP is not trying to specify how the external dependencies should be used, nor a mechanism to implement a name mapping from names of individual packages that are canonical for Python projects published on PyPI to those of other packaging ecosystems. Canonical names and a name mapping mechanism are addressed in PEP 804.

Rationale

Types of external dependencies

Multiple types of external dependencies can be distinguished:

  • Concrete packages that can be identified by name and have a canonical location in another language-specific package repository. E.g., Rust packages on crates.io, R packages on CRAN, JavaScript packages on the npm registry.
  • Concrete packages that can be identified by name but do not have a clear canonical location. This is typically the case for libraries and tools written in C, C++, Fortran, CUDA and other low-level languages. E.g., Boost, OpenSSL, Protobuf, Intel MKL, GCC.
  • “Virtual” packages, which are names for concepts, types of tools or interfaces. These typically have multiple implementations, which are concrete packages. E.g., a C++ compiler, BLAS, LAPACK, OpenMP, MPI.

Concrete packages are straightforward to understand, and are a concept present in every package management system. Virtual packages are a concept also present in a number of packaging systems – but not always, and the details of their implementation vary.

Cross compilation

Cross compilation is not yet (as of September 2025) well-supported by stdlib modules and pyproject.toml metadata. It is however important when translating external dependencies to those of other packaging systems (with tools like pyp2spec). Introducing support for cross compilation immediately in this PEP is much easier than extending [external] in the future, hence the authors choose to include this now.

Terminology

This PEP uses the following terminology:

  • build machine: the machine on which the package build process is being executed.
  • host machine: the machine on which the produced artifact will be installed and run.
  • build dependency: package required only during the build process. It must be available at build time and is built for the build machine’s OS and architecture. Typical examples include compilers, code generators, and build tools.
  • host dependency: package needed during the build and often also at runtime. It must be available during the build and is built for the host machine’s OS and architecture. These are usually libraries the project links against.
  • runtime dependency: package required only when the package is used after installation. It is not required at build time but must be available on the host machine at runtime.

Note that this terminology is not consistent across build and packaging tools, so care must be taken when comparing build/host dependencies in pyproject.toml to dependencies from other package managers.

Note that “target machine” or “target dependency” are not used in this PEP. That is typically only relevant for cross-compiling a compiler or other such advanced scenarios [3], [4] – this is out of scope for this PEP.

Finally, note that while “dependency” is the term most widely used for packages needed at build time, the existing key in pyproject.toml for PyPI build-time dependencies is build-requires. Hence this PEP uses the keys build-requires and host-requires under [external] for consistency.

Build and host dependencies

Clear separation of metadata associated with the definition of build and host platforms, rather than assuming that build and host platform will always be the same, is important [5].

Build dependencies are typically run during the build process - they may be compilers, code generators, or other such tools. In case the use of a build dependency implies a runtime dependency, that runtime dependency does not have to be declared explicitly. For example, when compiling Fortran code with gfortran into a Python extension module, the package likely incurs a dependency on the libgfortran runtime library. The rationale for not explicitly listing such runtime dependencies is two-fold: (1) it may depend on compiler/linker flags or details of the build environment whether the dependency is present, and (2) these runtime dependencies can be detected and handled automatically by tools like auditwheel.

Host dependencies are typically not run during the build process, but only used for linking against. This is not a rule though – it may be possible or necessary to run a host dependency under an emulator, or through a custom tool like crossenv. When host dependencies imply a runtime dependency, that runtime dependency also does not have to be declared, just like for build dependencies.

When host dependencies are declared and a tool which is executing an action unrelated to cross-compiling, it may decide to merge the host-requires list into build-requires - whether this is useful is context-dependent.

Specifying external dependencies

Concrete package specification

A “Package URL” or PURL is a widely used URL string for identifying packages that is meant to be portable across packaging ecosystems. Its design is:

scheme:type/namespace/name@version?qualifiers#subpath

The scheme component is a fixed string, pkg, and of the other components only type and name are required.

Since external dependencies are likely to be typed by hand, we propose a PURL derivative that, in the name of ergonomics and user-friendliness, introduces a number of changes (further discussed below):

  • Support for virtual packages via a new virtual type.
  • Allow version ranges (and not just literals) in the version field.

In this derivative, we replace the pkg scheme with dep. Hence, we will refer to them as DepURLs.

As an example, a DepURL for the requests package on PyPI would be:

dep:pypi/requests
# equivalent to pkg:pypi/requests

Adopting PURL-compatible strings to specify external dependencies in pyproject.toml solves a number of problems at once, and there are already implementations of the specification in Python and multiple other languages. PURL is also already supported by dependency-related tooling like SPDX (see External Repository Identifiers in the SPDX 2.3 spec), the Open Source Vulnerability format, and the Sonatype OSS Index; not having to wait years before support in such tooling arrives is valuable. DepURLs are very easily transformed into PURLs, with the exception of dep:virtual which doesn’t have an equivalent in PURL.

For concrete packages without a canonical package manager to refer to, either dep:generic/dep-name can be used, or a direct reference to the VCS system that the package is maintained in (e.g., dep:github/user-or-org-name/dep-name). Which of these is more appropriate is situation-dependent. This PEP recommends using dep:generic when the package name is unambiguous and well-known (e.g., dep:generic/git or dep:generic/openblas), and using the VCS as the type otherwise. Which name is chosen as canonical for any given package, as well as the process to make and record such choices, is the topic of PEP 804.

Virtual package specification

PURL does not offer support for virtual or virtual dependency specification yet. A proposal to add a virtual type is being discussed for revision 1.1.

In the meantime, we propose adding a new type to our dep: derivative, the virtual type, which can take two namespaces (extensible through the process given in PEP 804):

  • interface: for components such as BLAS or MPI.
  • compiler: for compiled languages like C or Rust.

The name should be the most common name for the interface or language, lowercased. Some examples include:

dep:virtual/compiler/c
dep:virtual/compiler/cxx
dep:virtual/compiler/rust
dep:virtual/interface/blas
dep:virtual/interface/lapack

Since there are a limited number of such dependencies, it seems like it will be understandable and map well to Linux distros with virtual packages and to the likes of conda and Spack.

Versioning

PURLs support fixed versions via the @ component of the URL. For example, numpy===2.0 can be expressed as pkg:pypi/numpy@2.0.

Support in PURL for version expressions and ranges beyond a fixed version is available via vers URIs (see specification):