PEP 777 – How to Re-invent the Wheel

Author:: Emma Harper Smith <emma at python.org>
Sponsor:: Barry Warsaw <barry at python.org>
PEP-Delegate:: Paul Moore <p.f.moore at gmail.com>
Discussions-To:: Discourse thread
Status:: Draft
Type:: Standards Track
Topic:: Packaging
Created:: 09-Oct-2024
Post-History:: 10-Oct-2024

Table of Contents

Abstract

The current wheel 1.0 specification was written over a decade ago, and has been extremely robust to changes in the Python packaging ecosystem. Previous efforts to improve the wheel specification were deferred to focus on other packaging specifications. Meanwhile, the use of wheels has changed dramatically in the last decade. There have been many requests for new wheel features over the years; however, a fundamental obstacle to evolving the wheel specification has been that there is no defined process for how to handle adding backwards-incompatible features to wheels. Therefore, to enable other PEPs to describe new enhancements to the wheel specification, this PEP prescribes compatibility requirements on future wheel revisions. This PEP does not specify a new wheel revision. The specification of a new wheel format (“Wheel 2.0”) is left to a future PEP.

Rationale

Currently, wheel specification changes that require new installer behavior are backwards incompatible and require a major version increase in the wheel metadata format. An increase of the wheel major version has yet to happen, partially because such a change has the potential to be catastrophically disruptive. Per the wheel specification, any installer that does not support the new major version must abort at install time. This means that if the major version were to be incremented without further planning, many users would see installation failures as older installers reject new wheels uploaded to public package indices like the Python Package Index (PyPI). It is critically important to carefully plan the interactions between build tools, package indices, and package installers to avoid incompatibility issues, especially considering the long tail of users who are slow to update their installers.

The backward compatibility concerns have prevented valuable improvements to the wheel file format, such as better compression, wheel data format improvements, better information about what is included in a wheel, and JSON formatted metadata in the “.dist-info” folder.

This PEP describes constraints and behavior for new wheel revisions to preserve stability for existing tools that do not support a new major version of the wheel format. This ensures that backwards incompatible changes to the wheel specification will only affect users and tools that are properly set up to use the newer wheels. With a clear path for evolving the wheel specification, future PEPs will be able to improve the wheel format without needing to re-define a completely new compatibility story.

Specification

Add Wheel-Version Metadata Field to Core Metadata

Currently, the wheel 1.0 PEP, PEP 427, specifies that wheel files must contain a WHEEL metadata file that contains the version of the wheel specification that the file conforms to. PEP 427 stipulates that installers MUST warn on installation of a wheel with a minor version greater than supported, and MUST abort on installation of wheels with a major version that is greater than what the installer supports. This ensures that users do not get invalid installations from wheels that installers cannot properly install.

However, resolvers do not currently exclude wheels with an incompatible wheel version. There is also currently no way for a resolver to check a wheel’s version without downloading the wheel directly. To make wheel version filtering easy for resolvers, the wheel version MUST be included in the relevant metadata file (currently METADATA). This will allow resolvers to efficiently check the wheel version using the PEP 658 metadata API without needing to download and inspect the .dist-info/WHEEL file.

To accomplish this, a new field, Wheel-Version, will be added to the Core Metadata Specification. This field is single use, and must contain the exact same version specified as the Wheel-Version entry in the WHEEL file, or any future replacement file defining metadata about the wheel file. If Wheel-Version is absent from the metadata file, then tools MUST infer the wheel file major version as 1.

Wheel-Version MUST NOT be included in source distribution metadata (PKG-INFO) files. If a tool encounters Wheel-Version inside of a source distribution metadata file, it SHOULD raise an error.

Wheel-Version