PEP 752 – Implicit namespaces for package repositories

PEP 752 – Implicit namespaces for package repositories

Author:

Ofek Lev <ofekmeister at gmail.com>, Jarek Potiuk <potiuk at apache.org>

Sponsor:

Barry Warsaw <barry at python.org>

PEP-Delegate:

Dustin Ingram <di at python.org>

Discussions-To:

Discourse thread

Status:

Accepted

Type:

Standards Track

Topic:

Packaging

Created:

13-Aug-2024

Post-History:

18-Aug-2024, 07-Sep-2024

Resolution:

29-Jun-2026

---

Table of Contents

• Abstract
• Motivation
• Rationale
• Terminology
• Specification
  • Naming
  • Semantics
  • Uploads
  • Repository Metadata
    • Project Detail
    • Namespace List
    • Namespace Detail
  • Grant Removal
• Community Buy-in
• Backwards Compatibility
• Security Implications
• How to Teach This
• Reference Implementation
• Rejected Ideas
  • Explicit Non-User Ownership
  • Organization Scoping
  • Artifact-level Namespace Association
  • Support HTML Simple API
  • Encourage Dedicated Package Repositories
  • Open Namespaces
  • Hidden Grants
  • Exclusive Reliance on Provenance Assertions
  • Asserting Package Owner Names
  • Use Fixed Prefixes
  • Use DNS
• Open Issues
• Footnotes
• Copyright

Abstract

This PEP specifies a way for organizations to reserve package name prefixes for future uploads.

“Namespaces are one honking great idea – let’s do more of those!” - PEP 20

Motivation

The current ecosystem lacks a way for projects with many packages to signal a verified pattern of ownership, who desire complete control over their namespace for safety and branding reasons. A few examples:

• Major cloud providers like Amazon, Google and Microsoft have a common prefix for each feature’s corresponding package [1]. For example, most of Google’s packages are prefixed by google-cloud- e.g. google-cloud-compute for using virtual machines.
• OpenTelemetry is an open standard for observability with official packages for the core APIs and SDK with contrib packages to collect data from various sources. All packages are prefixed by opentelemetry- with child prefixes in the form opentelemetry-<component>-<name>-. The contrib packages live in a central repository and they are the only ones with the ability to publish.
• Apache Airflow is a platform to programmatically author, schedule and monitor workflows. It has providers, where each provider package is prefixed by apache-airflow-providers-.
• Typeshed is a community effort to maintain type stubs for various packages. The stub packages they maintain mirror the package name they target and are prefixed by types-. For example, the package requests has a stub that users would depend on called types-requests. Unofficial stubs are not supposed to use the types- prefix and are expected to use a -stubs suffix instead.

Such projects are uniquely vulnerable to name-squatting attacks which can ultimately result in dependency confusion.

For example, say a new product is released for which monitoring would be valuable. It would be reasonable to assume that Datadog would eventually support it as an official integration. It takes a nontrivial amount of time to deliver such an integration due to roadmap prioritization and the time required for implementation. It would be impossible to reserve the name of every potential package so in the interim an attacker may create a package that appears legitimate which would execute malicious code (like secret exfiltration) at runtime. Not only are users more likely to install such packages but doing so taints the perception of the entire project. Community projects like Apache Airflow have also experienced this.

Although PEP 708 attempts to address this attack vector, it is specifically about the case of multiple repositories being considered during dependency resolution and does not offer any protection to the aforementioned use cases.

In recent years, typosquatting has become a popular attack vector [2]. The current protection against this used by PyPI is to normalize similar characters but that is insufficient for these use cases. Namespacing would drastically reduce the incidence of typosquatting:

• Typos would have to be in the prefix itself which is normalized and likely to be a short, well-known identifier like aws-.
• An index may require namespaces to be applied for and approved, reducing the likelihood of typosquatting of such events.
• An attacker would be unable to squat a name that includes a namespace.

Rationale

Other package ecosystems have generally solved this problem by taking one of two approaches: either minimizing or maximizing backwards compatibility.

• NPM has the concept of scoped packages which were introduced primarily to combat there being a dearth of available good package names (whether a real or perceived phenomenon). When a user or organization signs up they are given a scope that matches their name. For example, the package for using Google Cloud Storage is @google-cloud/storage where @google-cloud/ is the scope. Regular user accounts (non-organization) may publish unscoped packages for public use. This approach has the lowest amount of backwards compatibility because every installer and tool has to be modified to account for scopes.
• NuGet has the concept of package ID prefix reservation which was introduced primarily to satisfy users wishing to know where a package came from. A package name prefix may be reserved for use by one or more owners. Every reserved package has a special indication on its page to communicate this. After reservation, any upload with a reserved prefix will fail if the user is not an owner of the prefix. Existing packages that have a prefix that is owned may continue to release as usual. This approach has the highest amount of backwards compatibility because only modifications to indices like PyPI are required and installers do not need to change.

This PEP specifies the NuGet approach of authorized reservation across a flat namespace. Any solution that requires new package syntax must be built atop the existing flat namespace and therefore implicit namespaces acquired via a reservation mechanism would be a prerequisite to such explicit namespaces.

Although existing packages matching a reserved namespace would be untouched, preventing future unauthorized uploads and strategically applying PEP 541 takedown requests for malicious cases would reduce risks to users to a negligible level.

Terminology

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Owner

Owners are entities that are allowed to upload certain package names.

Grant

A grant is a reservation of a namespace for a package repository.

Parent Namespace

A namespace’s parent refers to the namespace without the trailing hyphenated component e.g. the parent of foo-bar is foo.

Child Namespace

A namespace’s child refers to the namespace with a single trailing hyphenated component e.g. foo-bar is a valid child of foo.

Specification

Naming

A namespace MUST be a valid project name and normalized internally e.g. foo.bar would become foo-bar.

Semantics

A namespace grant bestows ownership over the following:

1. A project that exactly matches the namespace itself.
2. Projects that start with the namespace followed by a hyphen. For example, the namespace foo would match the normalized project name foo-bar but not the project name foobar.

Package name matching acts upon the normalized namespace.

Namespaces are per-package repository and SHALL NOT be shared between repositories. For example, if PyPI has a namespace acme that is owned by the company Acme, packages starting with acme- that come from other non-PyPI mirror repositories do not confer the same level of trust.

Grants MUST NOT overlap ownership. For example, if there is an existing grant for foo-bar then a new grant for foo would only be possible for the owner of the former. An overlap is determined by comparing the normalized proposed namespace with the normalized namespace of every existing root grant. Every comparison must append a hyphen to the end of the proposed and existing namespace. An overlap is detected when any existing namespace starts with the proposed namespace.

Repositories SHOULD impose a depth limit on the number of hyphens in a namespace. For example, if the depth limit is 1 then the namespace foo-bar would be allowed but foo-bar-baz could not be granted.

Policies for granting and managing namespaces are not discussed here as they are specific to each index. The proposed namespace policy for PyPI is described in PEP 755.

Uploads

Uploads MUST fail with a 409 Conflict HTTP status code if the name of a package being uploaded matches a reserved namespace and the project owner does not have an active grant for the namespace.

Repositories SHOULD have an exception to this rule for projects that existed before the namespace was reserved.