PEP 594 – Removing dead batteries from the standard library

Author:: Christian Heimes <christian at python.org>, Brett Cannon <brett at python.org>
Discussions-To:: Discourse thread
Status:: Final
Type:: Standards Track
Created:: 20-May-2019
Python-Version:: 3.11
Post-History:: 21-May-2019, 04-Feb-2022
Resolution:: Discourse message

Table of Contents

Abstract

This PEP proposed a list of standard library modules to be removed from the standard library. The modules are mostly historic data formats (e.g. Commodore and SUN file formats), APIs and operating systems that have been superseded a long time ago (e.g. Mac OS 9), or modules that have security implications and better alternatives (e.g. password and login).

The PEP follows in the footsteps of other PEPS like PEP 3108. The Standard Library Reorganization proposal removed a bunch of modules from Python 3.0. In 2007, the PEP referred to maintenance burden as:

“Over the years, certain modules have become a heavy burden upon python-dev to maintain. In situations like this, it is better for the module to be given to the community to maintain to free python-dev to focus more on language support and other modules in the standard library that do not take up an undue amount of time and effort.”

The withdrawn PEP 206 from 2000 expresses issues with the Python standard library in an unvarnished and forthright manner:

“[…] the standard library modules aren’t always the best choices for a job. Some library modules were quick hacks (e.g. calendar, commands), some were designed poorly and are now near-impossible to fix (cgi), and some have been rendered obsolete by other, more complete modules […].”

Rationale

Back in the early days of Python, the interpreter came with a large set of useful modules. This was often referred to as “batteries included” philosophy and was one of the cornerstones to Python’s success story. Users didn’t have to figure out how to download and install separate packages in order to write a simple web server or parse email.

Times have changed. With the introduction of PyPI (née Cheeseshop), setuptools, and later pip, it became simple and straightforward to download and install packages. Nowadays Python has a rich and vibrant ecosystem of third-party packages. It’s pretty much standard to either install packages from PyPI or use one of the many Python or Linux distributions.

On the other hand, Python’s standard library is piling up with cruft, unnecessary duplication of functionality, and dispensable features. This is undesirable for several reasons.

Any additional module increases the maintenance cost for the Python core development team. The team has limited resources, reduced maintenance cost frees development time for other improvements.
Modules in the standard library are generally favored and seen as the de facto solution for a problem. A majority of users only pick third-party modules to replace a stdlib module, when they have a compelling reason, e.g. lxml instead of xml. The removal of an unmaintained stdlib module increases the chances of a community-contributed module to become widely used.
A lean and mean standard library benefits platforms with limited resources like devices with just a few hundred kilobyte of storage (e.g. BBC Micro:bit). Python on mobile platforms like BeeWare or WebAssembly (e.g. pyodide) also benefit from reduced download size.

The modules in this PEP have been selected for deprecation because their removal is either least controversial or most beneficial. For example, least controversial are 30-year-old multimedia formats like the sunau audio format, which was used on SPARC and NeXT workstations in the late 1980s. The crypt module has fundamental flaws that are better solved outside the standard library.

This PEP also designates some modules as not scheduled for removal. Some modules have been deprecated for several releases or seem unnecessary at first glance. However it is beneficial to keep the modules in the standard library, mostly for environments where installing a package from PyPI is not an option. This can be corporate environments or classrooms where external code is not permitted without legal approval.

The usage of FTP is declining, but some files are still provided over the FTP protocol or hosters offer FTP to upload content. Therefore, ftplib is going to stay.
The optparse and getopt modules are widely used. They are mature modules with very low maintenance overhead.
According to David Beazley [5] the wave module is easy to teach to kids and can make crazy sounds. Making a computer generate sounds is a powerful and highly motivating exercise for a nine-year-old aspiring developer. It’s a fun battery to keep.

Deprecation schedule

3.11

Starting with Python 3.11, deprecated modules will start issuing DeprecationWarning. The estimated EOL of Python 3.10, the last version without the warning, is October 2026.

3.12

There should be no specific change compared to Python 3.11. This is the last version of Python with the deprecated modules, with an estimated EOL of October 2028.

3.13

All modules deprecated by this PEP are removed from the main branch of the CPython repository and are no longer distributed as part of Python.

Deprecated modules

The modules are grouped as data encoding, multimedia, network, OS interface, and misc modules. The majority of modules are for old data formats or old APIs. Some others are rarely useful and have better replacements on PyPI, e.g. Pillow for image processing or NumPy-based projects to deal with audio processing.

Table 1: Proposed modules deprecations
Module	Deprecated in	To be removed	Added in	Has maintainer?	Replacement
aifc	3.11 (3.0*)	3.13	1993	yes (inactive)	-
asynchat	3.6 (3.0*)	3.12	1999	yes	`asyncio`
asyncore	3.6 (3.0*)	3.12	1999	yes	`asyncio`
audioop	3.11 (3.0*)	3.13	1992	yes	-
cgi	3.11 (2.0**)	3.13	1995	no	-
cgitb	3.11 (2.0**)	3.13	1995	no	-
chunk	3.11	3.13	1999	no	-
crypt	3.11	3.13	1994	yes (inactive)	legacycrypt, bcrypt, argon2-cffi, `hashlib`, passlib
imghdr	3.11	3.13	1992	no	filetype, puremagic, python-magic
mailcap	3.11	3.13	1995	no	-
msilib	3.11	3.13	2006	no	-
nntplib	3.11	3.13	1992	no	-
nis	3.11 (3.0*)	3.13	1992	no	-
ossaudiodev	3.11	3.13	2002	no	-
pipes	3.11	3.13	1992	no	`subprocess`
smtpd	3.4.7, 3.5.4	3.12	2001	yes	aiosmtpd
sndhdr	3.11	3.13	1994	no	filetype, puremagic, python-magic
spwd	3.11	3.13	2005	no	python-pam
sunau	3.11 (3.0*)	3.13	1993	no	-
telnetlib	3.11 (3.0*)	3.13	1997	no	telnetlib3, Exscript
uu	3.11	3.13	1994	no	-
xdrlib	3.11	3.13	1992/1996	no	-

Some module deprecations proposed by PEP 3108 for 3.0 and PEP 206 for 2.0. The added in column illustrates, when a module was originally designed and added to the standard library. The has maintainer column refers to the expert index, a list of domain experts and maintainers in the DevGuide.

Data encoding modules

uu and the uu encoding

The uu module provides uuencode format, an old binary encoding format for email from 1980. The uu format has been replaced by MIME. The uu codec is provided by the binascii module. There’s also encodings/uu_codec.py which is a codec for the same encoding; it should also be deprecated.

xdrlib

The xdrlib module supports the Sun External Data Representation Standard. XDR is an old binary serialization format from 1987. These days it’s rarely used outside specialized domains like NFS.

Multimedia modules

aifc

The aifc module provides support for reading and writing AIFF and AIFF-C files. The Audio Interchange File Format is an old audio format from 1988 based on Amiga IFF. It was most commonly used on the Apple Macintosh. These days only few specialized application use AIFF.

A user disclosed [6] that the post production film industry makes heavy use of the AIFC file format. The usage of the aifc module in closed source and internal software was unknown prior to the first posting of this PEP. This may be a compelling argument to keep the aifc module in the standard library. The file format is stable and the module does not require much maintenance. The strategic benefits for Python may outmatch the burden.

audioop

The audioop module contains helper functions to manipulate raw audio data and adaptive differential pulse-code modulated audio data. The module is implemented in C without any additional dependencies. The aifc, sunau, and wave modules depend on audioop for some operations.

The byteswap operation in the wave module can be substituted with little extra work. In case aifc is not deprecated as well, a reduced version of the audioop module is converted into a private implementation detail, e.g. _audioop with byteswap, alaw2lin, ulaw2lin, lin2alaw, lin2ulaw, and lin2adpcm.

chunk

The chunk module provides support for reading and writing Electronic Arts’ Interchange File Format. IFF is an old audio file format originally introduced for Commodore and Amiga. The format is no longer relevant.

imghdr

The imghdr module is a simple tool to guess the image file format from the first 32 bytes of a file or buffer. It supports only a limited number of formats and neither returns resolution nor color depth.

ossaudiodev

The ossaudiodev module provides support for Open Sound System, an interface to sound playback and capture devices. OSS was initially free software, but later support for newer sound devices and improvements were proprietary. Linux community abandoned OSS in favor of ALSA [1]. Some operating systems like OpenBSD and NetBSD provide an incomplete [2] emulation of OSS.

To best of my knowledge, FreeBSD is the only widespread operating system that uses Open Sound System as of today. The ossaudiodev hasn’t seen any improvements or new features since 2003. All commits since 2003 are project-wide code cleanups and a couple of bug fixes. It would be beneficial for both FreeBSD community and core development, if the module would be maintained and distributed by people that care for it and use it.

The standard library used to have more audio-related modules. The other audio device interfaces (audiodev, linuxaudiodev, sunaudiodev) were removed in 2007 as part of the PEP 3108 stdlib re-organization.

sndhdr

The sndhdr module is similar to the imghdr module but for audio formats. It guesses file format, channels, frame rate, and sample widths from the first 512 bytes of a file or buffer. The module only supports AU, AIFF, HCOM, VOC, WAV, and other ancient formats.

Python Enhancement Proposals