pyproject.toml

Problem Statement

Python packaging has historically relied on setup.py files, but you may encounter projects that only include a pyproject.toml file instead. This can be confusing when trying to install packages in editable mode or understand the modern Python packaging ecosystem.

Traditional setup.py files presented several challenges:

• Installation failures if setuptools wasn't installed
• Limited interoperability between different packaging tools
• No standardized configuration format across the ecosystem

What is pyproject.toml?

pyproject.toml is a configuration file introduced by PEP 518 and PEP 517 to address packaging system limitations. It serves as a unified configuration format that specifies:

1. Build system requirements - What tools are needed to build the package
2. Package metadata - Project information, dependencies, and other details
3. Tool-specific configurations - Settings for various development tools

Key Benefits

• Solves the "build-tool dependency chicken and egg" problem
• Provides a standardized configuration format
• Enables interoperability between different packaging tools
• Supports modern packaging backends beyond setuptools

Does pyproject.toml Replace setup.py?

The relationship between pyproject.toml and setup.py depends on your packaging tool:

• For setuptools-based packages: pyproject.toml complements rather than replaces setup.py, ensuring correct execution
• For modern packaging tools (Poetry, Flit, Hatch, PDM): pyproject.toml completely replaces setup.py as the configuration source
• Backward compatibility: Some projects may include both files for compatibility with older tools

PEP 621 Support

PEP 621 standardizes package core metadata in pyproject.toml. Most modern backends support this specification:

Backend	PEP 621 Support Version
enscons	0.26.0+
flit_core	3.2+
hatchling	0.3+
pdm-backend	0.3.0+
poetry-core	2.0.0+
setuptools	61.0.0+

Installing Packages with pyproject.toml

Standard Installation

For packages using pyproject.toml, you can install them with:

pip install .

Or from a local directory:

pip install /path/to/package

Editable Installs

Editable installation support depends on PEP 660 compatibility:

Modern Approach

# Requires pip >= 21.3 and PEP 660-compatible backend
pip install -e .

Legacy Approach

# For older setups, you might need a setup.py shim
#!/usr/bin/env python
import setuptools

if __name__ == "__main__":
    setuptools.setup()

Backend Support for Editable Installs

Not all packaging backends support editable installs. Here's the current support status:

Backend	PEP 660 Support Version
enscons	0.28.0+
flit_core	3.4+
hatchling	0.3+
pdm-backend	0.8.0+
poetry-core	1.0.8+
setuptools	64.0.0+

Tool-Specific Installation

Different packaging tools have their own installation commands:

Using Poetry:

poetry install  # Installs dependencies and package
poetry run pytest tests/  # Run tests through Poetry

Using PDM:

pdm install  # Similar to poetry install

Example pyproject.toml Structure

Here's a basic example showing common configurations:

[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-package"
version = "0.1.0"
description = "A sample Python package"
dependencies = [
    "requests>=2.25.0",
    "click>=7.0"
]

[project.optional-dependencies]
dev = ["pytest", "black", "flake8"]

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--verbose"

[tool.black]
line-length = 88
target-version = ['py38']

Beyond Packaging: Tool Configurations

pyproject.toml isn't just for packaging—many tools use it for configuration:

• pytest: Test discovery and configuration
• black: Code formatting settings
• isort: Import sorting rules
• mypy: Type checking configuration
• pre-commit: Hook configurations

Example Tool Configurations

[tool.black]
line-length = 88
target-version = ["py38"]

[tool.isort]
profile = "black"
multi_line_output = 3

[tool.mypy]
python_version = "3.8"
warn_return_any = true
warn_unused_configs = true

Migration Tips

If you're transitioning from setup.py to pyproject.toml:

1. Start with build requirements: Add [build-system] section first
2. Gradually move metadata: Transfer package information from setup.py to [project] section
3. Maintain compatibility: Keep setup.py temporarily if needed for older tools
4. Update tooling: Ensure your packaging tool supports PEP 621 metadata

Conclusion

pyproject.toml represents the future of Python packaging configuration, providing a standardized, tool-agnostic format that works across multiple packaging systems. While it doesn't completely eliminate setup.py in all cases, it significantly improves the packaging experience by solving dependency management problems and providing a consistent configuration format.

For new projects, starting with pyproject.toml and a modern packaging backend (such as Poetry, Flit, or Hatch) is recommended. Existing projects can gradually migrate to take advantage of the improved workflow and interoperability that pyproject.toml provides.