Referencing requirements.txt in pyproject.toml for setuptools

Problem Statement

When migrating setuptools-based Python projects from setup.py to modern pyproject.toml configuration, developers often face a challenge: How to maintain established pip-compile workflows using requirements.txt files while configuring dependencies directly through pyproject.toml.

The core conflict exists between: 🔒 Needing pinned dependencies (via requirements.txt) for reproducible installations and dependency conflict resolution 📦 Wanting modern configuration without maintaining a legacy setup.py script

This leaves developers wondering if they can reference external requirements.txt files directly in pyproject.toml while retaining setuptools as their build backend.

Recommended Solution: Dynamic Metadata in setuptools

Modern setuptools (v62.6+) provides native support for external dependency files through dynamic metadata declarations. This is currently the most robust solution for referencing requirements.txt in pyproject.toml.

Implementation Steps

1. Declare dependencies as dynamic in the [project] section
2. Reference requirements.txt in the [tool.setuptools.dynamic] section

[project]
name = "my-project"
version = "1.0.0"
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = { file = ["requirements.txt"] }

Important Limitations

• The requirements.txt file must follow PEP 508 standards
• These flags/commands are NOT supported in the referenced file:
  • -r (recursive requirements)
  • -c (constraints files)
  • -e (editable installs)

Handling Optional Dependencies

For development dependencies (e.g., requirements-dev.txt), declare them as an optional dependency group:

[project]
dynamic = ["dependencies", "optional-dependencies"]

[tool.setuptools.dynamic]
dependencies = { file = ["requirements.txt"] }
optional-dependencies = { 
    dev = { file = ["requirements-dev.txt"] } 
}

Version Caveats

• Requires setuptools ≥ 62.6
• Automatic inclusion in source distributions (sdist) requires setuptools ≥ 66.1.0 (older versions need manual MANIFEST.in configuration)
• All optional dependency groups must be declared dynamically if any are used

Compatibility Handling

To ensure compatibility across environments, explicitly declare setuptools version requirements in your build system:

[build-system]
requires = ["setuptools>=66.1.0"]
build-backend = "setuptools.build_meta"

Alternative Approaches

1. Using Hatch (Alternative Build System)

If you can migrate to Hatch, leverage the hatch-requirements-txt plugin:

[build-system]
requires = ["hatchling", "hatch-requirements-txt"]
build-backend = "hatchling.build"

[project]
name = "my-project"
version = "1.0.0"
dynamic = ["dependencies"]

[tool.hatch.metadata.hooks.requirements_txt]
files = ["requirements.txt"]

2. Using uv (Modern Pip Alternative)

The uv tool by Astral provides streamlined workflow integration:

uv pip compile requirements.in -o requirements.txt
uv add -r requirements.txt

When to Consider Alternatives

• If your dependency files require special flags (-r, -c, -e)
• If your project can adopt newer tools like Hatch or uv
• If you need functionality beyond current setuptools capabilities

Best Practices and Recommendations

1. Validate requirement syntax using packaging.requirements.Requirement before deployment
2. Maintain setuptools ≥ 66.1.0 to avoid manual file inclusion in source distributions
3. Keep requirement files clean by removing non-PEP 508 elements
4. Use separate base files like requirements.in for pip-compile input, compiling to PEP 508-compliant output

Migration Workflow

Conclusion

The dynamic metadata capability in modern setuptools provides the most straightforward solution for referencing requirements.txt in pyproject.toml. While alternative tools like Hatch and uv offer different approaches, the setuptools-native method ensures backward compatibility with existing workflows when using version 66.1.0 or newer.

For maximum stability:

1. Pin setuptools to ≥66.1.0
2. Ensure your requirements.txt follows strict PEP 508 format
3. Validate your build using python -m build before deployment

This approach maintains transparency in dependency management while fully embracing modern Python packaging standards.