python-packaging · diff
git:20260326.3e0cff6 to git:20260522.be57c0b
2 added, 346 removed. Audit A to A.
---
name: python-packaging
description: Create distributable Python packages with proper project structure, setup.py/pyproject.toml, and publishing to PyPI. Use when packaging Python libraries, creating CLI tools, or distributing Python code.
---
# Python Packaging
Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.
## When to Use This Skill
- Creating Python libraries for distribution
- Building command-line tools with entry points
- Publishing packages to PyPI or private repositories
- Setting up Python project structure
- Creating installable packages with dependencies
- Building wheels and source distributions
- Versioning and releasing Python packages
- Creating namespace packages
- Implementing package metadata and classifiers
## Core Concepts
### 1. Package Structure
- **Source layout**: `src/package_name/` (recommended)
- **Flat layout**: `package_name/` (simpler but less flexible)
- **Package metadata**: pyproject.toml, setup.py, or setup.cfg
- **Distribution formats**: wheel (.whl) and source distribution (.tar.gz)
### 2. Modern Packaging Standards
- **PEP 517/518**: Build system requirements
- **PEP 621**: Metadata in pyproject.toml
- **PEP 660**: Editable installs
- **pyproject.toml**: Single source of configuration
### 3. Build Backends
- **setuptools**: Traditional, widely used
- **hatchling**: Modern, opinionated
- **flit**: Lightweight, for pure Python
- **poetry**: Dependency management + packaging
### 4. Distribution
- **PyPI**: Python Package Index (public)
- **TestPyPI**: Testing before production
- **Private repositories**: JFrog, AWS CodeArtifact, etc.
## Quick Start
### Minimal Package Structure
```
my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── src/
│ └── my_package/
│ ├── __init__.py
│ └── module.py
└── tests/
└── test_module.py
```
### Minimal pyproject.toml
```toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "my-package"
version = "0.1.0"
description = "A short description"
authors = [{name = "Your Name", email = "you@example.com"}]
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
"requests>=2.28.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0",
"black>=22.0",
]
```
## Package Structure Patterns
### Pattern 1: Source Layout (Recommended)
```
my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── .gitignore
├── src/
│ └── my_package/
│ ├── __init__.py
│ ├── core.py
│ ├── utils.py
│ └── py.typed # For type hints
├── tests/
│ ├── __init__.py
│ ├── test_core.py
│ └── test_utils.py
└── docs/
└── index.md
```
**Advantages:**
- Prevents accidentally importing from source
- Cleaner test imports
- Better isolation
**pyproject.toml for source layout:**
```toml
[tool.setuptools.packages.find]
where = ["src"]
```
### Pattern 2: Flat Layout
```
my-package/
├── pyproject.toml
├── README.md
├── my_package/
│ ├── __init__.py
│ └── module.py
└── tests/
└── test_module.py
```
**Simpler but:**
- Can import package without installing
- Less professional for libraries
### Pattern 3: Multi-Package Project
```
project/
├── pyproject.toml
├── packages/
│ ├── package-a/
│ │ └── src/
│ │ └── package_a/
│ └── package-b/
│ └── src/
│ └── package_b/
└── tests/
```
- ## Complete pyproject.toml Examples
-
- ### Pattern 4: Full-Featured pyproject.toml
-
- ```toml
- [build-system]
- requires = ["setuptools>=61.0", "wheel"]
- build-backend = "setuptools.build_meta"
-
- [project]
- name = "my-awesome-package"
- version = "1.0.0"
- description = "An awesome Python package"
- readme = "README.md"
- requires-python = ">=3.8"
- license = {text = "MIT"}
- authors = [
- {name = "Your Name", email = "you@example.com"},
- ]
- maintainers = [
- {name = "Maintainer Name", email = "maintainer@example.com"},
- ]
- keywords = ["example", "package", "awesome"]
- classifiers = [
- "Development Status :: 4 - Beta",
- "Intended Audience :: Developers",
- "License :: OSI Approved :: MIT License",
- "Programming Language :: Python :: 3",
- "Programming Language :: Python :: 3.8",
- "Programming Language :: Python :: 3.9",
- "Programming Language :: Python :: 3.10",
- "Programming Language :: Python :: 3.11",
- "Programming Language :: Python :: 3.12",
- ]
-
- dependencies = [
- "requests>=2.28.0,<3.0.0",
- "click>=8.0.0",
- "pydantic>=2.0.0",
- ]
-
- [project.optional-dependencies]
- dev = [
- "pytest>=7.0.0",
- "pytest-cov>=4.0.0",
- "black>=23.0.0",
- "ruff>=0.1.0",
- "mypy>=1.0.0",
- ]
- docs = [
- "sphinx>=5.0.0",
- "sphinx-rtd-theme>=1.0.0",
- ]
- all = [
- "my-awesome-package[dev,docs]",
- ]
-
- [project.urls]
- Homepage = "https://github.com/username/my-awesome-package"
- Documentation = "https://my-awesome-package.readthedocs.io"
- Repository = "https://github.com/username/my-awesome-package"
- "Bug Tracker" = "https://github.com/username/my-awesome-package/issues"
- Changelog = "https://github.com/username/my-awesome-package/blob/main/CHANGELOG.md"
-
- [project.scripts]
- my-cli = "my_package.cli:main"
- awesome-tool = "my_package.tools:run"
-
- [project.entry-points."my_package.plugins"]
- plugin1 = "my_package.plugins:plugin1"
-
- [tool.setuptools]
- package-dir = {"" = "src"}
- zip-safe = false
-
- [tool.setuptools.packages.find]
- where = ["src"]
- include = ["my_package*"]
- exclude = ["tests*"]
-
- [tool.setuptools.package-data]
- my_package = ["py.typed", "*.pyi", "data/*.json"]
-
- # Black configuration
- [tool.black]
- line-length = 100
- target-version = ["py38", "py39", "py310", "py311"]
- include = '\.pyi?$'
-
- # Ruff configuration
- [tool.ruff]
- line-length = 100
- target-version = "py38"
-
- [tool.ruff.lint]
- select = ["E", "F", "I", "N", "W", "UP"]
-
- # MyPy configuration
- [tool.mypy]
- python_version = "3.8"
- warn_return_any = true
- warn_unused_configs = true
- disallow_untyped_defs = true
-
- # Pytest configuration
- [tool.pytest.ini_options]
- testpaths = ["tests"]
- python_files = ["test_*.py"]
- addopts = "-v --cov=my_package --cov-report=term-missing"
-
- # Coverage configuration
- [tool.coverage.run]
- source = ["src"]
- omit = ["*/tests/*"]
-
- [tool.coverage.report]
- exclude_lines = [
- "pragma: no cover",
- "def __repr__",
- "raise AssertionError",
- "raise NotImplementedError",
- ]
- ```
-
- ### Pattern 5: Dynamic Versioning
-
- ```toml
- [build-system]
- requires = ["setuptools>=61.0", "setuptools-scm>=8.0"]
- build-backend = "setuptools.build_meta"
-
- [project]
- name = "my-package"
- dynamic = ["version"]
- description = "Package with dynamic version"
-
- [tool.setuptools.dynamic]
- version = {attr = "my_package.__version__"}
-
- # Or use setuptools-scm for git-based versioning
- [tool.setuptools_scm]
- write_to = "src/my_package/_version.py"
- ```
-
- **In **init**.py:**
-
- ```python
- # src/my_package/__init__.py
- __version__ = "1.0.0"
-
- # Or with setuptools-scm
- from importlib.metadata import version
- __version__ = version("my-package")
- ```
-
- ## Command-Line Interface (CLI) Patterns
-
- ### Pattern 6: CLI with Click
-
- ```python
- # src/my_package/cli.py
- import click
-
- @click.group()
- @click.version_option()
- def cli():
- """My awesome CLI tool."""
- pass
-
- @cli.command()
- @click.argument("name")
- @click.option("--greeting", default="Hello", help="Greeting to use")
- def greet(name: str, greeting: str):
- """Greet someone."""
- click.echo(f"{greeting}, {name}!")
-
- @cli.command()
- @click.option("--count", default=1, help="Number of times to repeat")
- def repeat(count: int):
- """Repeat a message."""
- for i in range(count):
- click.echo(f"Message {i + 1}")
-
- def main():
- """Entry point for CLI."""
- cli()
-
- if __name__ == "__main__":
- main()
- ```
-
- **Register in pyproject.toml:**
-
- ```toml
- [project.scripts]
- my-tool = "my_package.cli:main"
- ```
-
- **Usage:**
-
- ```bash
- pip install -e .
- my-tool greet World
- my-tool greet Alice --greeting="Hi"
- my-tool repeat --count=3
- ```
-
- ### Pattern 7: CLI with argparse
-
- ```python
- # src/my_package/cli.py
- import argparse
- import sys
-
- def main():
- """Main CLI entry point."""
- parser = argparse.ArgumentParser(
- description="My awesome tool",
- prog="my-tool"
- )
-
- parser.add_argument(
- "--version",
- action="version",
- version="%(prog)s 1.0.0"
- )
-
- subparsers = parser.add_subparsers(dest="command", help="Commands")
-
- # Add subcommand
- process_parser = subparsers.add_parser("process", help="Process data")
- process_parser.add_argument("input_file", help="Input file path")
- process_parser.add_argument(
- "--output", "-o",
- default="output.txt",
- help="Output file path"
- )
-
- args = parser.parse_args()
-
- if args.command == "process":
- process_data(args.input_file, args.output)
- else:
- parser.print_help()
- sys.exit(1)
-
- def process_data(input_file: str, output_file: str):
- """Process data from input to output."""
- print(f"Processing {input_file} -> {output_file}")
-
- if __name__ == "__main__":
- main()
- ```
-
- ## Building and Publishing
-
- ### Pattern 8: Build Package Locally
-
- ```bash
- # Install build tools
- pip install build twine
-
- # Build distribution
- python -m build
-
- # This creates:
- # dist/
- # my-package-1.0.0.tar.gz (source distribution)
- # my_package-1.0.0-py3-none-any.whl (wheel)
-
- # Check the distribution
- twine check dist/*
- ```
-
- ### Pattern 9: Publishing to PyPI
-
- ```bash
- # Install publishing tools
- pip install twine
-
- # Test on TestPyPI first
- twine upload --repository testpypi dist/*
-
- # Install from TestPyPI to test
- pip install --index-url https://test.pypi.org/simple/ my-package
-
- # If all good, publish to PyPI
- twine upload dist/*
- ```
-
- **Using API tokens (recommended):**
-
- ```bash
- # Create ~/.pypirc
- [distutils]
- index-servers =
- pypi
- testpypi
-
- [pypi]
- username = __token__
- password = pypi-...your-token...
-
- [testpypi]
- username = __token__
- password = pypi-...your-test-token...
- ```
-
- ### Pattern 10: Automated Publishing with GitHub Actions
-
- ```yaml
- # .github/workflows/publish.yml
- name: Publish to PyPI
-
- on:
- release:
- types: [created]
-
- jobs:
- publish:
- runs-on: ubuntu-latest
-
- steps:
- - uses: actions/checkout@v3
-
- - name: Set up Python
- uses: actions/setup-python@v4
- with:
- python-version: "3.11"
-
- - name: Install dependencies
- run: |
- pip install build twine
-
- - name: Build package
- run: python -m build
-
- - name: Check package
- run: twine check dist/*
+ ## Detailed patterns and worked examples
- - name: Publish to PyPI
- env:
- TWINE_USERNAME: __token__
- TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
- run: twine upload dist/*
- ```
+ Detailed pattern documentation lives in `references/details.md`. Read that file when the navigation tier above is insufficient.
- For advanced patterns including data files, namespace packages, C extensions, version management, testing installation, documentation templates, and distribution workflows, see [references/advanced-patterns.md](references/advanced-patterns.md)