Virtual Environments and Packaging

Once your scripts grow past one file, you need real dependency and packaging hygiene. Get this right early and your future self will thank you.

Why Virtual Environments?

Pip-installing into the system Python is a recipe for pain. Different projects need different versions of boto3, requests, pyyaml, and you'll eventually hit conflicts. A virtual environment is just a directory containing its own Python and its own site-packages. Per-project, completely isolated.

The Built-in Way: venv

python -m venv .venv
source .venv/bin/activate           # Linux/macOS
# or
.venv\Scripts\activate              # Windows PowerShell

# Now pip installs go into .venv only
pip install requests boto3
deactivate                          # back to system Python

Add .venv/ to .gitignore. Never commit a virtual environment.

The Modern Way: uv

uv is a Rust-based replacement for pip + venv + pip-tools. It's roughly 10× faster and is becoming the new default in many teams:

curl -LsSf https://astral.sh/uv/install.sh | sh

# Initialise a project
uv init myproject
cd myproject

# Add dependencies — automatically updates pyproject.toml and lockfile
uv add requests boto3
uv add --dev pytest ruff mypy

# Run anything inside the project's environment
uv run python script.py
uv run pytest
uv run ruff check .

uv manages the venv invisibly — you almost never need to "activate" it.

Declaring a Project: pyproject.toml

The standard, since PEP 621. Replaces setup.py, setup.cfg, and requirements.txt all at once:

[project]
name = "snap-cleanup"
version = "0.2.0"
description = "Delete old EBS snapshots."
authors = [{ name = "Alex", email = "alex@example.com" }]
license = "MIT"
requires-python = ">=3.11"
dependencies = [
    "boto3>=1.34",
    "tenacity>=8.2",
    "typer>=0.9",
]

[project.optional-dependencies]
dev = [
    "pytest>=8",
    "pytest-cov",
    "ruff",
    "mypy",
    "moto",
]

[project.scripts]
snap-cleanup = "snap_cleanup.cli:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.ruff]
line-length = 100

[tool.pytest.ini_options]
addopts = "-v --cov=snap_cleanup"

Everything about your project — metadata, deps, tool config — in one TOML file. Clean.

Lockfiles

For reproducible installs you need a lockfile that pins exact versions of every dependency and transitive dependency. Options:

• uv lock — produces uv.lock, run uv sync to install
• pip-tools — pip-compile pyproject.toml produces requirements.txt
• Poetry — alternative project manager with its own lockfile
• PDM — another option in the same space

Commit the lockfile. CI installs from the lockfile so every build uses the exact same dependency tree.

Console Scripts: Turning a Function into a Command

Notice the entry point in pyproject.toml:

[project.scripts]
snap-cleanup = "snap_cleanup.cli:main"

This means: when the package is installed, create a snap-cleanup command on PATH that runs main() in snap_cleanup.cli. Now your tool is callable from anywhere:

pip install -e .
snap-cleanup --max-age-days 30 --dry-run

The -e ("editable") install symlinks your source so changes take effect without reinstalling.

pipx: Isolated Tool Installs

pipx installs each Python application in its own venv but exposes the script on your PATH. Perfect for CLI tools you use everywhere — black, ruff, aws-cdk, your own internal tools:

pipx install ruff
pipx install httpie
pipx install git+https://github.com/myorg/internal-cli.git
pipx upgrade-all

You get all the tools without polluting any single Python's site-packages.

Building and Distributing

pip install build
python -m build
# produces dist/myproject-0.2.0.tar.gz and *.whl

Distribution options:

• PyPI — public; pip install twine; twine upload dist/*
• Private PyPI — devpi, AWS CodeArtifact, Azure Artifacts, Google Artifact Registry, GitHub Packages
• Internal git URL — pip install git+ssh://git@github.com/org/repo.git
• OCI image — for Lambda / Cloud Run / Kubernetes, package as a container instead

Project Layout: src vs Flat

Two common layouts:

src/
  myproject/
    __init__.py
    cli.py
tests/
  test_cli.py
pyproject.toml

vs flat:

myproject/
  __init__.py
  cli.py
tests/
  test_cli.py
pyproject.toml

The src layout is recommended for libraries and tools — it forces you to install your package to import it, which catches bugs where tests accidentally pass only because of relative imports from the project root. For internal scripts, flat is fine.

Pinning Python Itself

Tell uv (or pyenv) which Python version a project needs:

echo "3.12" > .python-version
# uv and pyenv both honour this file

Combined with the lockfile, anyone cloning the repo can run uv sync and get an identical environment.

The Minimum You Should Do

For any project beyond a single file:

1. Create a venv (or let uv manage one)
2. Write a pyproject.toml
3. Pin dependencies in a lockfile, commit it
4. Add a .python-version
5. If it has a CLI, expose it via [project.scripts]
6. Add ruff, mypy, and pytest as dev dependencies

That's about ten minutes of setup that pays off forever.