Files, Paths, and CLI Arguments — Python for DevOps and Cloud | CertQnA

The first real DevOps Python you'll write usually reads a file or a few env vars and writes another file. This lesson covers the tools to do that cleanly and cross-platform.

Working with Paths: pathlib

Forget os.path. Use pathlib.Path — object-oriented, cross-platform, and far less error-prone:

from pathlib import Path

home = Path.home()
config_dir = home / ".myapp"
config_file = config_dir / "config.toml"

config_dir.mkdir(parents=True, exist_ok=True)
config_file.touch()

print(config_file.exists())       # True
print(config_file.is_file())      # True
print(config_file.suffix)         # '.toml'
print(config_file.stem)           # 'config'

for f in config_dir.glob("*.toml"):
    print(f)

Path handles the right separator on Windows vs Unix automatically. The / operator joins path segments — much cleaner than os.path.join().

Reading and Writing Files

from pathlib import Path

p = Path("notes.txt")

# Quick reads/writes (small files only — loads everything in memory)
p.write_text("hello\n")
content = p.read_text()

# Streaming, with-statement (any size)
with p.open("r", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip())

with p.open("a", encoding="utf-8") as f:
    f.write("another line\n")

# Binary
data = Path("image.png").read_bytes()

Always use with when opening files. It guarantees the file closes even if an exception is raised — and you'll get yelled at by linters if you don't.

JSON

import json
from pathlib import Path

config = {"region": "us-east-1", "instance_count": 3, "tags": ["prod", "web"]}

# Write
Path("config.json").write_text(json.dumps(config, indent=2))

# Read
data = json.loads(Path("config.json").read_text())
print(data["region"])

# Stream API
with open("big.json") as f:
    data = json.load(f)

YAML and TOML

# YAML — install: pip install pyyaml
import yaml
config = yaml.safe_load(Path("config.yaml").read_text())   # always safe_load, not load
Path("out.yaml").write_text(yaml.safe_dump(config))

# TOML reading is built-in (Python 3.11+)
import tomllib
config = tomllib.loads(Path("pyproject.toml").read_text())

# TOML writing — install: pip install tomli-w
import tomli_w
Path("config.toml").write_bytes(tomli_w.dumps(config).encode())

Environment Variables

import os

# Required — raises KeyError if missing
api_key = os.environ["API_KEY"]

# Optional with default
region = os.getenv("AWS_REGION", "us-east-1")

# Type conversion — env vars are always strings
debug = os.getenv("DEBUG", "false").lower() == "true"
port = int(os.getenv("PORT", "8080"))

For local development, store secrets in a .env file (gitignored) and load it with python-dotenv:

from dotenv import load_dotenv
load_dotenv()  # reads .env into os.environ

CLI Arguments: argparse

The standard-library option. No dependencies, gets you 80% of the way:

import argparse

def main() -> int:
    parser = argparse.ArgumentParser(description="Resize images.")
    parser.add_argument("source", help="source directory")
    parser.add_argument("--width", type=int, default=800)
    parser.add_argument("--quality", type=int, default=85)
    parser.add_argument("--dry-run", action="store_true")
    parser.add_argument("-v", "--verbose", action="count", default=0)

    args = parser.parse_args()
    print(args.source, args.width, args.dry_run)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Run ./resize.py --help and argparse generates a usage banner for free.

CLI Arguments: click and typer

For larger tools with subcommands, click or its type-hint sibling typer are cleaner:

# pip install typer
import typer

app = typer.Typer()

@app.command()
def deploy(env: str, dry_run: bool = False):
    """Deploy to ENV."""
    typer.echo(f"deploying to {env}, dry_run={dry_run}")

@app.command()
def rollback(env: str):
    """Rollback ENV to the previous version."""
    typer.echo(f"rolling back {env}")

if __name__ == "__main__":
    app()

Then ./tool.py deploy prod --dry-run works with full help text generated from the function signatures.

Stdin and Pipes

Real DevOps scripts often participate in shell pipelines:

import sys

# Read entire stdin
data = sys.stdin.read()

# Or line by line — friendly to large inputs
for line in sys.stdin:
    line = line.rstrip("\n")
    print(line.upper())

cat servers.txt | python upper.py

Putting It Together

"""Tag every JSON file in a directory with a 'tagged_at' timestamp."""
import json
from datetime import datetime, timezone
from pathlib import Path
import argparse


def tag_file(path: Path) -> None:
    data = json.loads(path.read_text())
    data["tagged_at"] = datetime.now(timezone.utc).isoformat()
    path.write_text(json.dumps(data, indent=2))


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("directory", type=Path)
    args = parser.parse_args()

    for json_file in args.directory.glob("*.json"):
        tag_file(json_file)
        print(f"tagged {json_file}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Cross-platform path handling, safe file I/O, JSON read/write, CLI args, sensible exit code — a complete small tool in 25 lines.