Docker Image Optimization for Python

Quick Reference

Typical Results: 1.2GB → 300MB (75% reduction) with multi-stage slim builds

1. Multi-Stage Build Pattern (Primary Optimization)

Concept: Separate build stage (compilers, build tools) from runtime stage (only compiled artifacts).

Benefits: 70-90% size reduction, faster deployments, cleaner runtime

Pattern

# Stage 1: Build
FROM python:3.11-slim as builder
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends gcc \
    && rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Stage 2: Runtime
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH
CMD ["python", "app.py"]

Result: 1.2GB → 300MB (75% reduction)

2. Base Image Selection

Recommendation: Use python:3.11-slim unless you have specific needs.

Alpine warning: Breaks packages with C extensions (numpy, scipy, pillow). Use only for pure Python apps.

3. Distroless Images (2025 Best Practice)

What: Minimal images with ONLY runtime dependencies. No shell, package manager, or utilities.

Security: Highest - no shell for attackers, fewer CVEs, meets strict compliance requirements.

Trade-off: Can't debug with docker exec. Use --target builder to debug.

Pattern

# Build stage
FROM python:3.11-slim as builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Distroless runtime
FROM gcr.io/distroless/python3-debian11
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH PYTHONUNBUFFERED=1
CMD ["app.py"]

Debugging:

# Debug build (has shell)
docker build --target builder -t myapp:debug .

# Production (no shell)
docker build -t myapp:prod .

4. Layer Caching Optimization

Key Rule: Order layers from least-changing to most-changing.

Optimal order:

1. System packages (rarely change)
2. requirements.txt (occasional changes)
3. Application code (frequent changes)

Pattern

FROM python:3.11-slim

# 1. System packages (cached unless Dockerfile changes)
RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 && rm -rf /var/lib/apt/lists/*

# 2. Python dependencies (cached unless requirements.txt changes)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# 3. Application code (rebuilt on every code change)
COPY . .

CMD ["python", "app.py"]

Impact: Fixing typo in app.py: 5 minutes → 10 seconds (with good layer order)

5. Dependency Installation Best Practices

Essential Flags

Pattern

# Install system dependencies + clean cache in SAME command
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

Critical: Clean apt cache in SAME RUN command (separate commands = cache persists in earlier layer)

Requirements.txt: Pin exact versions (flask==2.3.2, not flask>=2.3)

6. Image Size Reduction Checklist

• Use multi-stage builds (exclude build tools)
• Use slim/alpine base images
• Add .dockerignore file
• Use --no-cache-dir with pip
• Clean apt cache in same RUN command
• Combine RUN commands to reduce layers

.dockerignore Essential Patterns

.git/
__pycache__/
*.py[cod]
venv/
.venv/
.pytest_cache/
.coverage
.vscode/
.idea/
*.md
docs/
.github/
*.log

Impact: 500MB → 50MB build context (10x reduction)

Size Progression

7. Security Scanning with Trivy

Installation & Basic Usage

# Install (macOS)
brew install aquasecurity/trivy/trivy

# Scan image
trivy image python:3.11-slim
trivy image --severity HIGH,CRITICAL myapp:latest

Fixing Vulnerabilities

1. Update base image: FROM python:3.11.8-slim (newer patch)
2. Update dependencies: pip install --upgrade <package>
3. Rebuild and rescan: docker build . && trivy image myapp:latest

Best Practices

• Scan base images before using
• Fail CI/CD on CRITICAL vulnerabilities
• Scheduled scans of deployed images (weekly)
• Use distroless for fewer vulnerabilities

See reference.md for CI/CD integration examples (GitHub Actions, GitLab CI).

8. Build Performance Optimization

Enable BuildKit

# Enable globally
export DOCKER_BUILDKIT=1
echo 'export DOCKER_BUILDKIT=1' >> ~/.bashrc

# Enable per-build
DOCKER_BUILDKIT=1 docker build .

Cache Mounts (BuildKit)

# Reuse pip cache across builds
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Benefit: 2 minute builds → 10 seconds after first build

Build Performance Checklist

• Enable BuildKit (DOCKER_BUILDKIT=1)
• Use cache mounts for package managers
• Order layers for maximum cache hits
• Use .dockerignore to reduce context size

Typical improvements: 10 minute builds → 30 seconds (with warm cache)

9. Python-Specific Optimizations

Essential Environment Variables

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

Non-Root User (Security)

# Create non-root user
RUN useradd -m -u 1000 appuser
USER appuser
WORKDIR /home/appuser/app
COPY --chown=appuser:appuser . .

Why: Containers as root = security risk if compromised

10. Health Checks

Basic Pattern

HEALTHCHECK --interval=30s \
            --timeout=3s \
            --start-period=5s \
            --retries=3 \
    CMD python -c "import requests; requests.get('http://localhost:8000/health')"

Common Patterns

See reference.md for implementation details and app-specific patterns.

11. Quick Pattern Reference

See reference.md for complete Dockerfile examples.

12. Anti-Patterns to Avoid

13. Quick Reference Card

Dockerfile Checklist

# ✅ Multi-stage build
FROM python:3.11-slim as builder
# ... build stage ...

FROM python:3.11-slim
# ✅ Python env vars
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PATH=/root/.local/bin:$PATH

# ✅ Copy only dependencies first (cache-friendly)
COPY --from=builder /root/.local /root/.local

# ✅ Non-root user
RUN useradd -m -u 1000 appuser
USER appuser

# ✅ Copy app last
COPY --chown=appuser:appuser . .

# ✅ Health check
HEALTHCHECK CMD curl -f http://localhost:8000/health || exit 1

# ✅ Explicit CMD
CMD ["python", "app.py"]

Build Commands

# Enable BuildKit
export DOCKER_BUILDKIT=1

# Build with tag
docker build -t myapp:latest .

# Build specific stage (debugging)
docker build --target builder -t myapp:debug .

# Scan for vulnerabilities
trivy image --severity HIGH,CRITICAL myapp:latest

Additional Resources

Documentation:

• Docker multi-stage builds: https://docs.docker.com/build/building/multi-stage/
• BuildKit: https://docs.docker.com/build/buildkit/
• Distroless images: https://github.com/GoogleContainerTools/distroless
• Trivy security scanner: https://aquasecurity.github.io/trivy/

Best Practices:

• Docker best practices: https://docs.docker.com/develop/dev-best-practices/
• Python Docker guide: https://docs.python.org/3/using/docker.html

Security:

• CIS Docker Benchmark: https://www.cisecurity.org/benchmark/docker
• OWASP Docker Security Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html

For comprehensive examples and advanced patterns: See reference.md

Last Updated: 2025-10-27 Version: 1.1