File System Operations

Overview

Apply safe path handling and temporary file patterns so reads and writes are reliable, even with untrusted input or large intermediate data.

When to Use

• Use this skill only after the frontmatter triggers; otherwise start with a simple read/write path.

Decision Tree

1. Normalize or resolve the path?
   • Normalize only: use os.path.normpath() for lexical cleanup.
   • Resolve + check existence: use os.path.realpath(path, strict=True).
2. Are you writing to a target path that may already exist?
   • Yes: write to a temp file in the same directory and replace.
3. Is the file size unknown or potentially large?
   • Yes: use tempfile.SpooledTemporaryFile to avoid unnecessary disk writes.

Workflows

1. Normalize vs Resolve Paths

1. Normalize user input with os.path.normpath() to collapse redundant segments.
2. Resolve real paths with os.path.realpath(path, strict=True) when you must confirm the file exists.
3. Reject paths that fail resolution or escape your allowed base directory.

2. Atomic Replace Write

1. Create a temporary file in the destination directory.
2. Write and flush content to the temp file.
3. Replace the destination with os.replace/os.rename after the write completes.
4. Ensure the temp file and destination are on the same filesystem.

3. Spooled Temporary Buffer

1. Create tempfile.SpooledTemporaryFile(max_size=...).
2. Stream data into the spooled file as you process it.
3. Call .rollover() to force the buffer onto disk when needed.

Non-Obvious Insights

• normpath is lexical only and can change the meaning of a path with symlinks.
• realpath(..., strict=True) raises FileNotFoundError when the path does not exist.
• os.rename is atomic on POSIX when successful; keep temp and destination on the same filesystem.
• Temporary files may not have a visible name; do not rely on name visibility.
• Spooled temp files can be forced to disk with rollover() when a file grows.

Evidence

• "Normalize a pathname by collapsing redundant separators and up-level references so that A//B, A/B/, A/./B and A/foo/../B all become A/B. This string manipulation may change the meaning of a path that contains symbolic links." - Python Docs
• "FileNotFoundError is raised if path does not exist, or another OSError if it is otherwise inaccessible." - Python Docs
• "If successful, the renaming will be an atomic operation (this is a POSIX requirement)." - Python Docs
• "Rename the file or directory src to dst. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems." - Python Docs
• "TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers." - Python Docs
• "should not rely on a temporary file created using this function having or not having a visible name in the file system." - Python Docs
• "rollover() ... causes the file to roll over to an on-disk file regardless of its size." - Python Docs

Scripts

• scripts/file-system_tool.py: CLI for resolve, atomic write, copy, and safe reads.
• scripts/file-system_tool.js: Node.js CLI equivalents.

Dependencies

• Python standard library (os, tempfile, pathlib, shutil) or Node standard library (fs, path, os).

References

• references/README.md