Learn › Bash Redirection and File Descriptors › Safer file creation and replacement

Chapter 12. Safer file creation and replacement

Safer file creation and replacement. Try the examples, inspect their output, and explain the result. Practice feedback does not award XP or mastery.

Keep yesterday's report

A nightly conversion reads a report and writes a cleaner version. Someone chooses the same filename for input and output. Bash prepares the redirects before the converter runs. Opening the output truncates the shared file, so the converter sees empty input. Changing redirect order does not fix this defect.

This lesson replaces that fragile operation with a sequence you can explain: prepare a separate file, transform, validate, then publish. The example report is CSV, text with one row per line and fields separated by commas. Publish here means replacing one report pathname inside your private practice directory. It does not mean uploading anything.

Use Bash and GNU coreutils in a disposable Linux directory. You need no administrator access. Keep every example inside this private directory. Existing documents, shared folders, and system configuration are unsuitable practice targets.

In your managed Bash practice terminal

Run camp-redirection 12 to enter this chapter's private directory. While this lab machine is running, the same command returns to it after reconnecting; camp-redirection 12 --fresh creates a new one without deleting earlier files. Run the code blocks there and use cd to leave.

Quote the path and choose who can read it

A quoted expansion, such as "$report", supplies one pathname even when it contains spaces. Quoting controls shell interpretation; it does not grant permission or make an untrusted directory safe.

The umask removes permissions from those a program requests when creating a file. With the ordinary file creation request of 666, a mask of 077 leaves owner read and write permissions. It does not change an existing file's mode. Default access control lists can affect creation rules too.

Predict whether the quoted pathname creates one file or several, and which shell keeps the umask change. Read the file and inspect its mode afterward.

(
  umask 077
  report='daily report.txt'
  printf 'previous report\n' > "$report"
  cat -- "$report"
  stat -c 'mode=%a' -- "$report"
)

Expected output

previous report
mode=600

GNU utilities accept -- to end option parsing. Quoting and -- solve different problems: preserving one argument, and preventing a leading hyphen from becoming an option.

Use noclobber as a deliberate guard

Bash's set -C enables noclobber, which rejects ordinary > replacement of an existing regular file. It is useful when accidental overwriting is the immediate risk. It does not validate your report, serialize writers, or protect every kind of special file.

(
  printf 'keep\n' > guarded.txt
  set -C
  if { printf 'lost\n' > guarded.txt; } 2>/dev/null; then
    printf 'unexpected\n'
  else
    printf 'overwrite refused\n'
  fi
  cat guarded.txt
  printf 'replacement\n' >| guarded.txt
)

The output is overwrite refused, then keep. Bash refuses the ordinary overwrite. The last redirect deliberately uses >| to allow replacement.

The outer diagnostic redirect keeps a variable shell error message out of this transcript. The branch still checks the failing open. Suppressing a message without checking status would hide the failure.

Prepare a unique neighbor

A temporary file holds work that is not yet accepted. GNU mktemp creates a new file rather than merely suggesting an unused name. Avoid predictable names assembled from a process ID or a timestamp. Someone else could create that name first.

Create the temporary file in the destination directory. This keeps the final rename within one filesystem and avoids a cross-filesystem copy. The directory must be controlled by you. A unique filename inside a directory another user can rearrange is not a complete security design.

(
  umask 077
  temporary=$(mktemp './.report.tmp.XXXXXX') || exit 74
  trap 'status=$?; rm -f -- "$temporary"; exit "$status"' EXIT
  printf 'name,count\nELM,4\n' > "$temporary" || exit 74
  if grep -qx 'ELM,4' "$temporary"; then
    mv -T -- "$temporary" 'published report.csv' || exit 74
    cat -- 'published report.csv'
  else
    printf 'validation failed\n' >&2
    exit 65
  fi
)

Expected output is a two-line CSV: the header name,count, then ELM,4. This tiny validator checks one known row only. A real report validator must check the complete format and all required records.

Separate transformation from acceptance

The previous example has four failure points: creating the temporary file, writing it, validating it, and renaming it. Each is checked before the next step. A zero transformation status is necessary, but it cannot prove that output has the required meaning.

The exit trap runs when this subshell exits. It saves the current status before cleanup commands can replace it. Removing the temporary name after a successful rename is harmless because that name is already gone. GNU mv -T treats the destination as one target pathname, so a directory is not silently accepted as a container.

In a longer script, add explicit interrupt and termination handling. Traps cannot run after SIGKILL or a power loss. A leftover temporary file is possible; the old report should still be recognizable. Never run a broad wildcard cleanup in a shared directory.

Try a failure before looking at the repair

Your independent task is to keep previous report intact when a transformer writes partial report and returns status 23. Write the partial output to a temporary file. Preserve the transformer's status immediately. Publish only if both transformation and validation succeed.

Before reading the worked path, recall Chapter 2's input-first repair and explain why it does not protect a report once this transformer starts writing. Predict the old report's bytes after a direct > and after a rejected candidate, then inspect both outcomes in your private directory. Note whether you used the Chapter 2 explanation as a hint.

Before running your version, identify the only command allowed to change the accepted report pathname. If your answer is the transformer's > redirect, the design still exposes the old file.

Compare your attempt with this worked failure path:

printf 'previous report\n' > accepted.txt
(
  temporary=$(mktemp './.accepted.tmp.XXXXXX') || exit 74
  trap 'status=$?; rm -f -- "$temporary"; exit "$status"' EXIT
  if bash -c 'printf "partial report\n"; exit 23' > "$temporary"; then
    mv -T -- "$temporary" accepted.txt
  else
    status=$?
    exit "$status"
  fi
)
status=$?
printf 'status=%s\n' "$status"

It prints status=23. Capture the failed command in the else branch before running another command.

This example deliberately takes only the failure branch. Its unexecuted success branch is not a complete publication recipe: add the content validator before mv, as shown in "Prepare a unique neighbor", before testing a successful transformation.

Try in the practice terminal: inspect the preserved report

cat accepted.txt

Expected result

previous report

This repair never redirects the transformer onto accepted.txt. The old report survives because rename is unreachable after status 23. Cleanup removes the rejected partial output while preserving that status.

Understand what replacement promises

On one filesystem, a successful rename can atomically replace a pathname. Atomic means an observer opening that pathname sees the old file or the new file, without an intermediate missing-name gap. A reader that already opened the old file can continue reading that old file. Rename does not change an existing descriptor's destination. Linux rename documentation.

Atomic replacement is different from crash durability, which concerns survival after a crash or power loss. A durable publication protocol must also address synchronizing file data and the containing directory. This Bash exercise does not implement that protocol. Linux fsync documentation.

Replacement also replaces the file's metadata with the temporary file's metadata. Decide ownership and permissions intentionally. Concurrent publishers still need an agreed coordination policy.

Ready to practice

Repeat your repair with a destination containing spaces, a missing source, and a validator that rejects otherwise successful output. Compare the old report byte for byte after each failed run. Confirm that no temporary files remain after ordinary completion.

Explain why a successful converter can still produce an unacceptable report, why the temporary file is a neighbor of the destination, and why rename alone does not promise durability. The next lesson adds competing writers and output destinations that are not ordinary files.

Practice Chapter 12. Safer file creation and replacement in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.