Learn › Bash Redirection and File Descriptors › Here documents as a parsing and expansion boundary

Chapter 5. Here documents and expansion

Here documents and expansion. Try the examples, inspect their output, and explain the result. Practice feedback does not award XP or mastery.

Decide which shell should expand the text

You are generating a small script. Its output file should contain $USER, but the generated file already contains your username. Bash expanded the text while creating the file, before anyone ran the new script.

A here document supplies several lines of input to a command. The token after << identifies its delimiter, the line that ends the body. Quoting that delimiter determines whether Bash expands the body before supplying it as input.

This optional lesson uses Bash in a private practice directory. It separates two jobs: generating literal source text and generating a report from current values. Both are useful; choose deliberately.

In your managed Bash practice terminal

Run camp-redirection 05 to enter this chapter's private directory. While this lab machine is running, the same command returns to it after reconnecting; camp-redirection 05 --fresh creates a new one without deleting earlier files. Run the code blocks there and use cd to leave.

Generate literal text first

Place quotes around the delimiter word when the body should reach the command unchanged. The closing delimiter line itself contains no quotes. It must match the resulting delimiter exactly, without leading indentation or trailing spaces for ordinary <<.

Before running the block, predict whether literal.txt will contain the characters $USER and $(printf changed), or their expanded values. Explain what the quotes on TEXT control.

cat > literal.txt <<'TEXT'
user=$USER
command=$(printf changed)
legacy=`printf changed`
TEXT

cat reads its input and writes it to literal.txt. The body is input data; it is not a list of commands being executed by cat. Neither form of command substitution runs during this generation step.

Try in the practice terminal: inspect literal source text

cat literal.txt

Expected result

user=$USER
command=$(printf changed)
legacy=`printf changed`

The file contains three newline-terminated lines. The delimiter is syntax used by the shell and is not included in the file.

Expand a report deliberately

Without quoting the delimiter, Bash performs parameter expansion, command substitution, and arithmetic expansion in the body. $name reads a variable, $(...) captures a command's output, and $((...)) evaluates arithmetic.

Here, expansion is the job. We set every variable ourselves so the report does not depend on your account.

name='Rin Park'
cat > expanded.txt <<REPORT
name: "$name"
count: $((2 + 3))
source: $(printf local)
pattern: *.txt
REPORT

Try in the practice terminal: inspect an expanded report

cat expanded.txt

Expected result

name: "Rin Park"
count: 5
source: local
pattern: *.txt

Unlike an unquoted argument list, a here-document body does not undergo word splitting or pathname expansion. The space in Rin Park stays in the text, and *.txt does not become filenames. These rules describe the body. A separate output target should still be quoted when its pathname comes from a variable.

Understand partial and empty quoting

Quoting any part of a delimiter word disables body expansion. Quote removal produces the delimiter Bash will recognize. For example, E"ND" produces the delimiter END, with expansion disabled.

An empty quoted delimiter is legal. Its closing delimiter is a completely empty line. This is harder to review because a blank line can be mistaken for spacing, so prefer a descriptive delimiter in ordinary scripts.

cat > partial.txt <<E"ND"
$USER stays literal
END
cat > empty-marker.txt <<''
one line

# The empty line above ends this document.

Try in the practice terminal: inspect two delimiter choices

cat partial.txt empty-marker.txt

Expected result

$USER stays literal
one line

Delimiter recognition does not require EOF specifically. REPORT, END, and another distinctive token are choices made by the author. Choose a token that will not occur as a complete line in the intended body.

Keep quoting layers separate

There are two different places to reason about quotes. Quotes in the delimiter word control body expansion. Quotes written inside the body are data at this stage.

For an unquoted delimiter, literal double quotes around $name do not prevent the name from expanding. For a quoted delimiter, the same dollar sign stays literal regardless of quotes inside the body. If the file is later executed as a shell script, that later shell interprets its contents according to normal script rules.

Unquoted bodies also have special backslash rules. A backslash can protect a dollar sign, a backtick, or another backslash; backslash followed by newline joins lines. The next chapter inspects those bytes explicitly. Avoid escaping an entire generated script character by character when a quoted delimiter expresses the intended policy.

Generate a script without running it

This example creates source text. The later command substitution is harmless, but it must remain text during generation.

cat > greeting.sh <<'SCRIPT'
printf 'User: %s\n' "$USER"
printf 'Value: %s\n' "$(printf later)"
SCRIPT

Try in the practice terminal: review before execution

cat greeting.sh

Expected result

printf 'User: %s\n' "$USER"
printf 'Value: %s\n' "$(printf later)"

The generated file still contains both expressions. When someone later executes the file with Bash, its $USER and $(printf later) will be evaluated then. Delimiter quoting changes generation-time behavior; it does not make the resulting program safe to execute.

Repair a generation mistake

Try this before reading the worked answer. Generate a file named show-target.sh containing exactly one command: printf '%s\n' "$target". It must read target when the generated script runs, not while you create it.

Do not solve this by guessing how many backslashes each dollar sign needs. Use a delimiter policy that preserves the whole body.

cat > show-target.sh <<'SCRIPT'
printf '%s\n' "$target"
SCRIPT
target=destination bash show-target.sh

The output is destination. The temporary assignment supplies target to the child Bash that runs the generated script. The quoted delimiter kept the variable reference in the file until that second shell evaluated it.

If the result is empty, inspect the file first. An already-expanded empty string points to a generation problem. An intact $target with an empty result points to the execution environment.

Ready to practice

Create two files about the same value. The first should record today's chosen value immediately. The second should contain a script that reads its value later. Change the value between creation and execution, then explain why the outputs differ.

When a here document seems unfinished at the continuation prompt, press Ctrl+C to cancel the unfinished command. Check the opening delimiter, quoting, and exact closing line before retrying. Do not paste more shell commands into a body you have not closed.

Next, use byte-level evidence to diagnose tabs, line endings, multiple documents, and text that looks like a closing delimiter but is not one.

Practice Chapter 5. Here documents and expansion in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.