Learn › Bash Redirection and File Descriptors › Here-document edge cases and binary boundaries
Diagnose here-document edge cases. Try the examples, inspect their output, and explain the result. Practice feedback does not award XP or mastery.
A here document looks finished, but Bash keeps asking for input. A closing line can look correct while containing the wrong bytes: a space, a tab, or a carriage return can change what the shell recognizes.
This optional chapter uses a private practice directory and Bash. We inspect only small disposable scripts. Byte evidence means checking the actual stored characters, not trusting how an editor displays indentation or line endings.
If you are stuck at an interactive continuation prompt, press Ctrl+C before investigating. Canceling an unfinished body is a recovery step, not evidence that the earlier command completed successfully.
In your managed Bash practice terminal
Run camp-redirection 06 to enter this chapter's private directory. While this lab machine is running, the same command returns to it after reconnecting; camp-redirection 06 --fresh creates a new one without deleting earlier files. Run the code blocks there and use cd to leave.
A command may have several here documents. Bash reads their bodies in the order the redirects appear. If both redirects target fd 0, the last one supplies that command's input.
Predict which body cat will print. Run the block, then explain why Bash still had to read both bodies before executing cat.
cat <<'FIRST' <<'SECOND'
first body
FIRST
second body
SECONDOnly second body is printed. The first body still has to be present and correctly terminated. Replacing its destination does not remove it from the shell's syntax.
| Preparing cat | fd 0 | Body the shell must read |
|---|---|---|
| First redirect | Document A | FIRST body |
| Second redirect | Document B | SECOND body |
| Command starts | B | Both bodies have been consumed by parsing |
“Consumed by parsing” does not mean both bodies reached cat. Keep the shell's reading of source text separate from the command's reading of its input.
Choose a nonzero descriptor when one command needs two distinct inputs. The function below reads a label from fd 3, then ordinary content from fd 0. read -u 3 is Bash's way to select descriptor 3 explicitly.
show_inputs() { IFS= read -r -u 3 label; cat; printf 'label=%s\n' "$label"; }
show_inputs 3<<'LABEL' <<'CONTENT'
nightly
LABEL
report body
CONTENTThe output is report body, followed by label=nightly. In this function call, fd 3 refers to document A and fd 0 to document B. They are separate inputs.
If you instead duplicate fd 3 onto fd 0 using <&3, fd 3 must already be open at that point. Redirection order still applies to here documents. A later 3<<... cannot repair an earlier attempt to duplicate a closed descriptor.
<<- strips leading tab characters from body lines and the closing delimiter. It does not strip spaces. The following generator uses \t in printf to create a real tab, avoiding dependence on how your editor inserts the Tab key.
Both lines remain. The leading tab is removed by <<-, while the leading spaces remain; compare the indentation, not just the words.
printf "cat <<-'END'\n\ttab line\n space line\n\tEND\n" > indentation.sh; bash indentation.shExpected output
tab line
space lineBoth body lines end with a newline. Tabs in the middle of a line are not removed by this indentation rule. Ordinary << does not remove leading tabs either.
Prefer a quoted delimiter when the indented body should also stay literal. The minus sign controls tab stripping; quoting controls expansion. They solve different problems.
Linux text commonly uses LF, byte 0a, to end a line. CRLF adds carriage return 0d before LF. A trailing space is byte 20. An editor can hide each difference.
We create two deliberately broken closing lines. Only the closing line is changed; the opening delimiter remains ordinary END with an LF ending. The scripts are read as data in this step.
Compare the final-byte dumps: one closing line has a carriage return before newline, the other a trailing space; neither matches a bare END line.
printf "cat <<'END'\npayload\nEND\r\n" > crlf.sh
printf "cat <<'END'\npayload\nEND \n" > trailing.sh
printf 'CRLF closing line:'
tail -n 1 crlf.sh | od -An -tx1
printf 'spaced closing line:'
tail -n 1 trailing.sh | od -An -tx1Expected output
CRLF closing line: 45 4e 44 0d 0a
spaced closing line: 45 4e 44 20 0aAt end of file, Bash may warn that the expected delimiter was not found and supply the accumulated body anyway. Do not treat a warning as harmless or assume the command cannot run. Repair the source before using it for file changes.
For an unquoted delimiter, a backslash immediately followed by newline joins the two source lines. A quoted delimiter preserves the backslash and newline. Both bodies below otherwise contain identical text.
cat > joined.txt <<END
left\
right
END
cat > separate.txt <<'END'
left\
right
END
printf 'unquoted: '
cat joined.txt
printf 'quoted:\n'
cat separate.txtThe unquoted result is one line, leftright. The quoted result has left\ on one line and right on the next. Do not infer the result just by counting lines in the script source.
An unquoted body's \$ protects a dollar sign, and \\ produces one backslash. A backslash before an ordinary letter is retained. These are specific expansion rules, not a general “remove every backslash” rule.
A here document is shell source text. It is unsuitable as a general container for arbitrary binary data. Bash variables also cannot reliably store embedded NUL bytes, byte 00.
Keep binary content in a file or stream instead. This example creates three bytes directly with printf, without passing them through command substitution or a variable. od prints hexadecimal bytes so the NUL is visible.
Read the hexadecimal bytes and confirm the middle 00 byte remains in the file even though ordinary text display can hide it.
printf 'A\0B' > binary.dat; od -An -tx1 binary.datExpected output
41 00 42Quoting a delimiter prevents generation-time expansion. It is not a security boundary for a program that later treats that content as commands. Read generated source before executing it, and do not confuse literal storage with trust.
Try the repairs before reading the worked answer. Fix the CRLF closing line, fix the closing line with a trailing space, and rewrite the indented example so both output lines begin at column one. Preserve the body text intentionally.
For these tiny known fixtures, regenerating exact source is clearer than applying a broad whitespace-removal command to an unknown script.
printf "cat <<'END'\npayload\nEND\n" > crlf-fixed.sh
printf "cat <<'END'\npayload\nEND\n" > trailing-fixed.sh
printf "cat <<-'END'\n\ttab line\n\tspace line\n\tEND\n" > indentation-fixed.sh
bash crlf-fixed.sh
bash trailing-fixed.sh
bash indentation-fixed.shThe output is payload twice, then tab line and space line. The first two repairs restore an exact delimiter line. The third changes intentional body indentation to tabs so <<- removes it. It does not claim that spaces are stripped automatically.
Add one harmless line to each repaired body and inspect the bytes again. Verify that the delimiter itself never appears in command input. Then explain why the first of two here documents must still close even when the second replaces fd 0.
When diagnosing an unfamiliar script, record its delimiter spelling, quoting, indentation bytes, and line endings before changing it. This gives you a small, testable explanation instead of a series of guesses.
Next, study how a pipeline's initial wiring interacts with explicit redirections and which process owns a loop's variable changes.
Practice Chapter 6. Diagnose here-document edge cases in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.