Learn › Linux Foundations › Heredocs
Final test: Chapter 12, Heredocs. Twenty-five objectives on a real Debian 12 server: here documents filed and appended, an unquoted and a quoted closing word, the tab-stripping <<- form, here strings, and process substitution. Fifty minutes, graded on the bytes of the files you write and on the commands you run.
This chapter covers the following topics:
What you already know: Chapter 11 taught the three standard streams and the redirections that move them. You can send output to a file with >, add to a file with >>, take input from a file with <, and feed one command from another with a pipe. This chapter is about the third source of input, the one Chapter 11 named and left: text you type on the spot, inside the command itself.
Objectives
After this chapter you can:
Before the practice machine starts, a recorded exercise card rehearses only the opening line and shows example output from admin@server1. Once the machine is running, type the opening command, every body line, and the closing word in the live terminal. Read your own file or output to check what the shell did. The shared practice machine keeps your files while its session is active, even when you open another lesson; a newly opened terminal starts a new shell. The chapter assessment starts a separate Debian 12 server. Use the workspace timer for its remaining time; its checks inspect commands and resulting files.
Five questions before the material. A wrong answer costs nothing: the explanation states the fact and names the section that teaches it.
>>> Enter usually means run. A here document is the exception: the shell collects every line until the closing word appears on its own, then hands them all to the command as its input. Section 12.1.
>>> The word after the two arrows is yours to pick. EOF is only a habit. Whatever you pick, the document ends when that word appears alone on a line, and the word itself is never part of the text. Section 12.1.
>>> Quoting the closing word turns off every substitution for the whole document. Leaving it bare lets the shell fill in variables and command substitutions before the command sees the text. Sections 12.5 and 12.6.
>>> The dash form removes leading tab characters and nothing else, so a document can be indented to match the code around it. Indent with spaces and the spaces are filed. Section 12.7.
>>> The shell runs the command, connects its output to a file descriptor, and passes the name of that descriptor as the argument. The receiving command opens it like any other file. Section 12.11.
So far the examples have used complete commands on one line. A here document lets you supply several lines of input. You write two less-than signs and a word of your choosing after a command, press Enter, and the shell keeps reading lines from you. Finish by typing that word alone on a line, then the command runs.
The word is called the delimiter, and it is the closing word of the document. Most engineers write EOF, short for end of file, out of habit. Any word works, and a word that cannot appear in your text is the safer choice.
In a plain Bash terminal, the shell prints > before each new line. That is the continuation prompt: the command has not started yet. This lab's browser terminal uses a multiline editor instead. It shows -- MULTILINE -- and keeps your lines together while you type. Press Enter after the closing word to run the command. Both displays lead to the same Bash here document. Chapter 11 named the three standard streams. The lines you type here become the standard input of the command, the stream numbered 0, exactly as if they had come out of a file.
Do not type a displayed > prompt or the -- MULTILINE -- label. Type the closing word with the same spelling and case, with no spaces before or after it. If you get stuck, Ctrl-C cancels the unfinished command so you can start again. In plain Bash, Ctrl-D means end of input and may run the command with the lines collected so far; use Ctrl-C to cancel.
You start this chapter in your home directory, and every file it writes belongs in ~/messages. Later sections write files, so each of those paths is written with ~/ in front of it: that way the file lands in the messages directory whatever directory your shell is standing in. The first two exercises write nothing at all.
In the running terminal, type all four lines below, pressing Enter after each one. Before the machine starts, the recorded card rehearses the opening line only. On the real shell, that line starts a document; the command runs after the closing word. Inspect your actual output before continuing.
cat <<EOF
a here document holds several lines
the shell reads them all before the command runs
EOFprompt: admin@server1:~$ answer: cat <<EOF ||| cat << EOF output: a here document holds several lines the shell reads them all before the command runs hint: Section 12.1. The reader from Chapter 5, then two less-than signs and the closing word most engineers use, EOF. Nothing is written to disk, so it runs the same from any directory.
The two lines came back. cat with no file name reads standard input, and the document was that standard input, typed by hand. The closing word never appears in the output, because it is punctuation rather than text: it tells the shell where to stop reading, and then it is thrown away.
A here document is not a cat trick. It feeds any command that reads standard input, and Chapter 5 taught several: wc counts, tac reverses, head and tail take the ends.
The closing word is yours to choose, so this document uses COUNT to show that nothing depends on the letters EOF.
Type these five lines:
wc -l <<COUNT
first line
second line
third line
COUNTprompt: admin@server1:~$ answer: wc -l <<COUNT ||| wc -l << COUNT output: 3 hint: Section 12.2. The counting tool with its line option -l, then two less-than signs and the closing word COUNT.
Three, and no file name after it. Chapter 5 pointed out that wc prints the file name only when it was given one as an argument. Here it was given nothing to open, so it read standard input and had no name to print. The blank space where a name would be is how you know a command read a stream rather than a file.
A document on its own prints to the screen. Add the redirection Chapter 11 taught and the same document writes a file instead, in one move, with no editor involved. This is what here documents are used for on real servers: a configuration file, a short script, a note, written from the command line and repeatable.
The order on the line does not matter to the shell. cat > file <<EOF and cat <<EOF > file do the same thing. The form with the redirection first reads more naturally: send the output there, and take the input from here.
The file to write is ~/messages/greeting.txt. Type these five lines. They are the three lines of the first graded objective, so type them exactly:
cat > ~/messages/greeting.txt <<EOF
Chapter 12 notes
a here document feeds text to a command
the shell reads until the closing word
EOFprompt: admin@server1:~$ answer: cat > ~/messages/greeting.txt <<EOF ||| cat >~/messages/greeting.txt <<EOF ||| cat > messages/greeting.txt <<EOF output: hint: Section 12.3. The reader, the greater-than sign > and the path ~/messages/greeting.txt, then two less-than signs and the closing word EOF. Written with ~/ the file lands in the messages directory from anywhere.
Nothing came back, and that is correct. The output of cat went to the file instead of the screen, and a document that is filed prints nothing at all. Silence after a redirection is the normal answer. The next step reads the file to prove the bytes arrived.
A document that is filed prints nothing, so the only proof is the file. Read ~/messages/greeting.txt the ordinary way.
prompt: admin@server1:~$ answer: cat ~/messages/greeting.txt ||| cat messages/greeting.txt output: Chapter 12 notes a here document feeds text to a command the shell reads until the closing word hint: Section 12.3. The reader from Chapter 5 with the path to the file you just wrote, ~/messages/greeting.txt.
Three lines, in the order they were typed, and no closing word among them. The document became the file exactly as dictated. The graded lab compares these bytes with a copy it keeps, so a stray space or a changed word is the difference between a passed objective and an open one.
Chapter 11 separated the two output redirections: > replaces what the file held, >> adds to the end of it. A here document works with either one, which is how a file gets built up in stages.
Start the file with two lines:
cat > ~/messages/roster.txt <<ROSTER
web server
database server
ROSTERprompt: admin@server1:~$ answer: cat > ~/messages/roster.txt <<ROSTER ||| cat >~/messages/roster.txt <<ROSTER ||| cat > messages/roster.txt <<ROSTER output: hint: Section 12.4. The same shape as Exercise 12-3, with the path ~/messages/roster.txt and the closing word ROSTER.
Silent again. The file now holds two lines, and the next document adds two more without touching them.
The only change is the redirection. The closing word may be the same word: each document is closed by its own opening line, so reusing ROSTER cannot confuse the shell.
Type these four lines:
cat >> ~/messages/roster.txt <<ROSTER
mail server
backup server
ROSTERprompt: admin@server1:~$ answer: cat >> ~/messages/roster.txt <<ROSTER ||| cat >>~/messages/roster.txt <<ROSTER ||| cat >> messages/roster.txt <<ROSTER output: hint: Section 12.4. The appending redirection from Chapter 11, >>, with the same path ~/messages/roster.txt, then the document opened with ROSTER.
Silent, like every filed document. Had you written one greater-than sign by mistake, the first two lines would be gone and only the last two would be in the file. That mistake is silent too, which is why the read-back matters.
Read ~/messages/roster.txt back. Four lines should come back, in the order the two documents were typed.
prompt: admin@server1:~$ answer: cat ~/messages/roster.txt ||| cat messages/roster.txt output: web server database server mail server backup server hint: Section 12.4. The reader with the path ~/messages/roster.txt.
The first document is at the top and the second is under it. A file built this way is built in the order you wrote it, and each document is a complete, repeatable step.
Everything so far went into the file exactly as typed, because nothing in it needed filling in. Now the important rule of this chapter.
When the closing word on the opening line is left unquoted, the shell expands expressions in the body. Chapter 8 taught the two forms used here: a variable written $NAME is replaced by its value, and a command substitution written $(command) is replaced by what that command printed. Both replacements happen before the command at the front of the line sees the text. Quotes inside the body are ordinary text; they do not turn these expansions off.
The document below carries both. $USER is a variable the shell keeps, holding the name of the account you are logged in as. $(hostname) runs the command Chapter 1 taught and puts its answer in place.
In an unquoted document, a backslash can protect a dollar sign, a backtick or another backslash. For example, \$USER writes the five characters $USER instead of the account name. A backslash followed by a newline joins those two lines. Before other characters, the backslash is kept. To keep the whole body literal, use the quoted delimiter in the next section.
Type these five lines:
cat > ~/messages/signed.txt <<REPORT
daily report
written by $USER
on host $(hostname)
REPORTprompt: admin@server1:~$ answer: cat > ~/messages/signed.txt <<REPORT ||| cat >~/messages/signed.txt <<REPORT ||| cat > messages/signed.txt <<REPORT output: hint: Section 12.5. The same shape as before, the path ~/messages/signed.txt, the closing word REPORT left bare with no quotes.
Silent, as always when a document is filed. What went into the file is not what you typed, and the next step shows the difference.
Read ~/messages/signed.txt back and see what the shell put in place of the two dollar expressions:
prompt: admin@server1:~$ answer: cat ~/messages/signed.txt ||| cat messages/signed.txt output: daily report written by admin on host server1 hint: Section 12.5. The reader with the path ~/messages/signed.txt.
You typed five characters, $USER, and the file holds admin. You typed eleven characters, $(hostname), and the file holds server1. The shell did both replacements while it was collecting the lines, so cat was handed finished text and simply copied it. This is how one document writes a file that names the machine it was written on.
Sometimes the dollar sign itself is what you want in the file. A script that uses $USER at the time it runs, or a note that quotes a variable by name, both need the characters to survive the writing.
Quote the closing word on the opening line to keep the body literal: the shell leaves its dollar expressions and backslashes untouched. Any quoting counts: single quotes, double quotes, or a backslash in front of the word. Single quotes are the usual spelling, <<'REPORT'. The quotes appear on the opening line only; the line that closes the document is still the bare word.
Type these five lines. The three middle lines are the same three lines as before:
cat > ~/messages/literal.txt <<'REPORT'
daily report
written by $USER
on host $(hostname)
REPORTprompt: admin@server1:~$ answer: cat > ~/messages/literal.txt <<'REPORT' ||| cat >~/messages/literal.txt <<'REPORT' ||| cat > messages/literal.txt <<'REPORT' output: hint: Section 12.6. The same line as Exercise 12-8 with the closing word REPORT in single quotes and the path ~/messages/literal.txt.
Silent. The quotes on the opening line changed nothing you can see yet, which is exactly why the read-back is part of the work.
Read ~/messages/literal.txt back and compare it with the file from Section 12.5:
prompt: admin@server1:~$ answer: cat ~/messages/literal.txt ||| cat messages/literal.txt output: daily report written by $USER on host $(hostname) hint: Section 12.6. The reader with the path ~/messages/literal.txt.
The same characters you typed, all of them. Two files now hold the same three typed lines and different bytes, and the only difference between the commands that wrote them is a pair of quotes around the closing word. That pair of quotes is the whole rule: bare means fill in, quoted means leave alone.
A here document usually sits inside something indented, a loop or a function in a script. The plain form keeps whatever you put in front of a line, so an indented document files its own indentation. The plain form is also strict about the end: the closing word closes the document only when it starts at the left margin.
See it happen first. Copy the full four-line example below and paste it into the browser terminal, then press Enter. The two middle lines begin with real tab characters. Pasting the complete example preserves those tabs; spaces would produce different file bytes. Read the file with xxd in the next step to check.
cat > ~/messages/kept.txt <<KEEP
first indented line
second indented line
KEEPprompt: admin@server1:~$ answer: cat > ~/messages/kept.txt <<KEEP ||| cat >~/messages/kept.txt <<KEEP ||| cat > messages/kept.txt <<KEEP output: hint: Section 12.7. The same shape again, the path ~/messages/kept.txt, the closing word KEEP, and the two body lines indented with a tab.
Silent. Reading this file with cat would show the indentation but would not tell you whether it is tabs or spaces, because both look the same on a screen. The byte viewer settles it.
Chapter 5 taught xxd, which prints a file as numbers: the position on the left, the bytes in the middle, and the printable characters on the right. A tab is byte 09. Print the bytes of ~/messages/kept.txt.
pr
Practice Chapter 12. Heredocs: <<, <<-, <<< and process substitution in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.