Process Substitution

diff - a hands-on Linux lab on a real virtual machine.

A command dressed as a file: process substitution <(...) lets diff compare two live commands with no temp files, a one-line ; echo $? reads the identical-or-not verdict, and comm splits two sorted lists into shared and unique columns. Drilled in the practice terminal, then rebuilt from memory in three challenges.

diff is the comparison tool: hand it two files and it reports what changed between them, line by line. It gets a full lesson of its own later, in the Text Processing section. Today it is the test subject, because diff has one strict demand. It wants files. Real ones, on disk, with names.

Now the standoff. The two lists you need to compare are the output of two commands. They live nowhere. No files, no filenames, nothing for diff to open. You could save each output to a temporary file, run the comparison, then delete both files. Three steps and a cleanup, for one question.

Bash has a sneakier answer: an operator that dresses a running command up as a file, so convincingly that diff opens it, reads it, and never notices. It is called process substitution, written <(...), and it is the one move this lesson teaches. By the end you will compare two live commands in a single line, with zero files on disk, and read the verdict two different ways.

The black boxes below are a practice terminal: a safe sandbox that only checks the one command each step teaches, so you cannot break anything. On the real machine at the end of the module, the heredocs capstone mission saves these same lists as real files and you run the moves for real. One requirement matters here: the lab shell is bash, and <(...) is a bash feature. Process substitution first appeared in the Korn shell in the 1980s; bash adopted it, but the plain POSIX sh never did.

Time to catch bash in the act. The command true does nothing and prints nothing (it has a tiny lesson of its own later). So here is the experiment: if <(true) simply pasted in the output of true, then echo <(true) should print an empty line.

Before you read the result below, commit to a prediction: an empty line, or something else?

echo <(true)
/dev/fd/63

Something else. echo printed a file path. That is the secret of process substitution: <(command) does not become the command's text. Bash runs the command, parks its output behind a special file, and substitutes the path to that file. Any program that opens the path reads the command's output, exactly as if it were reading a file someone had saved.

The exact path varies on your machine. Here it reads /dev/fd/63, but it might be /dev/fd/62, a /proc/self/fd/... form, or a named-pipe path, depending on the system and how many files are open at that moment. Do not memorize the number; that is why this example is read-only instead of graded. The idea is what counts: <(...) becomes a file path the outer command can open and read. Every drill you type from here on has fixed, exact output.

Now hand diff two commands instead of two files. printf prints exactly the text you give it, and \n inside its argument means "start a new line," so printf 'alice\nbob\n' prints two lines. Below, each printf prints a four-name roster. The rosters are identical except the last name: dave on the first, erin on the second.

Before you run it, decide: will diff echo back all eight names, or only the part that changed?

diff <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n')

prompt: student@linuxcamp:~$ answer: diff <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n') output: 4c4 < dave --- > erin hint: diff, a space, then each roster wrapped in <(printf '...'). The full line is shown in the step above the box.

Only the change. Read the report top to bottom. 4c4 means line 4 of the first roster changed into line 4 of the second. < dave (pointing left) is what the first side had; > erin (pointing right) is what the second side has instead; the --- just separates the two sides. The three matching names never appear at all: diff speaks only about lines that differ.

Mark the moment, because this is the whole lesson in one line. diff demanded files, you had none, and <(...) handed it two paths that read like files. You compared two live commands with nothing on disk and nothing to clean up afterward.

Sometimes you do not want the change report; you want a yes or no. Are the two outputs identical? diff answers that too, silently. Remember from the herestring lesson: every command leaves behind an exit status, 0 for success and a small non-zero number otherwise, and tacking ; echo $? onto the end of a line runs your command first, then prints that command's status.

diff uses the same convention you met with grep. Exit status 0 means the two inputs are identical, and in that case diff prints nothing at all. Exit status 1 means differences were found. Both digits are exact and reliable, so a one-line diff of two process substitutions plus ; echo $? is a true-or-false test for "do these two commands agree?" You will assemble that line yourself in a moment: it is the first challenge.

diff reports what changed. Sometimes the question is membership instead: which names do two lists share, and which belong to only one side? The tool for that is comm, and it pairs beautifully with two process substitutions. It has one rule: both inputs must be sorted. Ours already are, since alice, bob, carol, then dave or erin is alphabetical order.

comm prints three columns separated by tab characters. Column one is lines only in the first input. Column two, pushed right by one tab, is lines only in the second. Column three, pushed right by two tabs, is lines in both. Before you run it, decide where dave will land: flush left, one tab in, or two tabs in?

comm <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n')

prompt: student@linuxcamp:~$ answer: comm <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n') output: alice bob carol dave erin hint: comm in front this time, then the same two wrapped rosters as the diff drill. The full line is shown in the step above the box.

dave landed flush left: column one, only in the first roster. erin sits one tab in: column two, only in the second. And alice, bob, and carol are pushed two tabs right: column three, in both. One glance says the teams share three members and differ by one on each side. comm when you want the overlap, diff when you want the change, and both fed straight from <(...) with no files on disk.

Scaffolding off. No command is shown from here to the end; you assemble each line yourself.

A teammate wants proof that two color lists are identical, and wants it as a single digit from a single line. Each side is printf printing the string 'red\ngreen\nblue\n'. Compare the two sides, then print the verdict digit on the same line.

prompt: student@linuxcamp:~$ answer: diff <(printf 'red\ngreen\nblue\n') <(printf 'red\ngreen\nblue\n'); echo $? output: 0 hint: Two identical <(printf ...) sides after diff, then a semicolon and the status question: echo $?

0, the one-digit answer for identical. diff compared the two sides, found nothing to report, printed nothing, and left exit status 0; your ; echo $? then put that digit on screen. Had even one color differed, you would have seen a change report and a 1 instead. You just built a yes-or-no test for two commands in one line.

Back to the two rosters, this time with no line to copy. The first roster is printf printing 'alice\nbob\ncarol\ndave\n'; the second is the same but ends in erin. Compare them so the output names only what changed.

prompt: student@linuxcamp:~$ answer: diff <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n') output: 4c4 < dave --- > erin hint: diff, then each roster wrapped in <(printf '...'), with \n between and after the names.

4c4, < dave, > erin: line four changed, dave out, erin in, and this time you produced the report from memory. That recall is the point. The shape diff <(...) <(...) is now yours to reuse on any two commands whose output might drift apart: two configs, two directory listings, two versions of anything.

Last one. Same two rosters, but now the teammate wants the membership picture: which names are shared and which are unique to each side, in the three-column layout where shared names sit furthest right.

prompt: student@linuxcamp:~$ answer: comm <(printf 'alice\nbob\ncarol\ndave\n') <(printf 'alice\nbob\ncarol\nerin\n') output: alice bob carol dave erin hint: The column tool goes in front, and the two wrapped rosters follow, exactly like the diff form but with comm.

Three shared names two tabs deep, dave flush left, erin one tab in. You drove all three moves with zero prompting: the change report, the one-digit verdict, and the overlap. Process substitution is officially in your hands.

You earned this cheat sheet. Every row is a move you just ran, or rebuilt from memory, in the practice terminal:

Place it beside the operators from the last three lessons and the family is complete. << feeds a block of lines, <<- feeds an indented block with the leading tabs stripped, <<< feeds one string, and <(...) feeds a whole command's output dressed as a file. All four hand input to a command with no temporary file in sight.

Remember <(...) by its shape: a command inside parentheses, wearing a file's clothes. And remember comm's one rule: both inputs sorted, or the columns lie.

The practice terminal has walked you through the whole move. You saw <(...) expand to a file path, used it to let diff compare two live commands, read the one-digit verdict with $?, and split two lists into shared and unique columns with comm.

The Heredocs module ends with one real Linux machine: the heredocs capstone mission. There a bash VM boots just for you with the same team and color lists saved as real files, and the mission hands you objectives with no commands shown. You read the objective, recall the operator, and type it. That recall is what makes it stick.

Finish the other Heredocs lessons, then go run it for real.

Practice Process Substitution in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.