Learn › Linux Foundations › Text Processing
Final test: Chapter 13, Text Processing. Twenty-five objectives on a real Debian 12 server covering all three parts: printf, grep and its pattern language, cut, sort, uniq, tr, paste, join, comm, diff, patch, column, sed and awk. Fifty minutes, eleven objectives graded on the exact bytes of the files you produce.
A configuration change is useful only if you can see what changed and verify the resulting file. This part starts with a before/after difference, applies a prepared patch, then formats and transforms text for a report.
Sections in this part: 13.19 diff; 13.20 patch; 13.21 column; 13.22 sed; 13.23 awk. The chapter lab follows these examples.
This is Part 3 of 3. Part 1 covered Sections 13.1 to 13.12 and Part 2 covered Sections 13.13 to 13.18. This part covers Sections 13.19 to 13.23. Those sections teach the two tools that carry a change from one file to another, the tool that prints a table, and the two tools that are small programming languages of their own. It ends in the chapter lab, which covers all twenty-three sections.
What you already know: Section 13.3 taught the idea of selecting text with a pattern. grep, sed and awk do not always spell a pattern the same way, so follow each tool's first example before carrying an expression across. Section 13.10 taught fields and delimiters; awk also splits lines into fields. Section 13.16 built the CSV file that Section 13.21 prints as a table.
Objectives
After this part you can:
Use Start machine to open the real practice server. The recorded output beside each command is an example from the prepared fixture; run the command in Terminal and inspect what your machine actually reports. Commands can change its files, so check the result before continuing. A reconnect may start a new shell. Run pwd to check where you are and cd ~ to return home. Prompt colors and symbols can differ from the examples. Paths beginning ~/analytics/ work from any directory; shorter analytics/ alternatives assume you are in /home/admin. The chapter assessment opens on a separate fresh machine and checks the commands and results requested there.
Five questions before the material. A wrong answer costs nothing: the explanation states the fact and names the section that teaches it.
>>> A difference report names the position, shows the lines that were removed and the lines that were added, and in the unified form includes a few unchanged lines around them as context. Section 13.19.
>>> A difference report is a complete instruction for turning one version into the other, so a tool can apply it. That is what a patch file is. It is plain text and small, so it can be sent in a mail message, which is how changes to shared source code were passed around. Section 13.20.
>>> sed replaces the first match on every line by default. The g flag at the end of the substitute command makes it replace every match on every line. Section 13.22.
>>> sed reads a file and writes the result to standard output, leaving the file alone. Running it without the in-place option first, reading the result, and only then adding the option is the standard way to work. Section 13.22.
>>> cut always prints the fields it is given, in file order, on every line. awk tests a pattern on each line and runs an action only where the pattern is true. Its action can print the fields in any order, or compute something new. Section 13.23.
diff reads two files and reports what would have to change to turn the first into the second. It says nothing about lines that are the same in both.
The area holds two versions of the web server configuration, in ~/analytics/config. Check pwd; run cd ~ if needed. Read the first one, ~/analytics/config/site-v1.conf:
prompt: admin@server1:~$ answer: cat analytics/config/site-v1.conf ||| cat ~/analytics/config/site-v1.conf output:
listen 80 server_name northwind.example.com root /home/admin/website index index.html access_log /home/admin/logs/access.log gzip off hint: Section 5.1. The printing tool and the first version, ~/analytics/config/site-v1.conf.
Seven lines. The first is a comment, and the rest are settings: the port, the name, the directory the site is served from, the default page, the log and whether responses are compressed.
Give diff two file names. The order matters: the first is treated as the old version and the second as the new one. Compare ~/analytics/config/site-v1.conf with ~/analytics/config/site-v2.conf.
prompt: admin@server1:~$ answer: diff analytics/config/site-v1.conf analytics/config/site-v2.conf ||| diff ~/analytics/config/site-v1.conf ~/analytics/config/site-v2.conf output: 7c7,8 < gzip off --- > gzip on > client_max_body_size 8m hint: Section 13.19. The comparison tool with no options and the two versions, ~/analytics/config/site-v1.conf first, then ~/analytics/config/site-v2.conf.
The report is read as an instruction. 7c7,8 means line 7 of the first file CHANGES into lines 7 and 8 of the second. The lines marked < come from the first file and the lines marked > from the second, with a row of dashes between them. Two other letters appear in this position: a for added and d for deleted.
The option -u prints the unified report, which is the form every version control tool prints and the form patch reads. Instead of two separate groups it prints one block holding the removals, the additions and a few unchanged lines around them. Compare the same two files again.
prompt: admin@server1:~$ answer: diff -u analytics/config/site-v1.conf analytics/config/site-v2.conf ||| diff -u ~/analytics/config/site-v1.conf ~/analytics/config/site-v2.conf output: --- analytics/config/site-v1.conf 2026-01-15 09:00:00.000000000 +0000 +++ analytics/config/site-v2.conf 2026-01-15 09:00:00.000000000 +0000 @@ -4,4 +4,5 @@ root /home/admin/website index index.html access_log /home/admin/logs/access.log -gzip off +gzip on +client_max_body_size 8m hint: Section 13.19. Exercise 13-67 with the option -u, which prints the unified report.
Four kinds of line. The two header lines name the old file with --- and the new file with +++. Each name is followed by a tab and that file modification time; the files here carry a fixed time, so yours match.
The @@ line gives the position: four lines starting at line 4 of the old file become five lines starting at line 4 of the new one. Then the block itself: a leading space means unchanged, a minus means removed, a plus means added.
When two files are identical, diff prints nothing at all and its exit code is 0. The area holds a third file, ~/analytics/config/site.conf, which starts as a copy of version 1. Compare version 1 with it.
prompt: admin@server1:~$ answer: diff analytics/config/site-v1.conf analytics/config/site.conf ||| diff ~/analytics/config/site-v1.conf ~/analytics/config/site.conf output: hint: Section 13.19. The comparison tool with no options, ~/analytics/config/site-v1.conf, then the working copy ~/analytics/config/site.conf.
Silence means no differences. The exit code is 0 when the files are identical and 1 when they differ, so diff a b && echo same from Chapter 10 prints its word only when they match. A third code, 2, means diff could not do the comparison at all.
A unified report is a complete instruction for turning one version of a file into another, so a program can carry it out. A file holding such a report is a patch, and the program that applies it is patch.
The area holds one, ~/analytics/config/site.patch. Read it before applying it. Predict which setting changes and which new line appears; the next readback will test that prediction.
prompt: admin@server1:~$ answer: cat analytics/config/site.patch ||| cat ~/analytics/config/site.patch output: --- site.conf +++ site.conf @@ -4,4 +4,5 @@ root /home/admin/website index index.html access_log /home/admin/logs/access.log -gzip off +gzip on +client_max_body_size 8m hint: Section 5.1. The printing tool and the patch file ~/analytics/config/site.patch.
The same shape Exercise 13-68 printed, with no times in the header lines, because this file was written by hand rather than by diff. It turns compression from off to on and adds a size limit. A patch file states exactly which lines it removes and which it adds, so reading it first tells you what applying it will do.
Give patch the file to change, ~/analytics/config/site.conf, and then the patch file, ~/analytics/config/site.patch, in that order. patch changes the file in place and reports which file it changed.
prompt: admin@server1:~$ answer: patch analytics/config/site.conf analytics/config/site.patch ||| patch ~/analytics/config/site.conf ~/analytics/config/site.patch output: patching file analytics/config/site.conf hint: Section 13.20. The applying tool, the file to change ~/analytics/config/site.conf, then the patch ~/analytics/config/site.patch.
One line of report. If a patch does not fit the file it is given, patch says so, applies what it can and writes the rest to a file whose name ends .rej, for rejected. Applying the same patch twice makes patch ask whether you meant to undo it, and answering that question is not something to do by accident: run analytics/reset and start again instead.
patch named the file it changed. The file itself, ~/analytics/config/site.conf, is the proof of what it did.
prompt: admin@server1:~$ answer: cat analytics/config/site.conf ||| cat ~/analytics/config/site.conf output:
listen 80 server_name northwind.example.com root /home/admin/website index index.html access_log /home/admin/logs/access.log gzip on client_max_body_size 8m hint: Section 5.1. The printing tool and the working copy you just changed, ~/analytics/config/site.conf.
Eight lines: gzip off became gzip on and the size limit was added. Verify the whole file, not just those two lines: run diff ~/analytics/config/site-v2.conf ~/analytics/config/site.conf. No output and exit code 0 mean the working copy matches version 2; a difference or read error needs investigation.
column prints its input as a table with the columns lined up. The option -t builds the table, and -s names the character that separates the input columns. It does for a whole file what the width in a printf format does for one line. Print ~/analytics/products.csv as a table, with a comma as the input separator.
prompt: admin@server1:~$ answer: column -t -s, analytics/products.csv ||| column -s, -t analytics/products.csv ||| column -t -s, ~/analytics/products.csv output: PRODUCT BAKER PRICE COUNTER BAGUETTE okafor 3.20 front CIABATTA rivera 4.10 front FOCACCIA chen 5.50 back RYE yamamoto 4.75 back SOURDOUGH dubois 6.00 front WHOLEMEAL kowalski 3.90 back hint: Section 13.21. The table tool with the option -t that builds the table and the option -s, that names the input separator, then ~/analytics/products.csv.
The same seven lines as Exercise 13-24, readable. Every column is as wide as its widest value plus two spaces, which is what lines the values up.
column is for reading rather than for further processing. The commas are gone from the output, so cut -d, can no longer find the fields, and a value holding a space would look like two columns.
Without -s, column splits the input on whitespace, which suits the output of most Unix tools, and it suits ~/analytics/prices.txt.
prompt: admin@server1:~$ answer: column -t analytics/prices.txt ||| column -t ~/analytics/prices.txt output: BAGUETTE 3.20 CIABATTA 4.10 FOCACCIA 5.50 RYE 4.75 SOURDOUGH 6.00 WHOLEMEAL 3.90 hint: Section 13.21. The table tool with -t only, then ~/analytics/prices.txt, because whitespace is the default separator.
Two aligned columns. Put column at the end of a pipeline whenever the result is a table you are going to read: join a b | column -t is a common ending.
sed is a stream editor: it reads a file line by line, applies an editing command to each line, and writes the result to standard output. Its most used command is s, substitute, written s/old/new/.
The pattern between the first two slashes is the basic pattern language of Section 13.3, the one grep reads without -E. A plus there is an ordinary plus, and sed -E turns on the extended language the same way grep does.
The file is ~/analytics/notice.txt, a shop notice naming a product line that has been renamed.
prompt: admin@server1:~$ answer: cat analytics/notice.txt ||| cat ~/analytics/notice.txt output: NORTHWIND BAKERY NOTICE, WEEK 45 The STONEMILL loaf moves to the morning bake. All STONEMILL orders go to the wholesale desk first. File STONEMILL returns under STONEMILL, never under the old name. No STONEMILL crate leaves the yard without a signed slip. hint: Section 5.1. The printing tool and the notice ~/analytics/notice.txt.
Five lines. STONEMILL appears five times, and line four holds two of them. A line with two matches on it is where the difference between the next two commands shows.
The old text goes between the first two slashes and the new text between the last two. Substitute STONEMILL with HARVEST in ~/analytics/notice.txt.
prompt: admin@server1:~$ answer: sed 's/STONEMILL/HARVEST/' analytics/notice.txt ||| sed 's/STONEMILL/HARVEST/' ~/analytics/notice.txt output: NORTHWIND BAKERY NOTICE, WEEK 45 The HARVEST loaf moves to the morning bake. All HARVEST orders go to the wholesale desk first. File HARVEST returns under STONEMILL, never under the old name. No HARVEST crate leaves the yard without a signed slip. hint: Section 13.22. The stream editor, then the substitute command 's/STONEMILL/HARVEST/' in single quotes, then ~/analytics/notice.txt.
Look at line four: the first STONEMILL became HARVEST and the second did not. s/old/new/ replaces the FIRST match on each line and then moves to the next line. The whole file was printed, changed lines and unchanged lines alike, because sed prints every line it reads.
A letter after the last slash is a flag. The flag g, for global, makes the substitution apply to every match on the line instead of the first. Run it on ~/analytics/notice.txt again.
prompt: admin@server1:~$ answer: sed 's/STONEMILL/HARVEST/g' analytics/notice.txt ||| sed 's/STONEMILL/HARVEST/g' ~/analytics/notice.txt output: NORTHWIND BAKERY NOTICE, WEEK 45 The HARVEST loaf moves to the morning bake. All HARVEST orders go to the wholesale desk first. File HARVEST returns under HARVEST, never under the old name. No HARVEST crate leaves the yard without a signed slip. hint: Section 13.22. Exercise 13-76 with the flag g after the last slash, which replaces every match on a line.
Line four now says HARVEST twice. The FILE is still unchanged: everything you have seen so far went to standard output. The next section adds the option that writes the result back into the file, so read the printed result first: a wrong pattern written into a file cannot be undone.
The option -i, in place, writes the result back into the file instead of to standard output. There is no undo. Run the command without -i first, read what it printed, and only then add -i. The file is ~/analytics/notice.txt.
prompt: admin@server1:~$ answer: sed -i 's/STONEMILL/HARVEST/g' analytics/notice.txt ||| sed -i 's/STONEMILL/HARVEST/g' ~/analytics/notice.txt output: hint: Section 13.22. Exercise 13-77 with the option -i, which writes the result back into ~/analytics/notice.txt.
Nothing printed, which is the whole point: the result went into the file. On this machine analytics/reset puts the notice back, and on a real server the equivalent is a backup.
GNU sed can make that backup. A suffix written against the option, as in -i.bak, keeps the original as notice.txt.bak. The suffix has to touch the option: a separated word is read as the sed program instead.
sed printed nothing, so ~/analytics/notice.txt is the only place the result can be.
prompt: admin@server1:~$ answer: cat analytics/notice.txt ||| cat ~/analytics/notice.txt output: NORTHWIND BAKERY NOTICE, WEEK 45 The HARVEST loaf moves to the morning bake. All HARVEST orders go to the wholesale desk first. File HARVEST returns under HARVEST, never under the old name. No HARVEST crate leaves the yard without a signed slip. hint: Section 5.1. The printing tool and `~/analyt
Practice Chapter 13, Part 3. Text Processing: diff, patch, column, sed and awk in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.