Learn › Linux Foundations › Text Processing

sed

sed - a hands-on Linux lab on a real virtual machine.

Transform text line by line with sed, the stream editor. Substitute with s/OLD/NEW/, replace every match with the g flag, print one line with -n and p, delete with d, use a range, swap the delimiter, and edit in place with -i (no undo). All in the practice terminal on a tiny three-line poem.

So far in this module you have READ text and you have COUNTED it. Now you change it. You want to swap one word for another across a whole file, or drop a line, or pull out just line 2, without ever opening an editor and without touching the original by accident.

That job belongs to one tool: sed, the stream editor. The name is the idea. Text flows through sed like water through a filter, one line at a time, and sed edits each line as it passes, then prints the result. Point it at a file, hand it a rule, and the changed text comes out the other side. That is the whole command, and this lesson is all of it.

The black boxes in this lesson are a practice terminal: a safe sandbox that only checks the one command each step teaches, so you cannot break anything. At the end of the module one real Linux machine boots just for you, and you run the full text-processing mission there.

There is one small file in front of you, poem.txt. It holds exactly three lines:

roses are red
violets are blue
sugar is sweet

Before sed changes anything, look at it whole. The classic sed job is substitution: find a word, print a different one. Ask sed to swap the word red for crimson. Type this exact command and press Enter:

sed "s/red/crimson/" poem.txt

prompt: student@linuxcamp:~$ answer: sed "s/red/crimson/" poem.txt output: roses are crimson violets are blue sugar is sweet hint: The substitute rule is s slash old slash new slash: sed "s/red/crimson/" poem.txt

Line 1 came out as roses are crimson; the other two lines passed through untouched. That is sed in its most common form. The next step breaks that little rule apart, because every piece of s/red/crimson/ has a job.

The rule you just typed was s/red/crimson/. It has four parts, and the slashes are the fences between them:

So s/OLD/NEW/ reads as "substitute NEW wherever you find OLD." sed applied that rule to every line as it flowed through, but only line 1 held the word red, so only line 1 changed. The whole file still printed, changed line and all.

Read s/OLD/NEW/ out loud as "swap OLD for NEW." The s is the verb, the first slash starts what you are looking for, the second slash starts what you are replacing it with, and the third slash closes the rule.

Here is the surprise that catches every new sed user, and it is better to meet it in the sandbox. By default, s/OLD/NEW/ replaces only the first match on each line, not every match. If a word appears twice on one line, the second copy is left alone.

To see it clearly you need a line with a repeat. Imagine a one-line scratch file whose only line is a a a, three letter-a's separated by spaces. Ask sed to swap a for X:

echo "a a a" | sed "s/a/X/"

prompt: student@linuxcamp:~$ answer: echo "a a a" | sed "s/a/X/" output: X a a hint: Send the line into sed with a pipe, then substitute a for X: echo "a a a" | sed "s/a/X/"

Only the first a became X; the output is X a a, not X X X. sed walked the line, found the first a, replaced it, and then moved on to the NEXT line without looking for more matches on this one. That is the default, and it is the number-one reason a sed replacement "does not work": it worked, it just stopped after the first hit. The next step fixes that.

To replace every match on a line, not just the first, you add one letter after the closing slash: g, short for global. The rule becomes s/OLD/NEW/g. Everything before is identical; the trailing g tells sed to keep going along the line and replace every hit.

Run the very same line through with g added and watch the difference:

echo "a a a" | sed "s/a/X/g"

prompt: student@linuxcamp:~$ answer: echo "a a a" | sed "s/a/X/g" output: X X X hint: Same rule as before, but add g after the last slash: echo "a a a" | sed "s/a/X/g"

Now all three came out as X X X. The only change from the last step was one letter, g on the end, and it turned "first match" into "every match." That is the single most useful flag on sed: s/OLD/NEW/ for the first hit on each line, s/OLD/NEW/g for all of them. Burn that contrast in. When a replacement misses copies, you almost always forgot the g.

sed is not only about swapping words. It can also pick out lines. By default sed prints every line it sees, which is why substitution shows the whole file. Two pieces let you print just ONE line instead.

First, -n (for no auto-print) silences that default, so sed prints nothing on its own. Then the p command (for print) prints only the line you point it at. Put a line number in front of p and you get just that line. Ask for line 2 of the poem:

sed -n "2p" poem.txt

prompt: student@linuxcamp:~$ answer: sed -n "2p" poem.txt output: violets are blue hint: The -n flag hushes the default, then 2p prints line 2: sed -n "2p" poem.txt

One line, violets are blue, and nothing else. Two parts did that together: -n turned off the automatic printing of every line, and 2p said "print line 2." Leave -n off and you would see the whole file with line 2 printed a SECOND time, because the default print and the p command would both fire. -n and p are a pair; they travel together.

The mirror image of printing one line is deleting one. The d command (for delete) removes a line from the stream, so it never prints. Put a line number in front, exactly like p, and that line is dropped while the rest flow through unchanged. No -n here; you WANT the other lines to print.

Delete line 2 from the poem and see what is left:

sed "2d" poem.txt

prompt: student@linuxcamp:~$ answer: sed "2d" poem.txt output: roses are red sugar is sweet hint: A line number then d deletes that line: sed "2d" poem.txt

The violets are blue line is gone; lines 1 and 3 came through as roses are red and sugar is sweet. 2d told sed to delete line 2 as it passed and print everything else. Notice the pattern forming: a number in front of a command is an address, and it aims the command at one line. 2p prints line 2, 2d deletes line 2. You can also give a RANGE, like 2,4d, to delete lines 2 through 4 in one go. Same idea, two line numbers instead of one.

Every command so far printed the changed text to the screen and left poem.txt on disk exactly as it was. That is the safe default: sed reads the file and writes to your terminal, never back to the file. Often that is what you want, so you can eyeball the result first.

When you are sure, the -i flag (for in place) writes the changes back INTO the file instead of printing them. It prints nothing, because the result went to disk. This is the flag to respect. There is no undo. Once you asked to swap red for crimson in place, poem.txt now says crimson and the old red is gone for good.

Run the in-place edit. Notice it prints nothing at all:

sed -i "s/red/crimson/" poem.txt

prompt: student@linuxcamp:~$ answer: sed -i "s/red/crimson/" poem.txt output: hint: Add -i before the rule to write changes into the file: sed -i "s/red/crimson/" poem.txt

Empty output is not nothing happening; it is the OPPOSITE. With -i, the change went straight into poem.txt, so there is nothing to print. Because there is no undo, the safe habit is to run the command WITHOUT -i first, read the output, and only add -i once you trust it. If you want a safety net, -i.bak edits in place AND saves the original as poem.txt.bak, so you can recover it.

The slash in s/OLD/NEW/ is just a separator, and sed lets you swap it for another character. This matters the day your OLD or NEW text contains a slash of its own, like a file path. Writing s/\/home/\/root/ is a thicket of backslashes; picking a different fence is cleaner. Any character right after the s becomes the separator, so s#OLD#NEW# uses a hash instead of a slash.

You do not need a slash in the text to try the form. Swap blue for green in the poem using # as the fence:

sed "s#blue#green#" poem.txt

prompt: student@linuxcamp:~$ answer: sed "s#blue#green#" poem.txt output: roses are red violets are green sugar is sweet hint: Use the hash as the fence right after s: sed "s#blue#green#" poem.txt

Line 2 became violets are green, and the rest passed through. The # did exactly the job the / did; it just marked the three slots instead. sed decides the separator from the FIRST character after s, so s#old#new# and s|old|new| and s/old/new/ are all the same rule with different fences. Reach for a non-slash fence whenever your text is full of slashes, and your rule stops looking like a picket fence.

Scaffolding off. No command is shown this time. You have every piece you need.

Send the line a a a into sed and turn EVERY a into X, so the output reads X X X. Remember the trap: a plain substitute stops after the first match. You need the one letter that makes it replace every match on the line.

prompt: student@linuxcamp:~$ answer: echo "a a a" | sed "s/a/X/g" output: X X X hint: Every match means the global flag. Add g after the closing slash: echo "a a a" | sed "s/a/X/g"

All three became X, from memory. The whole difference between X a a and X X X was the single g, and you recalled it without a prompt. That is the skill this lesson is really about: knowing that substitute defaults to the first match, and reaching for g when you mean all of them. It is exactly the move the real machine will ask of you.

You earned this cheat sheet. Every row is a form of sed you have already run:

The one rule to carry away: substitute changes only the first match per line unless you add g. And -i writes to the file with no undo, so run it once without -i first to check your work.

sed shines when it is fed a stream, not just a file. Later in the camp you will pipe the output of another command into sed to clean it up on the fly, the same s/OLD/NEW/g you learned here. Sibling tools in this module, grep to find lines, cut to slice columns, and awk for fields, each get their own lesson and pair naturally with sed.

The practice terminal has shown you every form of sed: substitution with s/OLD/NEW/, the g flag for every match, -n with p to print one line, d to delete one, a range like 2,4, an alternate delimiter, and -i to edit in place. Every one of those you typed yourself.

The Text Processing module ends with one real Linux machine: the text processing capstone mission. There a VM boots just for you, and the mission hands you objectives with no commands shown. You read the objective, recall the rule, and type it. Alongside sed you will use the module's other text tools, grep, cut, sort, and the rest, each of which has its own lesson. One difference from here: the mission shows no commands. That recall is what makes it stick.

Finish the other Text Processing lessons, then go edit some streams for real.

Practice sed in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.