sed Basics - Text Substitution with the Stream Editor

2026-05-20 Reading time: About 12 min Difficulty: Intermediate

What You'll Learn

• The five core sed patterns (substitute / delete / print / address / multi-expression) on the shortest path to fluency
• The safe pattern for -i (in-place) edits so you stop trashing files
• The difference between BRE and ERE (-E) so your regex stays consistent with grep -E and awk

What Is sed?

sed (stream editor) reads input line by line, applies an editing script, and writes to stdout. Unlike interactive editors like Vim, sed is built for non-interactive use inside shell pipelines — log normalization, config rewrites, text preprocessing.

$ echo "hello world" | sed 's/world/Linux/'
hello Linux

The s/old/new/ (substitute) command makes up roughly 80% of real-world sed usage.

Why Preview to Stdout First?

sed -i overwrites the file in place. If you realize after the fact that the pattern was wrong or you forgot to escape a /, recovery is hard. Always preview to stdout, then commit with -i.bak — this is the only reliable habit.

# 1. Preview to stdout
$ sed 's/foo/bar/g' config.txt

# 2. Commit with backup
$ sed -i.bak 's/foo/bar/g' config.txt

# 3. config.txt.bak is generated automatically
$ ls config.txt*
config.txt  config.txt.bak

1. Substitution: s/old/new/

1-1. First match on each line (default)

$ echo "apple apple apple" | sed 's/apple/orange/'
orange apple apple

1-2. All matches on the line (g flag)

$ echo "apple apple apple" | sed 's/apple/orange/g'
orange orange orange

1-3. Case-insensitive (I flag)

$ echo "Apple APPLE apple" | sed 's/apple/orange/gI'
orange orange orange

The I flag is a GNU sed extension. Avoid it in portable scripts (POSIX sed doesn't support it).

2. Changing the Delimiter from /

When the pattern contains paths like /usr/local, escaping every / quickly becomes unreadable. The character right after s is the delimiter — pick |, #, or , to keep things legible.

# Bad: backslash hell
$ sed 's/\/usr\/local\/bin/\/opt\/bin/' file

# Good: pipe as delimiter
$ sed 's|/usr/local/bin|/opt/bin|' file

# Also good: hash as delimiter
$ sed 's#/usr/local/bin#/opt/bin#' file

3. Addresses: Targeting Specific Lines

Every sed editing command can take an address prefix that restricts which lines it applies to.

3-1. By line number

$ sed '3s/foo/bar/' file       # Only line 3
$ sed '1,5s/foo/bar/g' file    # Lines 1–5
$ sed '10,$s/foo/bar/g' file   # Line 10 to EOF ($ = last line)

3-2. By pattern

# Only on lines containing "ERROR"
$ sed '/ERROR/s/foo/bar/g' file

# Between /START/ and /END/
$ sed '/START/,/END/s/foo/bar/g' file

3-3. Negation (!)

# All lines except line 1
$ sed '1!s/foo/bar/g' file

# Everything except comment lines (starting with #)
$ sed '/^#/!s/foo/bar/g' file

4. Delete (d) and Print (p)

4-1. Delete lines

$ sed '/^$/d' file              # Delete empty lines
$ sed '/^#/d' file              # Delete comment lines
$ sed '1,5d' file               # Delete lines 1–5
$ sed '$d' file                 # Delete the last line

4-2. Print only specific lines (-n + p)

By default sed prints every line. The -n flag suppresses default output, so you only see lines you explicitly print with p.

# Print lines containing "ERROR" (like grep "ERROR")
$ sed -n '/ERROR/p' file

# Print lines 10–20
$ sed -n '10,20p' file

5. Multiple Edits in One Pass

5-1. Multiple -e expressions

$ sed -e 's/foo/bar/g' -e 's/hoge/fuga/g' file

5-2. Semicolon-separated (GNU sed)

$ sed 's/foo/bar/g; s/hoge/fuga/g' file

5-3. Script file with -f

Save reusable transformation sets to a file.

$ cat replace.sed
s/foo/bar/g
s/hoge/fuga/g
/^#/d

$ sed -f replace.sed file

6. BRE vs ERE: Should You Always Use -E?

sed defaults to BRE (Basic Regular Expressions). Using +, ?, |, or (...) requires backslash escaping, which gets ugly fast. Adding -E switches to ERE (Extended Regular Expressions) — the same flavor grep -E and awk use.

6-1. BRE (default)

# Groups and quantifiers require \( \) and \{ \}
$ echo "2026-05-20" | sed 's/\([0-9]\{4\}\)-\([0-9]\{2\}\)-\([0-9]\{2\}\)/\3\/\2\/\1/'
20/05/2026

6-2. ERE (-E)

# No escaping — much easier to read
$ echo "2026-05-20" | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/'
20/05/2026

6-3. Backreferences (\1, \2, ...)

Captured groups are referenced as \1, \2, etc. in the replacement. Same syntax in BRE and ERE.

# "Last, First" -> "First Last"
$ echo "Doe, John" | sed -E 's/(.+), (.+)/\2 \1/'
John Doe

7. Pitfalls and How to Avoid Them

7-1. Escaping literal metacharacters

# Bad: . matches any character
$ echo "192x168x0x1" | sed 's/192.168.0.1/X/'
X

# Good: escape the dot
$ echo "192x168x0x1" | sed 's/192\.168\.0\.1/X/'
192x168x0x1

7-2. Embedding shell variables

Shell variables don't expand inside single quotes. Switch to double quotes — but watch out for $ and \ inside.

NEW="bar"

# Bad: $NEW is literal
$ sed 's/foo/$NEW/g' file

# Good: double quotes for expansion
$ sed "s/foo/$NEW/g" file

8. Practical Recipes

# Convert CRLF to LF
sed -i.bak 's/\r$//' file

# Strip empty lines and # comments
sed -e '/^$/d' -e '/^#/d' /etc/nginx/nginx.conf

# Mask IP addresses
sed -E 's/\b([0-9]{1,3}\.){3}[0-9]{1,3}\b/x.x.x.x/g' access.log

# Trim trailing whitespace
sed -i.bak 's/[[:space:]]*$//' file

# Insert a line before lines matching a pattern
sed -i.bak '/^pattern/i\
inserted line before' file

# Rewrite a YAML value (after key:)
sed -i.bak -E 's/^(version:\s*).*/\1"1.2.3"/' config.yaml

sed vs awk vs grep: When to Use Which