How to Use find, grep, and awk - Linux Text Search Tutorial

2025-09-16 Reading time: About 20 min Difficulty: Intermediate+

Once you're comfortable with basic Linux commands, the next three to learn are find, grep, and awk. With these three, you can handle file hunting, log investigation, and aggregation in a single line.

This fundamentals guide covers three things: the role of each command and how to choose between them, the regex basics that work across all three, and the search features of find.

What You'll Learn

• You will be able to choose between find, grep, and awk based on your goal.
• You will be able to read and write regex anchors, quantifiers, and character classes.
• You will be able to combine find conditions and actions to run batch jobs safely.

Who this is for: anyone who has used basic commands such as ls, cd, and cat at least once.

Prerequisites: find and grep need sudo when the target files belong to another user. Some examples use pipes (|) and redirection (>). If those are new to you, read Pipe and Redirect Basics first.

Terms defined up front

Each term below is defined once, here.

• A regular expression is a way to describe "strings shaped like this" as a pattern. It is also called a regex or regexp. This article uses "regex" throughout.
• A metacharacter is a symbol such as ^, $, ., or * that carries a special meaning instead of standing for itself. It is also called a special character.
• An anchor is a metacharacter that points at a position — start of line, end of line, or word boundary. It never matches a character itself.
• A quantifier specifies how many times the preceding pattern repeats. It is also called a repetition operator.
• A character class is a form such as [0-9] meaning "any single character in this range".
• Escaping means putting \ in front of a metacharacter so it is treated as the literal symbol.
• Standard error (stderr) is the outlet that error messages flow to. 2>/dev/null discards that outlet.

Overview and Command Selection

Conclusion: find locates files, grep searches contents, awk processes data.

First, understand the characteristics and use cases of each command. Choosing the right command is the first step toward efficient work.

find: File and Directory Search

• Search files by name
• Filter by size or date
• Search by permissions or owner
• Batch processing on found files

Specialty: "Finding files when you don't know where they are".

find /home -name "*.txt" -size +1M

grep: Text Content Search

• Search text within files
• Advanced search using regex
• Log file analysis
• Configuration file inspection

Specialty: "Finding specific text inside files".

grep -r "ERROR" /var/log/

-r means recursive: it walks down through the given directory. Most files under /var/log/ are owned by root, so use sudo grep -r "ERROR" /var/log/ when they cannot be read.

awk: Text Processing and Data Manipulation

• Extract and calculate column data
• Process CSV files
• Aggregate log files
• Format conversion

Specialty: "Processing, aggregating, and transforming data".

awk '{sum+=$3} END {print sum}' sales.csv

awk reads input one line at a time and treats each whitespace-separated item as a field (column). $1 is the first column, $3 the third, and NF the number of columns. The example above sums the third column and prints the result once every line has been read (END).

Decision Flow

Regular Expression Masterclass

Conclusion: Master anchors, quantifiers and classes across BRE, ERE and PCRE.

Regular expressions are essential for unlocking the true power of find, grep, and awk. Master patterns from basics to ones immediately usable in production work.

Types of Regular Expressions

Regex comes in three dialects. The same pattern can be interpreted differently depending on the tool, so start with this distinction.

grep uses BRE unless told otherwise. Add -E to switch to ERE when you want to write + or ? directly.

Position Anchors

# Lines starting with ERROR
grep "^ERROR" logfile.txt

# Lines ending with .log
grep "\.log$" filelist.txt

# The word "port" (excludes "report" etc.)
grep -E "\bport\b" config.txt

Character Classes

# 192.168.1.x IP addresses
grep "192\.168\.1\." access.log

# Time format (HH:MM)
grep "[0-9][0-9]:[0-9][0-9]" log.txt

# Lines containing non-alphanumeric characters
grep "[^a-zA-Z0-9]" data.txt

Quantifiers

# Zero or more (error and failed on the same line)
grep "error.*failed" log.txt

# One or more (ERE)
grep -E "[0-9]+" data.txt

# Zero or one (http or https)
grep -E "https?" urls.txt

# Between n and m occurrences (2-4 digit numbers)
grep -E "[0-9]{2,4}" data.txt

Advanced Pattern Matching

Grouping and OR:

# Multiple keywords with OR
grep -E "(error|warning|critical)" log.txt

Lookahead and Lookbehind (PCRE):

# Numbers before "yen"
grep -P "\d+(?=yen)" price.txt

# "test" not followed by ".txt"
grep -P "test(?!\.txt)" filelist.txt

# Numbers after $ sign
grep -P "(?<=\$)\d+" invoice.txt

Practical Regex Patterns

Log Analysis:

# IPv4 addresses
grep -E "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" access.log

# Apache date format
grep -E "\[[0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{2} [+-][0-9]{4}\]" access.log

# HTTP status code aggregation
grep -E "\" [1-5][0-9]{2} " access.log | awk '{print $9}' | sort | uniq -c

# Log levels
grep -E "\b(DEBUG|INFO|WARN|ERROR|FATAL|CRITICAL)\b" app.log

The status aggregation uses $9 because in Apache combined format the status is the ninth field. The User-Agent gets split on spaces, so the column count varies per line and a position counted from the end, such as $(NF-1), cannot be used.

Data Validation:

# Email addresses (simple)
grep -E "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b" contacts.txt

# URLs (http/https)
grep -E "https?://[^[:space:]\"']+" webdata.txt

The URL example uses [^[:space:]...] rather than [^\s...] because inside a bracket expression ([ ]), \s is not read as the whitespace class. [^\s"'] means "anything except backslash, s, ", and '", so it stops as soon as the URL contains an s. Inside brackets, use the POSIX class [:space:].

Code Analysis:

# Function definitions (JavaScript/Python)
grep -E "^(function|def)\s+[a-zA-Z_][a-zA-Z0-9_]*\s*\(" *.js *.py

# Variable declarations (JavaScript)
grep -E "^(var|let|const)\s+[a-zA-Z_][a-zA-Z0-9_]*" *.js

# TODO/FIXME comments
grep -E "(TODO|FIXME|XXX|HACK|NOTE):" -n *.py

Regex Debugging

Build complex regexes incrementally.

# Step 1: Lines with digits
grep "[0-9]" test.txt

# Step 2: One or more digits
grep "[0-9]\+" test.txt

# Step 3: Only digits
grep "^[0-9]\+$" test.txt

Use -o to verify partial matches:

echo "test123abc456" | grep -o "[0-9]\+"

123
456

Performance Optimization

find Command: Mastering File Search

Conclusion: find filters by name, size or date, then runs actions on matches.

find is a powerful command that lets you search the filesystem in any way you need.

Basic Syntax

find [search_path] [conditions] [actions]

It locates files matching the conditions in the search path, then runs actions.

Search by Name

# Files with .txt extension
find /home -name "*.txt"

# Files starting with "config"
find . -name "config*"

# Case-insensitive .log search
find /var -iname "*.LOG"

Search by File Type

-type narrows the search to one kind of entry: f for a regular file, d for a directory, l for a symbolic link (a "shortcut" that points at another file — symbolic link, symlink, and soft link all mean the same thing).

# Regular files only
find /home -type f

# Directories starting with "log"
find /var -type d -name "log*"

# Symbolic links
find /tmp -type l

Search by Size

-size has one catch: it rounds up to the unit you specify before comparing. -size -1k therefore means "fewer than one 1KB unit", which matches only 0-byte files. A 500-byte file rounds up to 1KB and does not match.

# Larger than 100MB
find /var -size +100M

# 0-byte files (they round up to zero units)
find /home -size -1k

# Strictly smaller than 1024 bytes (c = bytes)
find /home -size -1024c

# Larger than 1GB and smaller than 10GB (compared after rounding up)
find . -size +1G -size -10G

To filter on an exact byte boundary, use c (bytes) instead of k, M, or G. No rounding happens, so the boundary is the one you intended.

Search by Date and Time

There are three time conditions. mtime (modification time) is when the contents changed, atime (access time) is when the file was read, and ctime (change time) is when attributes such as permissions or owner changed. The unit is days: -7 means "within 7 days" and +30 means "more than 30 days ago".

# Modified within the last 7 days (mtime)
find /home -mtime -7

# Modified more than 30 days ago
find /var/log -mtime +30

# Not accessed for more than 1 day (atime)
find /tmp -atime +1

# Newer than reference.txt
find /home -newer reference.txt

Search by Permission and Owner

-perm filters by permission. A setuid bit makes a program run with the file owner's privileges instead of the caller's, which is why it can become a stepping stone for privilege escalation and shows up in security audits.

# Files with permission 755
find /home -perm 755

# Files with setuid bit (security check)
find / -perm -4000 2>/dev/null

# Files owned by www-data
find /var -user www-data

# Files in the developers group
find /home -group developers

Execute Actions

The true power of find is being able to automatically run actions on found files. This is also the part where accidents happen most easily.

# Step 1: only list the targets (changes nothing)
find /tmp -name "*.tmp" -print

# Step 2: delete once the list looks right
find /tmp -name "*.tmp" -delete

Delete files:

# Delete temporary files in bulk
find /tmp -name "*.tmp" -delete

# Delete log files older than 30 days
find /var/log -name "*.log" -mtime +30 -delete

-delete is handled by find itself, which is both faster and safer than -exec rm {} \;. It also handles names containing spaces or newlines correctly. Note that -delete implicitly enables -depth (process deeper entries first), so it cannot be combined with -prune.

Change permissions:

# Set PHP files to 644
find /var/www -name "*.php" -exec chmod 644 {} \;

# Set directories to 755
find /home -type d -exec chmod 755 {} \;

{} is a placeholder replaced by each matching filename, and \; marks "run once per file". Replacing \; with + passes several files at once, which is faster.

# Before: save the numeric permissions to a file
find /var/www -name "*.php" -exec stat -c '%a %n' {} + > ~/perm-backup.txt

# To roll back: write the saved values back, line by line
while read -r mode path; do chmod "$mode" "$path"; done < ~/perm-backup.txt

Gather information:

# Show details of .txt files
find /home -name "*.txt" -exec ls -lh {} \;

# Show sizes of files larger than 100MB
find /var -size +100M -exec du -h {} \;

Best Practices

find / -name "*.txt" 2>/dev/null

# Logs larger than 1MB modified in the last 7 days
find /home -name "*.log" -size +1M -mtime -7

Troubleshooting

Conclusion: Almost every problem is one of four things — missing permissions, missing quotes, the wrong regex dialect, or too broad a search scope.

Symptom: floods of Permission denied

Cause: the search walks directories you have no read permission for.

Check:

find /var -name "*.log"

Fix: discard the errors, or borrow privileges with sudo.

find /var -name "*.log" 2>/dev/null   # discard errors
sudo find /var -name "*.log"          # read with elevated rights

Symptom: find . -name *.txt returns something unexpected

Cause: without quotes, the shell expands *.txt before find ever sees it.

Check:

find . -name *.txt

Fix: always quote the search pattern.

find . -name "*.txt"

Symptom: grep "[0-9]+" matches nothing

Cause: grep defaults to BRE, where + is read as a literal plus sign.

Check:

echo "abc123" | grep "[0-9]+"

Fix: switch to ERE with -E, or escape the +.

echo "abc123" | grep -E "[0-9]+"    # ERE
echo "abc123" | grep "[0-9]\+"      # BRE with escaping

Symptom: grep never returns to the prompt

Cause: the filename was left out, so grep is waiting for input on standard input.

Check: nothing is printed and the terminal is accepting keystrokes.

Fix: interrupt with Ctrl+C and re-run with a filename. If the input should come from a pipe, supply the producing command.

grep -E "ERROR" app.log          # name the file
tail -100 app.log | grep "ERROR" # feed it through a pipe

Symptom: find never comes back

Cause: the search scope is too broad (all of /, for example).

Check: interrupt with Ctrl+C and review the path you passed.

Fix: narrow the target directory. Limiting depth with -maxdepth also helps.

find /home/user -maxdepth 3 -name "*.log"

Completion Checklist

• [ ] Picked the command from the goal (locate a file / search contents / transform data)
• [ ] Stayed aware of whether the regex runs as BRE or ERE
• [ ] Confirmed the targets with -print before -delete or -exec
• [ ] Narrowed the search to a concrete directory instead of /