pv: Monitoring Pipe Progress with a Progress Bar

pv: Monitoring Pipe Progress with a Progress Bar
Reading time: About 9 min Difficulty: Intermediate

What Is pv?

Conclusion: pv (Pipe Viewer) measures the data flowing through a pipe and shows a live progress bar, elapsed time, transfer rate, and ETA. It cures the "is this thing even working?" anxiety of dd and cp.

When you feed data into dd, gzip, tar, or mysql, the command often sits silent until it finishes, looking like it has frozen. Drop pv into the pipe and it reveals how many bytes have been processed and roughly how long is left.

$ pv bigfile.iso | dd of=/dev/sdb
 1.2GiB 0:00:42 [29.0MiB/s] [=========>          ] 48% ETA 0:00:45

The six fields mean:

  • 1.2GiB: bytes transferred so far
  • 0:00:42: elapsed time
  • [29.0MiB/s]: current transfer rate
  • [====>]: progress bar
  • 48%: percentage complete
  • ETA 0:00:45: estimated time to finish

What you'll learn

  • The basic pattern for putting a progress bar on a pipe
  • How to get an ETA even when the pipe size is unknown (-s)
  • Practical tricks: rate limiting, line counting, and watching a running process

How Do You Install pv?

Conclusion: pv is usually not installed by default. Use apt on Debian/Ubuntu, dnf on RHEL-family systems, and brew on macOS.

# Debian / Ubuntu
$ sudo apt install pv

# RHEL / Rocky / AlmaLinux / Fedora
$ sudo dnf install pv

# macOS (Homebrew)
$ brew install pv

Verify the install:

$ pv --version
pv 1.6.20

Many minimal servers ship without pv. If you hit command not found, install it as shown above. When you cannot add packages to a production box, the -d flag (watch a running process, covered below) is often a usable fallback.

What's the Basic Usage?

Conclusion: Pass a file as an argument and pv detects its size automatically, giving you percentage and ETA. Otherwise, insert pv into the middle of an existing pipe.

Pass a file (size auto-detected)

Given a filename, pv reads the size via stat, so it can show percentage and ETA.

$ pv access.log | grep "404" > errors.txt

Think of it as using pv in place of cat.

Insert into an existing pipe

If a pipe already exists, splice pv in wherever you want to measure.

$ gzip -dc backup.sql.gz | pv | mysql mydb

Here pv reads from standard input and cannot know the total size, so percentage and ETA are omitted (only the byte count, elapsed time, and rate are shown). To get an ETA, use -s next.

How Do You Get an ETA When the Size Is Unknown?

Conclusion: When reading from stdin with an unknown size, declare the expected size with -s. Combine it with du or the pre-compression size to produce an ETA.

-s (--size) takes the total byte count, which pv treats as 100% to compute percentage and ETA. Suffixes k/m/g are accepted.

$ pv -s 2g backup.sql.gz | gunzip | mysql mydb

When archiving a directory with tar, the common idiom is to compute the real size with du and pass it in.

$ tar -cf - mydir | pv -s "$(du -sb mydir | cut -f1)" > mydir.tar

du -sb returns the total size in bytes (-b = bytes). cut -f1 extracts just the size column to feed -s. Compression shrinks the output, so treat the ETA as an estimate when piping through gzip.

How Do You Customize the Display?

Conclusion: Pick the fields with flags: -p bar, -t timer, -e ETA, -r rate, -b byte count, -N name. With no flags, you get everything.

Specifying even one option restricts the output to only the requested fields.

Flag Shows
-p progress bar
-t elapsed time
-e ETA (time remaining)
-r current transfer rate
-a average transfer rate
-b bytes transferred
-N a name label

When you line up several pv instances, -N labels make them easy to tell apart.

$ pv -N "read" -cb source.dat | gzip | pv -N "compressed" -cb > out.gz

-n (--numeric) prints the percentage as a bare number, which is ideal as input to a shell script or dialog --gauge.

$ pv -n bigfile 2>&1 >/dev/null | while read p; do echo "progress $p%"; done

How Do You Limit the Transfer Rate?

Conclusion: -L (--rate-limit) caps the throughput. It is useful for copies where you want to avoid saturating a production disk or network.

$ pv -L 10m bigfile.iso > /mnt/backup/bigfile.iso

This limits the transfer to 10 MiB per second. Use it when a backup must not hog disk I/O or bandwidth and disrupt live services.

The -L value is bytes per second. 10m means 10 MiB/s, not 10 Mbps (megabits per second). It is easy to confuse with network bandwidth units.

How Do You Track Progress by Line Count?

Conclusion: -l (--line-mode) counts lines instead of bytes, which suits log processing and record-count progress.

$ pv -l -s 1000000 access.log | grep "POST" > posts.log

Pass the total line count to -s to display "line N of one million" as a percentage. Handy for batch jobs or imports where the line count is known up front.

How Do You Monitor Multiple Stages of a Pipe?

Conclusion: To place pv at several points in one pipe, add -c (--cursor). Without it the displays collide and break.

$ pv -cN "in" raw.dat | gzip | pv -cN "out" > raw.gz

-c manages the terminal cursor so each pv draws its progress on a separate line. Combine it with -N labels to make clear which stage each bar belongs to.

How Do You View the Progress of a Running Process?

Conclusion: Attach to an already-running cp or dd with pv -d PID. It is the rescue move for when you forgot to splice pv in from the start.

Given a PID, -d (--watchfd) shows the read/write progress across all file descriptors that process has open.

# Find the PID of a dd running in another terminal
$ pgrep -a dd
4821 dd if=/dev/sda of=backup.img bs=1M

# Peek at its progress after the fact
$ pv -d 4821

To watch a single file descriptor, use the PID:FD form.

$ pv -d 4821:1

pv -d reads the target's /proc/<PID>/fd and /proc/<PID>/fdinfo to estimate progress. It is Linux-only and the go-to recovery for a long cp, dd, or gzip you forgot to wrap.

Common Problems and Fixes

Conclusion: Most cases of a missing percentage or ETA come down to an unknown size. Supply it with -s, or pass the file as an argument directly.

No percentage or ETA

pv is reading from stdin and cannot determine the size. Pass the file directly as pv file, or give an expected size with -s.

command not found: pv

Not installed. See the install steps. If you cannot add it to production, consider external monitoring with pv -d PID.

The bar jumps straight to 100%

The data is small, or the -s value is smaller than the actual data. Check that the -s size is correct.

bash: /usr/bin/pv: Argument list too long

This is the shell's expansion limit, not pv itself. Instead of expanding many files into pv, pass a line stream like find ... | pv -l | xargs.

Summary: Next Reading

Splice pv into a single point in a pipe and a previously silent job starts telling you how many seconds remain. Writing images with dd, backing up with tar/gzip, importing into mysql — the feel of every long-running job changes. Start by replacing cat with pv.