How to Read Log Files - Introduction to System Log Analysis

How to Read Log Files - Introduction to System Log Analysis
Reading time: About 12 min Difficulty: Intermediate

What You'll Learn

  • How the /var/log/ directory is organized
  • How to use tail -f, grep, and less for efficient log reading
  • The roles of key log files: syslog, auth.log, kern.log
  • How to parse syslog format (timestamp, process name, PID, message)
  • Practical log investigation patterns for real troubleshooting

Quick Summary (Log Investigation Workflow)

  1. Run ls /var/log/ to identify the target file
  2. Use tail -n 100 to start from the end; tail -f for live monitoring
  3. Narrow down with grep using error keywords
  4. Use less to scroll through context around findings

What Is /var/log?

Nearly all Linux system logs are stored under /var/log/. This directory holds records written by the kernel, authentication subsystem, and applications — making it the starting point for any troubleshooting session.

ls /var/log/
auth.log   dpkg.log  kern.log  syslog    ubuntu-advantage.log
boot.log   faillog   lastlog   ufw.log   wtmp

Log writers fall into two categories: the syslog daemon (rsyslog or syslog-ng) and applications that write their own log files directly.

What Are the Common Log Files and Their Roles?

Files under /var/log/ are organized by purpose. Knowing which file covers which area lets you jump straight to the relevant log rather than searching blindly.

File Contents Use Case
syslog General system messages Unexplained system events
auth.log Auth, sudo, SSH Failed logins, privilege escalation
kern.log Kernel messages Hardware errors, OOM kills
dpkg.log Package management Install/update history
ufw.log UFW firewall Blocked connections
boot.log Boot-time messages Boot failure diagnosis

On Ubuntu 20.04+, syslog may not exist in some configurations. Use journalctl to view system logs in that case.

What Are the Basic Commands for Reading Logs?

Three commands cover most log-reading scenarios: tail, less, and cat. Choose based on whether you need the end of the file, interactive scrolling, or a full dump.

tail — Read from the end

# Show last 50 lines
tail -n 50 /var/log/syslog

# Follow in real time (Ctrl+C to stop)
tail -f /var/log/syslog

-f (follow) keeps the output open and streams new lines as they are appended — essential when monitoring a live incident.

less — Scroll interactively

less /var/log/auth.log

Key bindings inside less:

  • G: Jump to end
  • g: Jump to beginning
  • /keyword: Search forward
  • n: Next match
  • q: Quit

cat — Dump the entire file (for short files only)

cat /var/log/boot.log

For large logs, cat produces a wall of output. Prefer less or tail unless the file is short.

How to Filter Logs with grep

grep turns a wall of log text into targeted results. It works best piped with tail or as a direct filter on a log file.

Extract error lines

grep -i "error" /var/log/syslog

-i makes the match case-insensitive.

Match multiple keywords

grep -E "error|warn|failed" /var/log/syslog

Real-time grep

tail -f /var/log/syslog | grep "error"

Filter by date

grep "May 31" /var/log/auth.log

Show surrounding context

grep -A 5 -B 5 "Failed password" /var/log/auth.log

-A 5 shows 5 lines after each match; -B 5 shows 5 lines before.

Some log files require sudo to read. If you see Permission denied, prefix the command with sudo: sudo less /var/log/kern.log.

How to Parse the Syslog Format

Syslog-format entries follow a consistent structure. Recognizing each field speeds up analysis significantly.

May 31 10:23:45 myserver sshd[12345]: Failed password for root from 192.168.1.100 port 22 ssh2
Field Value Meaning
Timestamp May 31 10:23:45 When the event was recorded
Hostname myserver Host that generated the message
Process name sshd Process that wrote the log entry
PID [12345] Process ID (useful for correlation)
Message Failed password... The actual log content

Log timestamps reflect the server's local time. If the server is set to UTC, displayed times may differ from your local time zone.

Common Log Investigation Patterns

These patterns cover the most frequent troubleshooting scenarios encountered in production environments.

Check for SSH login failures

sudo grep "Failed password" /var/log/auth.log | tail -20

Review sudo command history

sudo grep "sudo" /var/log/auth.log | grep "COMMAND"

Check for kernel errors

sudo grep -i "error\|oops\|panic" /var/log/kern.log | tail -30

List package installation history

grep " install " /var/log/dpkg.log

Review boot-time errors

sudo less /var/log/boot.log

journalctl vs. Log Files — When to Use Which?

Since Ubuntu 16.04, systemd introduced journalctl as the primary log viewer for services. /var/log/ files and the systemd journal coexist; knowing which to use prevents wasted time.

Scenario Tool to Use
systemd service logs journalctl -u nginx
Kernel messages journalctl -k
Boot-time messages journalctl -b
Legacy app writing to its own file tail -f /var/log/app.log
/var/log/auth.log contents Either works 
# Follow all logs in real time
journalctl -f

# Follow a specific service
journalctl -u nginx -f

journalctl reads from the systemd journal binary store, not from /var/log/ text files. Depending on rsyslog configuration, both may receive the same events — or they may contain different subsets.

Next Reading