curl and wget Basics: HTTP Communication from the Command Line

curl and wget Basics: HTTP Communication from the Command Line
Reading time: About 12 min Difficulty: Beginner

What you'll be able to do

  • Pick `curl` or `wget` based on what you actually need to do
  • Check where a download lands and whether it will overwrite something
  • Follow redirects, inspect headers, and POST JSON from copy-paste examples

Prerequisites (read these first)

What You'll Learn

  • The role difference between curl and wget
  • Where a downloaded file lands before you download it
  • How to avoid overwriting a file you still need
  • How to inspect headers and call APIs from copy-paste commands

Words used here (sorted out first)

  • Download: copying data from a machine on the network onto your own machine.
  • Redirect: a note from the server that says "this page moved elsewhere." This is not the shellAn interactive program that reads the commands you type and runs them.'s > redirection. One word, two meanings.
  • Header: a label sent alongside the body. It states the type and the size of the content.
  • Status code: a three-digit number for the result. 200 means success, 404 means not found.
  • API: a doorway that programs use to talk to each other. The reply is usually JSON, not HTML.

Quick Summary

  • One-shot file download → wget URL
  • Hit an API, inspect headers, or POST → curl URL
  • Want to follow redirects → curl -L URL
  • Download was interrupted → wget -c URL

Environment

  • OS: Ubuntu / typical Linux
  • curl is usually preinstalled. Install wget via sudo apt install wget
  • We focus on HTTP/HTTPS (curl also supports ftp, sftp, smtp, etc.)

1. curl vs wget: Different Roles

Conclusion: wget is a downloader that saves by default; curl is an HTTP client that prints.

Lina: Senpai, both curl and wget "fetch data from a URL," right? I always hesitate over which one to use.
Linny-senpai: Great question. They have different design goals. In one line: wget is a downloader, curl is a multi-purpose HTTP client.
Lina: Downloader versus client?
Linny-senpai: A downloader is a tool dedicated to fetching files. A client is the side that talks to a server.
Lina: I see. So curl covers more ground.
Linny-senpai: Right. wget's default is "given a URL, save the file." curl's default is "given a URL, print the response." That single difference drives every other choice.

At a glance

Aspect curl wget
Default behavior Prints to stdout Saves as a file
HTTP methods GET/POST/PUT/DELETE/... Mostly GET
Follow redirects Need -L flag Automatic
Recursive download Weak Strong (-r)
Resume -C - -c
Best for APIs, debugging, POST Mirroring sites, large DL

Rule of thumb: APIs, headers, or POST involved → curl. Just grabbing a file → wget.

2. curl Basics: Print Before You Save

Conclusion: curl prints to the screen by default; add -o name or -O to save to a file.

2-1. Hit a URL and Print the Response

$ curl https://example.com
<!doctype html>
<html>
<head>
    <title>Example Domain</title>
...
Lina: Whoa, the entire HTML just flooded my screen.
Linny-senpai: That's curl's default behavior — dump everything to stdout. To save to a file, add -o (specify name) or -O (use the URL's filename).

2-2. Save With a Custom Name: -o

$ curl -o page.html https://example.com

-o stands for output. Lowercase o takes the name you want as its argument.

2-3. Save Using the URL's Filename: -O

$ curl -O https://example.com/sample.tar.gz

Uppercase O reuses the filename at the end of the URL. Here that is sample.tar.gz.

Mixing up -o and -O ends in tears

  • -O requires the URL to end with a real filename
  • curl -O https://example.com/ (trailing slash) fails — there is no filename to use
  • When in doubt, use -o name and be explicit

2-4. Where Does It Land? Will It Overwrite?

Check two things before any download: where the file is created, and what happens if that name already exists.

Save location and overwriting, up front

  • The file lands in the directoryA container that organizes files. Same idea as a "folder" on Windows or macOS. you are in right now. Check it with pwd
  • curl -O and curl -o overwrite a file of the same name without asking
  • Unlike a GUI file manager, there is no confirmation dialog. The old content is gone
  • To block overwriting outright, use curl --no-clobber -O URL. If the name is taken, curl does nothing
  • wget does not overwrite by default. It saves sample.tar.gz.1 with a number appended instead
  • curl and wget do not run in this site's virtual terminal. Run them in your own terminal
  • Practice inside the empty directory you create in "10. Mini Exercises". Nothing you already had can break there
Lina: Senpai, I downloaded a config file with curl -O and edited it. Then I ran the same command again. My edits were gone.
Linny-senpai: That is an overwrite. curl never asks "a file with this name exists, is that fine?" It just replaces it.
Lina: No confirmation at all? And nothing goes to a trash folder...
Linny-senpai: Nothing. So run ls before you download to see whether the name is taken. If you want a different name, choose it yourself with -o. Those two habits prevent almost every accident.
Lina: I see. "Run ls before downloading" would have saved me.
# Before downloading, check where you are and what is already there
$ pwd
$ ls sample.tar.gz

# When overwriting worries you, pick the name yourself
$ curl -o sample-20260605.tar.gz https://example.com/sample.tar.gz

2-5. Show Download Progress

$ curl -O --progress-bar https://example.com/big.iso

--progress-bar gives a clean progress bar. Long transfers stay readable.

3. The #1 Beginner Trap: Redirects and -L

Conclusion: curl does not follow redirects by default; add -L to reach the final page.

Lina: Senpai, I ran curl on https://github.com/torvalds/linux and the response looks wrong.
Linny-senpai: Classic. GitHub URLs and short URLs often involve an HTTP redirect.
Lina: A redirect — the "this page moved" note?
Linny-senpai: Right. curl does not chase that note by default. So you get the note itself, not the page it points to. Add -L and curl follows it.

3-1. Follow Redirects With -L

# BAD: only the redirect placeholder is returned
$ curl https://github.com/torvalds/linux

# GOOD: fetch the final destination
$ curl -L https://github.com/torvalds/linux

-L stands for Location. Location is the HTTP header that carries the new address.

Memorize the pattern

  • When hitting external sites with curl, always add -L to avoid surprises
  • File downloads often require -L too (GitHub Releases, S3 redirects, etc.)
# Standard safe pattern
$ curl -LO https://github.com/some/repo/releases/download/v1.0/binary.tar.gz

-LO is -L and -O joined together. Single-letter options can be combined like this. Writing -L -O means the same thing.

Conclusion: -I fetches headers only without the body; the status code reveals the state.

Want to check "is this link alive?" or "where does this redirect to?" without downloading the body? Use -I.

$ curl -I https://example.com
HTTP/2 200
content-type: text/html; charset=UTF-8
content-length: 1256
date: Sun, 26 May 2026 09:00:00 GMT
server: ECS
Lina: So HTTP/2 200 means "success," right?
Linny-senpai: Exactly. The number is the HTTP status code. Memorize these and you cover 90% of real-world cases.

HTTP status cheat sheet

Code Meaning Examples
2xx Success 200 OK, 204 No Content
3xx Redirect 301 (permanent), 302 (temp)
4xx Client-side error 404 (not found), 401/403
5xx Server-side error 500 (bug), 503 (overload)

4-1. Trace Where a Redirect Leads

$ curl -ILs https://bit.ly/3xxxxx | grep -i location

-ILs is -I, -L, and -s combined into one. -L follows redirects, -s silences progress and errors, -I shows headers only. Each hop adds its own location: line, and the last one is the final destination.

5. POST and JSON: Talking to APIs

Conclusion: Hit APIs with three flags: -X for method, -H for headers, -d for the body.

Lina: I want to call a REST API with curl. How do I send JSON?
Linny-senpai: API testing is where curl shines. Use -X for the HTTP method, -H for headers, and -d for the body. Remember those three flags and you're done.

5-1. GET With Query Parameters

$ curl "https://api.example.com/users?id=42"

Always quoteWrapping a string in quote marks (' or ") so it is treated as one single value, e.g. one with spaces in it. URLs containing ? and &. Without quotes the shell reads & as background-execution.

5-2. POST JSON

$ curl -X POST https://api.example.com/users \
    -H "Content-Type: application/json" \
    -d '{"name":"lina","role":"beginner"}'

The three-flag combo

  • -X POST: the HTTP method. -d already makes curl send a POST, so you usually do not need to write it
  • -H "Content-Type: application/json": declare "this is JSON"
  • -d '...': the request body. Use single quotes so the inner " need no escaping

5-3. Bearer Token Authentication

$ curl https://api.example.com/me \
    -H "Authorization: Bearer YOUR_TOKEN_HERE"

Keep tokens out of your shell history

# BAD: visible in `history` and `ps`
$ curl -H "Authorization: Bearer abc123..." ...

# GOOD: pass via environment variable
$ export API_TOKEN=abc123...
$ curl -H "Authorization: Bearer $API_TOKEN" ...

Tokens visible in history or ps are an incident waiting to happen. Use env vars or ~/.netrc.

5-4. Basic Auth

$ curl -u username:password https://example.com/private

-u accepts user:pass. HTTPS only.

6. wget's Strength: Reliable Downloads

Conclusion: wget saves by default and follows redirects; -O renames, -c resumes a download.

6-1. Basic: Save as a File

$ wget https://example.com/sample.tar.gz
Lina: wget saves the file with no flags at all. Easy.
Linny-senpai: Right — wget saves by default. And unlike curl, it follows redirects automatically. For simple downloads, wget is harder to mess up.

wget does not overwrite on its own

If the name is already taken, wget saves sample.tar.gz.1 — the same name with a number appended. The exception is -O name: that form does overwrite the name you gave. Keep that difference in mind.

6-2. Save With a Custom Name: -O

$ wget -O custom.tar.gz https://example.com/sample.tar.gz

-O means the opposite in curl and wget

  • curl's -O: save using the URL's filename
  • wget's -O: specify the filename (equivalent to curl's -o)

Everyone who uses both gets bitten by this at least once.

6-3. Resume an Interrupted Download: -c

$ wget -c https://example.com/big.iso

-c stands for continue. If your connection dropped halfway, this picks up from where it stopped instead of restarting.

6-4. Retries and Timeouts

$ wget --tries=5 --timeout=30 https://example.com/file.zip

Survives flaky networks by retrying.

7. Recursive Download wget -r (Handle With Care)

Conclusion: wget -r recurses through links; always cap depth with -l and scope with -np.

$ wget -r -l 2 -np https://example.com/docs/
  • -r: recurse through links
  • -l 2: cap depth at 2 levels
  • -np: don't ascend to parent directories

8. Encoding and Line-Ending Traps

Conclusion: curl and wget hand you raw bytes; fix encoding with iconv and CRLF with tr.

Lina: The page I fetched is full of unreadable symbols where the text should be.
Linny-senpai: That's a character-encoding mismatch. Browsers detect the encoding and fix it for you. curl and wget hand you the raw bytes instead. For pages that are not UTF-8, run the output through iconv.
Lina: Line endings look off too.
Linny-senpai: Files from Windows servers often use \r\n (CRLF). On Linux, strip them with dos2unix or tr -d '\r'.
# Convert Shift_JIS to UTF-8 while saving
$ curl -s https://example.com/sjis.html | iconv -f SHIFT_JIS -t UTF-8 > page.html

# Strip CRLF to LF
$ curl -s https://windows-server.example/data.csv | tr -d '\r' > data.csv

> here is shell redirection. It also replaces the target file's contents without asking. Do not point it at a name you still need.

9. Common Beginner Pitfalls

Conclusion: Common traps: forgetting -O/-o to save, forgetting -L, and unquoted & in URLs.

9-1. curl URL Saved Nothing

Cause: curl prints to stdout by default. To save, use -O or -o.

$ curl -O https://example.com/file.zip
$ curl -o my.zip https://example.com/file.zip

9-2. Got the Redirect Page Instead of the Content

Cause: forgot -L.

$ curl -L https://github.com/...

9-3. Part of the Query Is Ignored, or an Odd Error Appears

Cause: the shell read & as the background-execution operator. Everything before & goes to curl; the rest becomes a separate command.

  • With &page=2 (an = is present): no error appears. &page=2 is silently dropped and only the result differs. This is the hardest case to notice
  • With &page (no =): you get page: command not found
# BAD
$ curl https://api.example.com/search?q=linux&page=2

# GOOD
$ curl "https://api.example.com/search?q=linux&page=2"

9-4. SSL Certificate Error

Cause: self-signed cert, expired cert, internal CA, etc.

# With verification (recommended)
$ curl https://internal.example.com

# Skip verification (emergencies only — never in production)
$ curl -k https://internal.example.com

-k disables verification, opening the door to man-in-the-middle attacks. For production, fix the certificate chain instead.

9-5. Proxy Environment Can't Connect

$ export http_proxy=http://proxy.example.com:8080
$ export https_proxy=http://proxy.example.com:8080
$ curl https://example.com

In corporate environments, set http_proxy / https_proxy.

10. Mini Exercises

Conclusion: Use httpbin.org to practice status checks, JSON POST, and following redirects.

Lina: Theory's in! Let me try this in the terminal.
Linny-senpai: Three exercises. https://httpbin.org is a safe sandbox for HTTP testing — hit it freely.

Make an empty practice directory and work inside it. That way you never overwrite a file you already had.

# Prepare a practice directory and move into it
$ mkdir -p ~/curl-practice && cd ~/curl-practice
  • ~ stands for your home directory, the space that belongs to you
  • && means "run the right side only if the left side succeeded." Two separate lines work the same

Exercise 1: Check the HTTP status code and content-type of https://httpbin.org/get.

Show Hint 1 (Direction)

You do not need the body. There is a flag that fetches only the labels.

Show Hint 2 (Command name)

The command is curl. The flag is the single letter from "4. Inspect Headers Only".

Show Answer 
$ curl -I https://httpbin.org/get
HTTP/2 200
date: ...
content-type: application/json
...

The leading 200 means success. content-type: application/json says the reply is JSON.

Exercise 2: POST the JSON {"hello":"world"} to https://httpbin.org/post.

Show Hint 1 (Direction)

You need three things: the method, the "this is JSON" declaration, and the body.

Show Hint 2 (Command name)

The command is curl. The flags are the three used in "5-2. POST JSON".

Show Answer 
$ curl -X POST https://httpbin.org/post \
    -H "Content-Type: application/json" \
    -d '{"hello":"world"}'

You'll see "json": {"hello": "world"} echoed back in the response.

Exercise 3: Follow the redirect chain at https://httpbin.org/redirect/3 and fetch the final page.

Show Hint 1 (Direction)

curl ignores the "moved" note unless you tell it to chase the note.

Show Hint 2 (Command name)

The command is curl. The flag is the single letter from "3. The #1 Beginner Trap" — the first letter of Location.

Show Answer 
$ curl -L https://httpbin.org/redirect/3

After three hops it lands on /get and returns that response.

11. Copy-Paste Templates

Conclusion: Copy-paste curl and wget templates for saving, redirects, headers, and POST.

curl templates

# Print to screen (debug)
curl URL

# Save (using URL's filename; overwrites same name)
curl -O URL

# Save (custom name; overwrites same name)
curl -o name URL

# Follow redirects and save (GitHub Releases, etc.)
curl -LO URL

# Headers only
curl -I URL

# Get just the status code
curl -s -o /dev/null -w "%{http_code}\n" URL

# POST JSON
curl -X POST URL \
    -H "Content-Type: application/json" \
    -d '{"key":"value"}'

# Bearer token
curl URL -H "Authorization: Bearer $API_TOKEN"

# Basic auth
curl -u user:pass URL

# With progress bar
curl -O --progress-bar URL

wget templates

# Basic download (appends a number if the name is taken)
wget URL

# Save with custom name (overwrites that name)
wget -O custom.name URL

# Resume an interrupted download
wget -c URL

# Retries and timeout
wget --tries=5 --timeout=30 URL

# Quiet download (minimal log)
wget -q URL

# Recursive (mind the manners)
wget -r -l 2 -np --wait=2 https://example.com/docs/

Looking Back

Conclusion: Pick by purpose, look before you download, and add -L on external sites.

Lina: Let me sum up. Just grabbing a file means wget. Looking at an API or headers means curl.
Linny-senpai: Exactly. When unsure, ask "am I saving this, or reading it?"
Lina: And run pwd and ls before downloading, because curl -O overwrites without asking.
Linny-senpai: Perfect. Plus -L when curl hits an external site. Take those three home and you're set.

Three-Line Recap

Conclusion: Pick the tool by purpose, check where files land, and add -L for external sites.

  1. Use curl for APIs and headers. Use wget when you only need the file
  2. Run pwd and ls first. curl -O and curl -o overwrite the same name without asking
  3. Add -L when curl hits an external site so it follows the redirect

Summary: What to Read Next

Share this article

Next steps

Next article → base64 Command: Encoding and Decoding Basics Practice with quiz → Verify your understanding Practice with lesson → Run commands in the terminal