jq Basics: How to Process JSON in the Shell

2026-05-21 Reading time: About 12 min Difficulty: Intermediate

What You'll Learn

• How to shape JSON from curl or API responses into readable form
• The essential jq vocabulary you actually need: select, map, -r, and more
• Safe patterns that avoid the common traps: stray quotes leaking into shell variables, null propagation, and broken file writes

What Is jq?

jq is a JSON-aware filter language and command-line tool. Unlike forcing grep/awk onto JSON, jq understands structure, which means fewer regex disasters in API pipelines, log aggregation, and CI scripts.

• Follows the Unix pipe model: stdin → filter → stdout
• Filter expressions are evaluated left to right, just like shell pipes
• Has a real type system: numbers, strings, arrays, objects, null, and booleans

How Do You Install jq?

The fastest path is your system package manager. If you script jq into CI, pin the version with jq --version because behavior differs between major releases.

# Ubuntu / Debian
$ sudo apt update && sudo apt install jq

# RHEL / Rocky / AlmaLinux
$ sudo dnf install jq

# macOS (Homebrew)
$ brew install jq

# Verify
$ jq --version

How Do You Read a Basic Filter?

A jq filter is a transformation applied to its input. The empty filter . is the identity (pretty-print and return). Use .field to read an object key and .[] to explode an array.

1. Pretty-print

$ echo '{"name":"linny","age":3}' | jq .

{
  "name": "linny",
  "age": 3
}

2. Extract a key

$ echo '{"name":"linny","age":3}' | jq '.name'

"linny"

3. Iterate over an array

$ echo '[{"id":1},{"id":2}]' | jq '.[]'

{"id":1}
{"id":2}

How Do You Work with Objects and Arrays?

Combine .[] (explode array), , (parallel evaluation), and | (pipe filters). It feels exactly like grep | awk.

Pull a field from every element

$ echo '[{"id":1,"tag":"a"},{"id":2,"tag":"b"}]' \
  | jq '.[].tag'

"a"
"b"

Build a tuple of selected fields

$ echo '[{"id":1,"tag":"a"},{"id":2,"tag":"b"}]' \
  | jq '.[] | {id, tag}'

{"id":1,"tag":"a"}
{"id":2,"tag":"b"}

{id, tag} is shorthand for {id: .id, tag: .tag}. Works for plain identifier keys only.

Safely descend into nested fields

$ echo '{"a":{"b":{"c":42}}}' | jq '.a.b.c'
42

# Missing keys produce null; the ? suppresses iteration errors
$ echo '{}' | jq '.a.b?.c?'
null

How Do You Filter with select?

select(condition) passes through only values where the condition is true. The idiom is to explode an array with .[] first, then pipe into select. Think SQL WHERE.

Equality filter

$ echo '[{"s":"ok"},{"s":"ng"},{"s":"ok"}]' \
  | jq '.[] | select(.s=="ok")'

{"s":"ok"}
{"s":"ok"}

Numeric and compound conditions

$ jq '.[] | select(.score >= 80 and .active)'
$ jq '.[] | select(.tag=="a" or .tag=="b")'
$ jq '.[] | select(.tag | startswith("v1"))'

Check for the presence of a key

$ echo '[{"a":1},{"b":2}]' \
  | jq '.[] | select(has("a"))'

{"a":1}

jq '.[] | select(.status=="error") | .message'

What Do map, length, and keys Do?

map(f) applies f to every element of an array, length returns the size, and keys lists the keys of an object. These are the bulk operations for working on an array as a whole.

map: transform an array

$ echo '[1,2,3]' | jq 'map(. * 10)'

[
  10,
  20,
  30
]

map(f) is equivalent to [.[] | f]. The difference: .[] | f streams one element at a time, while map(f) returns the array intact.

length: count elements

$ echo '[{"id":1},{"id":2},{"id":3}]' | jq 'length'
3

For strings it returns character count, for objects the number of keys, for null it returns 0. Type-sensitive.

keys: list the keys

$ echo '{"a":1,"c":3,"b":2}' | jq 'keys'

[
  "a",
  "b",
  "c"
]

keys is sorted, keys_unsorted preserves insertion order. Use keys when you need deterministic output for tests.

How Do You Format the Output? (-r / -c)

jq . is for humans, -r (raw) is for shell variables, and -c (compact) is for piping into line-oriented tools. Eight out of ten jq bugs come from picking the wrong output mode.

-r: strip string quotes for raw output

$ echo '{"name":"linny"}' | jq '.name'
"linny"

$ echo '{"name":"linny"}' | jq -r '.name'
linny

When you assign to a shell variable with NAME=$(...), always use -r. Otherwise the literal "linny" (with quotes) lands in the variable.

-c: one JSON per line

$ echo '[{"id":1},{"id":2}]' | jq -c '.[]'

{"id":1}
{"id":2}

Pair -c with while read line loops or xargs -I {}. It's the bridge between JSON tooling and traditional Unix tools.

TSV / CSV output (@tsv / @csv)

$ echo '[{"id":1,"name":"a"},{"id":2,"name":"b"}]' \
  | jq -r '.[] | [.id, .name] | @tsv'

1	a
2	b

@csv quotes string values; @tsv uses tab separators. Pack values into an array first, then pipe into the formatter.

Practical Patterns for Daily Work

API response shaping, log aggregation, and config-file updates: three patterns you'll reuse forever. Memorize the templates and adapt.

Fetch with curl and extract specific fields

$ curl -sS "https://api.example.com/users" \
  | jq -r '.users[] | "\(.id)\t\(.name)"'

-sS means silent on success, loud on error. The string interpolation "\(.id)\t\(.name)" lets you pick any separator.

Aggregate with group_by

$ jq '[.[] | {status}] | group_by(.status) | map({status: .[0].status, count: length})'

group_by(f) returns an array of arrays grouped by f. Wrap with map({key: ..., count: length}) to produce a count table.

Rewrite part of a config file

# Bump version in package.json
$ jq '.version = "1.2.3"' package.json > package.json.tmp \
  && mv package.json.tmp package.json

# Safer alternative with sponge
$ jq '.version = "1.2.3"' package.json | sponge package.json

Append to an array

$ echo '{"items":[1,2]}' | jq '.items += [3]'

{
  "items": [
    1,
    2,
    3
  ]
}

|= is reassignment, += is add-and-assign. Combine with deep paths: .items |= map(. * 2).

What Are the Common Pitfalls?

Three traps catch nearly everyone: null propagation, type errors, and passing shell values into filters. Reading the error message before searching cuts debug time in half.

1. Cannot iterate over null

# .users is missing
$ echo '{}' | jq '.users[]'
jq: error (at <stdin>:1): Cannot iterate over null (null)

Fix: provide a default with // [].

$ echo '{}' | jq '.users // [] | .[]'
# (no output, exit 0)

2. Embedding shell variables in a filter

# Bad: quotes collide
$ KEY="name"
$ jq ".$KEY" file.json   # after shell expansion becomes '."name"' on some shells

# Good: pass it through --arg
$ jq --arg key "$KEY" '.[$key]' file.json

--arg passes the value as a string; --argjson passes it as JSON. Use --argjson for numbers, arrays, or booleans.

3. "It looks array-like but .[] fails"

$ echo '"abc"' | jq '.[]'
jq: error (at <stdin>:1): Cannot iterate over string ("abc")

A string is not an array. Use the type filter to confirm what you actually have.

$ echo '"abc"' | jq 'type'
"string"

4. Drive a script with the exit code

jq -e returns exit code 1 when the output is false or null, so you can chain it with || for control flow.

$ echo '{"ok":false}' | jq -e '.ok' || echo "failed"
false
failed

# Pretty-print and page through
jq . file.json | less

# Capture into a variable (always use -r)
NAME=$(curl -sS "$URL" | jq -r '.name')

# Stream array elements as 1-line JSON into xargs
curl -sS "$URL" | jq -c '.users[]' \
  | xargs -I {} sh -c 'echo "USER: {}"'

# Null-safe extraction with default
jq -r '.path.to.value // "DEFAULT"'