jq | Parse JSON with ease

You are a developer // devops engineer that works a lot on distributed systems? Maybe in the field of IoT and frequently review, query or manipulate semi complex or big JSON data structures like a Digital Twin? You are tired of doing repetitive, time consuming tasks? If at least on is a yes then this blog Post is for you!

What is jq?

jq is a cli based JSON processor that lets you read, query, filter and transform JSON data structures with ease ;).
jq can parse and pretty print JSON data. It lets you extract fields of any JSON structure. Manipulating values or changing enire keys of the structure is possbile with the power of jq.

How do I get jq?

You can get jq for Linux, Mac or Windows. jq is prepackaged for the major (Linux) package managers out there. Also you can build it from source.

Debian derivates: sudo apt install jq
Alpine Linux: sudo apk add jq
Brew: brew install jq
Or build it from source
Windows goes here ;)

Can we (finally) see it in action?

Sure. First of this is the basic usage of jq.

Basic usage: jq <filter> <file>
Or Pipe(d): echo '{"jq": "is really nice!"}' | jq .jq
- Output:
```
"is really nice!"
```

Lets assume a simple JSON data structure that represents some students that have attended some courses:

{
  "students": [
    {
      "name": "Alice",
      "age": 25,
      "courses": ["Math", "Science"]
    },
    {
      "name": "Bob",
      "age": 35,
      "courses": ["Art", "History"]
    },
    {
      "name": "Charlie",
      "age": 22,
      "courses": ["Math", "History", "Programming"]
    }
  ]
}

Usage

You want to see some queries? - Lets get our feets wet :)

Query the name of the first person.
- Query: jq '.students[0].name' data.json
```
Output:
"Alice"
```
Count all students within the structure
- Query: jq '.students | length' data.json
- Output:
```
Output:
3
```
Count all properties of a person object
- Query: jq '.students[0] | length' data.json
```
Output:
3
```
Show students that are older than 30
- Query: jq '.students[] | select(.age > 30) | .name' data.json
```
Output:
Bob
```
Show the names of all students that attended the Math course
- Query: jq '.students[] | select(.courses | index("Math")) | .name' data.json
```
Output:
"Alice"
"Charlie"
```
Show the name and age of all students that attended the Math course
- Query: jq '.students[] | select(.courses | index("Math")) | {name: .name, age: .age}' data.json
- Output
```
{
  "name": "Alice",
  "age": 25
}
{
  "name": "Charlie",
  "age": 22
}
```

Real World jq use cases

Get your public IPv4 and user agent by piping a http response from curl to jq and parse as JSON.
- Command: curl -s ifconfig.me/all.json | jq '. | {ip: .ip_addr, agent: .user_agent}'
- Output:
```
{
  "ip": "<redacted>",
  "agent": "curl/7.88.1"
 }
```
You need to know what host directories are used in all of your docker containers? And have no time to review each single container and mount point in Docker Desktop? Get all mapped host directories that are used by your docker containers with ease and only show the information of interest the mounts.
- Command: docker inspect $(docker ps -aq) | jq '.[].Mounts[].Source' | uniq
- Output:
```
"/home/christian/redis"
 "/home/christian/llama3"
 "/home/christian/go-queueing"
```

You only want to see all mounts used by stopped containers?
Sure, we can use the select operator to first filter for stopped containers (only) and then select the mounts property (array) and finally project only the Source property. Before we print to std out we make sure the mounts are distinct and possibly filter out duplicates.
docker inspect $(docker ps -aq) | jq '.[] | select(.State.Running == false) | .Mounts[].Source' | uniq

You need this more frequently and are to lazy to type this?
You can use an alias for this:
alias dockermnts='docker inspect $(docker ps -aq) | jq ".[] | select(.State.Running == false) | .Mounts[].Source" | uniq'

or even better, paste it in your favorite shell script (.bashrc, zshrc,…)

Conclusion

In this Post we learned what jq is, how it’s used and how jq query syntax works.
We also dived into some real world examples - how we can query the docker api with jq and parsing a http response with jq projection. If you have repetitive tasks on JSON data structures and need to go fast by beeing lazy then jq might be for you. Also as we saw in the docker api example it does not matter if the data structure is simple or complex or even small or big.
jq does the job for you.

For me personally jq has become an invaluable tool. It brought me the effectiveness, speed and comfort in querying, reviewing and manipulating JSON data structures in my daily job.

I hope you liked this Post and jq is also for you. If you want to see more about jq or have questions or suggestions, feel free to reach out.

Shout out to freepik for providing nice looking and meaningful illustrations.

Christian Baumann