FZF + JQ = Interactive JQ (+ a vim bonus)

Problem

Do you frequently query an API and get back a large JSON payload, like:

$ curl https://jsonplaceholder.typicode.com/todos
[
  {
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": false
  },
  ...
]

So you pipe it to jq, but wish you could interactively query the data?

You may be familiar with some utilities that provide this interactivity, like jqp, but these come with their own downsides, namely subtle bugs due to the re-implementation of the original JQ in language X, but also they are additional dependencies you have to track and install for your platform architecture and OS/distribution.

Solution

Here is a hack (err... ✨ pro-tip ✨) to get JQ interactivity for “free” (as long as you have fzf on your machine):

$ echo '' | fzf --print-query --preview "cat $JSON_FILE_ON_DISK | jq {q}"
$ echo '' | fzf --print-query --preview "cat <(curl $API_URL) | jq {q}"

How does this work?

echo '' | fzf --print-query --preview "cat $JSON_FILE_ON_DISK | jq {q}"
^^^^^^^   ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [1]          [2]                          [3]

1. When running fzf, by default it will prompt you to fuzzy search through a list of files in your current working directory. So, we suppress this by echoing an empty string into fzf.

2. --print-query is an optional flag that will print out the query we typed upon hitting Enter.

3. --preview flag is what enables all of this. We are using fzf's previewing capability to cat out JSON content, pipe it to jq, injecting our query as jq arguments via {q} placeholder, and we see the results in fzf's preview window.

Further Improvements

We can improve this further by creating a shell script in our global $PATH to turn it into a quick handy utility.

For example, assuming this script is named fjq:

#!/bin/bash

echo '' | fzf-tmux -p '80%' --print-query --preview "cat ${1} | jq {q}"

We can now use this like:

# interactive jq on a single JSON file
$ fjq file.json

# interactive jq on a directory with multiple JSON files
$ fjq /path/to/*.json

# interactive jq on-the-fly
$ fjq <(curl https://jsonplaceholder.typicode.com/todos)

Voila! Interactive json querying with just jq and fzf.

Vim Tip

A semi-interactive experience can be achieved (without FZF) by leveraging Vim's :! and :help filter to filter contents of Vim buffers through an external program and write the output back into Vim's buffers, like:

$ curl https://jsonplaceholder.typicode.com/todos | vim -

# inside vim ...

:%! jq .[].completed
:%! sort | uniq -c

We can even use our previous script for full JQ interactivity:

:! fjq %
" or
:%! fjq %
" or 
:0read ! fjq %

And so on.

Conclusion

We've seen how terminal programs and shell utilities can be composed to create powerful workflows.

We took 2-3 tools that already exist on our system and achieved 80% of what other tools try to accomplish by reimplementing jq in Go, Ruby, Rust, Python, Node.js ...etc.

This is not to say you should not reach for these extra tools, but, in many cases, learning how to leverage existing tools enables you to solve more problems than what these tools were created for.

Happy hacking! 😄