DEV Community

jhashimoto
jhashimoto

Posted on • Updated on

How to fix "jq: error string and number cannot be added" in AWS CloudShell?

Note: This article is an English translation of my original article, which you can find here.

Overview

This article describes the causes and solutions to the error when using jq join in AWS CloudShell.

jq error in AWS CloudShell

This section describes an error that occurs when using jq in AWS CloudShell.

If you try to join an array that contains numbers, you will get jq: error string and number cannot be added.

Expected:



$ echo '["example",0]' | jq 'join(".")'
"example.0"


Enter fullscreen mode Exit fullscreen mode

Actual:



$ echo '["example",0]' | jq 'join(".")'
jq: error (at <stdin>:1): string (".") and number (0) cannot be added


Enter fullscreen mode Exit fullscreen mode

Cause

This section describes the cause of this error.

The version of jq preinstalled in CloudShell is 1.51.



$ jq --version
jq-1.5


Enter fullscreen mode Exit fullscreen mode

A bug in the 1.5 version of join does not convert number types into strings, resulting in this error.

L1422 - jq/builtin.c at jq-1.5 · stedolan/jq



  "def join($x): reduce .[] as $i (null; (.//\"\") + (if . == null then $i else $x + $i end))//\"\";",


Enter fullscreen mode Exit fullscreen mode

Note:

Ver 1.6 fixed this bug. You can confirm that in the joined source, which converts the type number to a string with tostring.



def join($x): reduce .[] as $i (null;
            (if .==null then "" else .+$x end) +
            ($i | if type=="boolean" or type=="number" then tostring else .//"" end)
        ) // "";


Enter fullscreen mode Exit fullscreen mode

The issue:

string and number cannot be used for join() #1595

If you run the join example, it errors:

jq: error (at <stdin>:0): string (" ") and number (1) cannot be added
exit status 5

 jq 'join(" ")' input: ["a",1,2.3,true,null,false] output: "a 1 2.3 true false"

version 1.5

From the docs:

Numbers and booleans in the input are converted to strings. Null values are treated as empty strings. Arrays and objects in the input are not supported.

Solution 1

I would describe two means of resolving this error.

First, I explain how to fix it to work with jq1.5.

If you convert the number to a string before passing it to join, it will work as expected in 1.5. And, of course, it also works fine with 1.6.



$ echo '["example",0]' | jq 'map(.|tostring) | join(".")'
"example.0"


Enter fullscreen mode Exit fullscreen mode

Solution 2

Next, I explain how to install jq1.6 in CloudShell.

However, the yum provided with CloudShell does not upgrade the pre-installed jq2, so you will download the binary in an appropriate directory.

Note: (about directory persistence)

Please note that if you place binary files in /usr/bin, the files will be deleted when CloudShell is restarted.

If you place the file in your home directory, the data will be persistent, but please note that the file will be deleted when you do not use that CloudShell environment for more than 120 days.

Service quotas and restrictions for AWS CloudShell - AWS CloudShell

Practical Examples

Here is a concrete example to explanate the solution.

Create an installation directory in your home directory and download the binary. Of course, the directory is arbitrary, but here we assume jq1.6.



cd ~
mkdir jq1.6
cd jq1.6
wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
chmod +x ./jq 


Enter fullscreen mode Exit fullscreen mode

As mentioned earlier, this bug has been Fixed in 1.6, so it works as expected.



$ ~/jq1.6/jq --version
jq-1.6
$ echo '["example",0]' | ~/jq1.6/jq 'join(".")'
"example.0"


Enter fullscreen mode Exit fullscreen mode

It is helpful to set up an alias.



$ echo "alias jq=~/jq1.6/jq" >> ~/.bashrc
$ . ~/.bashrc  # Reflect the alias in the shell where you are logged in:
$ echo '["example",0]' | jq 'join(".")'
"example.0"


Enter fullscreen mode Exit fullscreen mode

To run as a shell script

Note that when running as a shell script, the login shell environment is not inherited, so the alias is undefined, and it executes with jq 1.5.

./join.sh:



#!/bin/bash
echo '["example",0]' | jq 'join(".")'


Enter fullscreen mode Exit fullscreen mode


$ bash ./join.sh 
jq: error (at <stdin>:1): string (".") and number (0) cannot be added


Enter fullscreen mode Exit fullscreen mode

In this case, if you specify interactive behavior with the -i option, the shell in the execution environment will read ~/.bashrc and execute the script, as expected.



$ bash -i ./join.sh 
"example.0"


Enter fullscreen mode Exit fullscreen mode

Conclusion

This section summarizes how to deal with the error when using jq in CloudShell.

I have described two solutions.

Solution 1 has the advantage of not requiring any changes to the CloudShell environment and is portable as it works with both 1.5 and 1.6. 3

Solution 2, however, has the advantage of running on 1.6, so it can also handle behaviors other than join error.

One of the changes other than join in 1.6:

Different behavior when omitting jq arguments

Please use one of these methods depending on your situation.

Reference

I'd like to print the value of members and id from below jq output:

$ cat test_|jq -r '.[] | select(.name=="AAA") | .'
{
  "name": "AAA",
  "members": 10,
  "profiles": 0,
  "templates": 0,
  "ldapGroups": 0,
  "ldapMembers": 0,
  "id": "20"
}

Unfortunately it works for one of each only:

$ cat test_|jq

  1. This is the version at the time of the article submission. It may be upgraded in the future. 

  2. I confirmed with AWS support. 

  3. If you have a development PC or other environment where you can install freely, you will likely install the latest version, 1.6. 

Top comments (0)