Note: This article is an English translation of my original article, which you can find here.
Overview
This article describes the causes and solutions to the error when using jq join in AWS CloudShell.
jq error in AWS CloudShell
This section describes an error that occurs when using jq in AWS CloudShell.
If you try to join an array that contains numbers, you will get jq: error string and number cannot be added
.
Expected:
$ echo '["example",0]' | jq 'join(".")'
"example.0"
Actual:
$ echo '["example",0]' | jq 'join(".")'
jq: error (at <stdin>:1): string (".") and number (0) cannot be added
Cause
This section describes the cause of this error.
The version of jq preinstalled in CloudShell is 1.51.
$ jq --version
jq-1.5
A bug in the 1.5 version of join does not convert number types into strings, resulting in this error.
L1422 - jq/builtin.c at jq-1.5 · stedolan/jq
"def join($x): reduce .[] as $i (null; (.//\"\") + (if . == null then $i else $x + $i end))//\"\";",
Note:
Ver 1.6 fixed this bug. You can confirm that in the joined source, which converts the type number to a string with tostring
.
def join($x): reduce .[] as $i (null;
(if .==null then "" else .+$x end) +
($i | if type=="boolean" or type=="number" then tostring else .//"" end)
) // "";
The issue:
string and number cannot be used for join() #1595
If you run the join
example, it errors:
jq: error (at <stdin>:0): string (" ") and number (1) cannot be added
exit status 5
jq 'join(" ")'
input: ["a",1,2.3,true,null,false]
output: "a 1 2.3 true false"
version 1.5
From the docs:
Numbers and booleans in the input are converted to strings. Null values are treated as empty strings. Arrays and objects in the input are not supported.
Solution 1
I would describe two means of resolving this error.
First, I explain how to fix it to work with jq1.5.
If you convert the number to a string before passing it to join, it will work as expected in 1.5. And, of course, it also works fine with 1.6.
$ echo '["example",0]' | jq 'map(.|tostring) | join(".")'
"example.0"
Solution 2
Next, I explain how to install jq1.6 in CloudShell.
However, the yum provided with CloudShell does not upgrade the pre-installed jq2, so you will download the binary in an appropriate directory.
Note: (about directory persistence)
Please note that if you place binary files in /usr/bin
, the files will be deleted when CloudShell is restarted.
If you place the file in your home directory, the data will be persistent, but please note that the file will be deleted when you do not use that CloudShell environment for more than 120 days.
Service quotas and restrictions for AWS CloudShell - AWS CloudShell
Practical Examples
Here is a concrete example to explanate the solution.
Create an installation directory in your home directory and download the binary. Of course, the directory is arbitrary, but here we assume jq1.6
.
cd ~
mkdir jq1.6
cd jq1.6
wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
chmod +x ./jq
As mentioned earlier, this bug has been Fixed in 1.6, so it works as expected.
$ ~/jq1.6/jq --version
jq-1.6
$ echo '["example",0]' | ~/jq1.6/jq 'join(".")'
"example.0"
It is helpful to set up an alias.
$ echo "alias jq=~/jq1.6/jq" >> ~/.bashrc
$ . ~/.bashrc # Reflect the alias in the shell where you are logged in:
$ echo '["example",0]' | jq 'join(".")'
"example.0"
To run as a shell script
Note that when running as a shell script, the login shell environment is not inherited, so the alias is undefined, and it executes with jq 1.5.
./join.sh
:
#!/bin/bash
echo '["example",0]' | jq 'join(".")'
$ bash ./join.sh
jq: error (at <stdin>:1): string (".") and number (0) cannot be added
In this case, if you specify interactive behavior with the -i
option, the shell in the execution environment will read ~/.bashrc
and execute the script, as expected.
$ bash -i ./join.sh
"example.0"
Conclusion
This section summarizes how to deal with the error when using jq in CloudShell.
I have described two solutions.
Solution 1 has the advantage of not requiring any changes to the CloudShell environment and is portable as it works with both 1.5 and 1.6. 3
Solution 2, however, has the advantage of running on 1.6, so it can also handle behaviors other than join error.
One of the changes other than join in 1.6:
Different behavior when omitting jq arguments
Please use one of these methods depending on your situation.
Reference
I'd like to print the value of members and id from below jq output:
$ cat test_|jq -r '.[] | select(.name=="AAA") | .'
{
"name": "AAA",
"members": 10,
"profiles": 0,
"templates": 0,
"ldapGroups": 0,
"ldapMembers": 0,
"id": "20"
}
Unfortunately it works for one of each only:
$ cat test_|jq
…
Top comments (0)