API testing is a crucial part of cloud development strategy, and we have plenty of frameworks and tools like Mocha, RestAssured, Postman/Newman etc to help use write them. Today, I'd like to introduce a new framework that was developed recently to make writing tests very easy.
Vibranium
Vibranium is a command line based API testing framework developed using Node JS and utilises JSON to write tests. To know more about Vibranium, head over to Vibranium documentation or NPM page. It is a pretty new framework and was released less than a month ago.
Full disclosure: I'm the author of this module and so feel free to drop in your feedbacks and help me improve it.
The biggest advantage we get with using JSON for writing tests is that it is very easy to understand, maintain and reuse. We don't need to know any programming language like JS or Java to get started with writing JSON and so getting started with it is pretty easy.
Installation
Installing Vibranium is pretty simple. Just like for any other npm module, install the latest version of Node JS and then run the following command in terminal
npm install -g vibranium-cli
To verify the installation, try running vc --version
Workspace, Endpoints, Scenarios and Collections
In Vibranium, a workspace refers to the directory in your system where you'll have your tests written.
An endpoint refers to the actual test JSON object inside the scenario. It'll have a name, description of the endpoint test case, the API url, API method, payload etc. An endpoint is also referred to as an API in Vibranium.
A scenario refers to the json file that contains all the endpoints belonging to one particular use case. A scenario can be any use case related, like for all CRUD operations for an object or an end to end flow.
A collection refers to a folder/directory/package with scenarios inside it. All the collections will be placed inside the scenarios directory inside the tests directory. It is just a means to group or organise scenario files.
Setup and Configuring Vibranium
Open Command Line/ Terminal and change the directory to the directory that you with to make as your workspace and run the following command and enter the answers to the questions that follow
vc setup
Make sure you navigate to the workspace directory before running this command as it sets the current working directory as the workspace.
Example:
cd ˜
mkdir workspace
cd workspace
vc setup
Please enter your user id: tester1
Please enter your email id: some.email@domain.com
Please enter your name: VibraniumTester
This step basically registers the current working directory as the workspace in the internal Vibranium configuration, and creates the initial set of directories like jobs and logs.
After setting up, we need to configure Vibranium and provide the details of the system on which we'll be doing the tests. This is done with the config.json file. Details on the config file can be obtained in the documentation. For now, let's add a system with the following JSON entry:
Writing Tests
Let's write our first few tests.
For the examples used in this page, let's assume that there's an app available that has a list of endpoints for creating, viewing and updating user data.
Creating a scenario
As mentioned earlier, a scenario refers to the JSON file in which we write tests. We can either create a JSON file inside a collection and fill in the fields, or, to make things easy, use ask Vibranium to create a scenario file for you, pre-filling all the essential data.
In order to create a new scenario, run the command:
vc c -c {collectionName} -s {scenarioName}
Here vc c or vc create refers to the create command in Vibranium, and {collectionName} and {scenarioName} refers to the collection name (directory/package name) and the scenario JSON file name respectively. Refer to create command for more details
Once you run this command, Vibranium will create a new scenario file and open it in your default JSON viewer. You can either delete the entries inside the endpoints key in the scenario of keep it for reference. For the examples below, we will create fresh tests, so it is recommended that you remove them.
Write your first test case
Let's start writing our first test.
Assume that we are writing a test for the endpoint GET /api/v1/users. The simplest test case that you can write is to check if the endpoint is up and running and returns status 200 OK when called. For this, add the following JSON object into the endpoints array in the scenario:
The above mentioned JSON object will call the /api/v1/users using GET (if we don't specify any method, it takes GET as default) and checks whether the API returns a status of 200. Writing a basic test case is this simple!
If you use VS Code, there will be auto fill and suggestion available for Vibranium tests automatically available!
Now say we want to validate more things, other than the status code. For this, we can use the expect key in the endpoint. For example if we want to call the users list api to get all user details and validate if
- API returns a status code of 200
- The time to first byte should be less than 300ms
- The total time taken by API should be less than 700ms
- The response is of content type JSON
- Response should have at least one entry
- At least one user is admin
- All users have proper IDs
Was that a little too much? let's discuss it line by line.
- Till the expect key, it is pretty much the same as the before example, so I'm leaving that part
- "status": 200 tells Vibranium that the expected HTTP status code is 200 and to fail the test if we get a different value. By default it takes 200, so even if you don't specify this line, the check is automatically handled
- "headers": {...} refers to checks related to the response headers. It is a key-value pair, with key referring to the repose header key and the value referring to the expected value for the key.
- "content-type": "application/json" as mentioned in the previous point, this means that the expected value for the content-type header in the response is application/json
- "response": {...} refers to checks related to the response body of the endpoint. This is also a key-value pair, with the key containing the test name/description and the value being simple JS snippets that does the check. There is a special response check when you specify the key as schema. We'll go through this later.
- "There should be atleast one entry": "{response.length} > 0" The key refers to the check description and the value contains a variable (any string enclosed in curly brackets '{' and '}'). {response} is a variable that contains the response body. We use dot notation to parse the object, so {response.id} means the id key inside the response. For more details, refer to Variables.
- "Atleast one user should be admin": "{response.all.isAdmin}.filter(isAdmin => isAdmin).length >= 1" As mentioned in the previous point, we use dot notation here and response is a special variable with our response value in it. We have special keys like all, any, any_n etc about which we'll discuss in detailed later, but for now, it just means that we are taking all the entries in the array. For example, {response.all.id} is the same as response.map(r => r.id) in JavaScript.
- "All users have an ID of 32 characters": "{response.all.id}.every(id => id.length === 32)" If you have understood the previous point, this is very simple. {response.all.id} gives you an array of IDs and we are evaluating JS every function on the array to see if all the IDs have a length of 32.
- "timing": {...} refers to the checks related to the response timing. You can specify a max value for the response timing and fail the test if it takes more than a certain amount of time. All timing values are in milliseconds. The available timing values that you can check are total, firstByte, wait, dns, tcp and download
- "total": "<700" fail the test if the endpoint takes more than 700ms in total
- "firstByte": "<300" fail the test if the endpoint takes more than 300ms for the first byte of the response
Time for a slightly more complex test…
Let's write test to update the details for a particular user. The basic requirements of this test is that we first need to have a user in the system. There are two ways of proceeding with this. Either we can take a user from the users list api and update it, or create a new user and then update it. Since in many cases, there is no guarantee that the system has data already available, so we'll proceed by creating a new user. In most cases, we might already have the test for this, but for the sake of this example, the endpoint is also mentioned here. Now we'll learn how to use this endpoint as a dependency in our update API test.
Here is the JSON for update user test. I'll go through the important parts of this example below.
- All the keys till variables are self explanatory and so I'm skipping them
- variables is a key used to define variables in the test. Head over to Variables if you need more details on this, but to explain in simple terms, it is just a key-value pair, with the key denoting the variable name and the value denoting the value for the variable.
- "newUserName": "{dataset.names}" means that we are creating a new variable named newUserName with the value {dataset.names}. dataset is a special keyword in Vibranium, used to denote pre-defined data values. {dataset.names} means use any value from the inbuilt names dataset. More details on datasets is also available in the previously mentioned Variables page.
- payload key is used to denote the payload to be used in the endpoint. It can be of any type, depending on the endpoint. A payload can also be a string, starting with the ! symbol to denote that the payload needs to be pulled from a file. So if the payload value is !payloadForUpdateUser, then the payload values is taken from the file named payloadForUpdateUser.json inside the payloads directory inside the tests directory.
- dependencies key is used to denote the list of dependencies to be executed before executing the given endpoint. In this case, we need to run the create user api before running update user, and hence we define that api as a dependency. Head over to Dependencies for more details of dependencies.
- "api": "create_a_user" indicates that the api with the name create_a_user is a dependency for this endpoint. If the dependency is in the same scenario file, you just need to mention the api name, else if it in the same collection, we have to mention both api name and the scenario name and if the api is in a different collection, we need to specify api name, scenario and the collection. In this case, the endpoint is in the same file (as mentioned above) and so we define only the api key
- variable key denotes the variables that are to be pulled from the dependency response. So if we define "userId": "response.id", it means that after the create endpoint is executed, the id field from the response is taken and assigned to the variable named userId, so that we can use that value in our endpoint.
Datasets
Vibranium also provides different datasets to be used in tests. All datasets can be used by specifying {dataset_datasetName}, where datasetName can be any of the following
- names a random name
- harrypotter a random Harry Potter character name
- starWars a random star wars character name
- space a random celestial object name
- pokemon a random pokemon name (from generation 1)
- quotes a random quote
- got a random Game of Thrones character
- marvel a random Marvel character name
- spells a random Harry Potter spell/charm
- countries a random country name
Summary
As you can see, Vibranium tests can be written easily and since they are written in JSON, they are much more readable and maintainable than writing API tests in other frameworks. Even though there is a small learning curve when it comes to understanding the keys in the JSON and using them, once you get used to them, you can write all the tests in a very short time.
So what are you waiting for? Give Vibranium a try and let me know your feedback! :)
This article is also available in medium:
Medium
Top comments (0)