Run API tests with Newman and send reports to Telegram

How about running your Postman collections of API requests and automatically getting a meaningful report right in Telegram? In this article, I will show you how to implement this trick in one small script! 🔥

First of all, it would be appropriate to note that this article is only a training example for creating a small part of a continuous integration pipeline, which can be easily done locally using Node.js. That will help you to practice running collections with (or without) executing tests and sending API requests using node-modules and without Postman. But for practical use, there are many great built-in solutions for the most widely known CI tools that you can easily find in the public domain. ♾️

Export of initial collection and environment from Postman 🚀

In order to run any collections of API requests, we first need to have them available as a separate file that can be pulled from Postman, for example. For this case, in my Postman, I had a collection for the CRM Project "ClientBase v5" which contains several dozen of the most likely API requests and more than 400 built-in tests and in some cases also pre-request-scripts.

In all tests, I deliberately used Chai Assertion Library BDD syntax to make it easier to develop automated tests based on them. Some tests contain the same assertions but use a slightly different syntax. The point is that the Postman and Chai Library syntax allows us to implement a specific assertion in different ways, which is certainly great. 🍵

All I needed to do here was to export the collection as a json file and save it locally. Further, it will be important for me to know only the path to this file.

I also exported as a json the set of environment variables. Most of these variables were created as dynamic, in other words, they are generated using the faker library. The values ​​of variables, as you can see in the picture, are not initially set and are filled automatically during the running of requests:

If you are interested, this Postman collection is available here (still growing).

Telegram Bot API 🤖

As I mentioned, аfter running the collection and generating a report, I will send a notification and that report to Telegram. Moreover, it’s important to note here that the receiver, in this case, may not necessarily be Telegram. For example, it can be Slack or any other resource which is available for interaction by API.

Well, for the training example, I chose Telegram because I just wanted to learn more about the capabilities of the Telegram Bot API. To get started, I needed to select an existing or create a new Telegram group to receive notifications and reports there. It could be any Telegram group (public or private). So I created a new group called "API alerts". The most important step was getting my group's chat ID. And I got it very easily with the help of a special bot inside Telegram - the "Get My ID" bot. I just forwarded one message from my newly created group to this bot and already had the chat ID. There were other ways to do this, but this one seemed faster.

Then it was just a matter of creating a bot with which I was going to interact using the request API. Well, I did it even faster and here is a detailed guide on how to create a Telegram bot using @Botfather. The main thing here is to get from Botfather a unique bot authentication token that looks something like 123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11. Further, I added my new bot to the created "API alerts" group and gave the admin permission to send messages to the group.

It seems there is no point in explaining this step in more detail because everything here is extremely simple and there are a lot of instructions on the Internet on how to do it.

Well, it's time to create a project.

Creating the project and installing the required dependencies 🗂️

Having completed the preparatory steps in Postman and Telegram, I finally created a new project in the IDE, initialized it, and proceeded to install the necessary dependencies. So for running the collection I used Newman which is a command-line Collection Runner for Postman. In order to install Newman locally in the project as a library, used the following command:

npm i newman

As for the report of collection running, in general, we can integrate it with our CI pipeline using the Postman CLI or Newman as a pure command-line Collection Runner and generate some reports for example with Allure or at least JUnit. However, we are also able to use Newman reporters and get at least (and sometimes much more) informative and user-friendly reports with a dashboard-style summary landing page and a set of different tabs which contain the detailed request information.

To be more precise, I often prefer to use Danny Dainton’s html-extra reporter which is an updated version of the standard Newman HTML reporter containing a more in-depth data output and a few helpful extras. 📑

So this reporter was installed in the project using the command:

npm i newman-reporter-htmlextra

In addition to using environment variables in the project, I also needed another module dotenv that loads environment variables from a .env file into process.env. After installing this module with the command:

npm i dotenv

I created a .env file in the root of the project, in which I put the necessary environment variables:

• COLLECTION_PATH - the path to a locally hosted json file with my Postman collection;
• ENVIRONMENT_PATH - the path to locally hosted json with environment variables which also was exported from Postman;
• TOKEN - the token of my Telegram bot;
• CHAT_ID - my Telegram group's chat ID.

In my case, it looked like this (the values ​​of sensitive variables have of course been changed):

COLLECTION_PATH=C:/Users/user1/.vscode/Newman-Collection-Run-Reporter-on-CRM-Project-example/CRM_Project_ClientBase_v5_collection.json
ENVIRONMENT_PATH=C:/Users/user1/.vscode/Newman-Collection-Run-Reporter-on-CRM-Project-example/ClientBase_v5_environment.json
TOKEN=270485614:AAHfiqksKZ8WmR2zSjiQ7_v4TMAKdiHm9T0
CHAT_ID=-1002956968

Well, the basic dependencies are installed, it's time for coding. 👨‍💻 In order to do this, I created a runner.js file at the root of the project in which I included the first block of code:

require('dotenv').config()
const newman = require('newman');

//Running collection with Newman, report generation:

newman.run({
    collection: process.env.COLLECTION_PATH.toString(),
    environment: process.env.ENVIRONMENT_PATH.toString(),
    reporters: ['htmlextra'],
    iterationCount: 1,
    reporter: {
        htmlextra: {
            export: 'report' + '/report.html',
            // template: './template.hbs'
            // logs: true,
            // showOnlyFails: true,
            // noSyntaxHighlighting: true,
            // testPaging: true,
            browserTitle: 'My Newman report',
            title: 'My Newman Report',
            titleSize: 4,
            // omitHeaders: true,
            // skipHeaders: "Authorization",
            // omitRequestBodies: true,
            // omitResponseBodies: true,
            // hideRequestBody: ["Login"],
            // hideResponseBody: ["Auth Request"],
            showEnvironmentData: true,
            // skipEnvironmentVars: ["API_KEY"],
            // showGlobalData: true,
            // skipGlobalVars: ["API_TOKEN"],
            // skipSensitiveData: true,
            // showMarkdownLinks: true,
            showFolderDescription: true,
            // timezone: "US/Boston",
            // skipFolders: "folder name with space,folderWithoutSpace",
            // skipRequests: "request name with space,requestNameWithoutSpace",
            displayProgressBar: true
        }
    }
})

As you can see, in the collection and environment fields, I put the COLLECTION_PATH and ENVIRONMENT_PATH as environment variables using the process.env property.

In reporter settings in the export field, I specified the directory where Newman will save my report file. In other words, after Newman finishes its work, a new report directory will appear in the root of the project in which the report.html will be placed.

Also as users, we are able to customize report templates just for our needs. Html-extra reporter allowed us to do this by uncommenting the lines of code inside the htmlextra field.

The problem of asynchronous and its solution 💡

At this stage, we may be in for some problem related to the asynchronous execution of the collection pass and sending the API call to Telegram. It is clear that Newman will take some time to run through all the requests and nested tests, as well as generate a report. Of course, this is only a couple of seconds, but the execution of the code will hasten to send a call to Telegram without waiting for the Newman.

To avoid the problem of asynchrony and ensure sequential execution of script blocks, I had to turn to the help of the module DeAsync which turns the async function into sync. It was installed with the command:

npm i deasync

This package allowed me to use loopWhile(predicateFunc) where predicateFunc is a function that returns a boolean loop condition. So I added the following block to my code:

let done
newman.run({
//...
})
    .on('start', () => {
        console.log(`Running a collection. Please wait a few seconds...`);
    }).on('done', () => {
        done = true;
    })
    require('deasync').loopWhile(() => {
        return !done;
    })

Well, it remains only to set up API calls to Telegram.

Sending files with Fetch API 📨

At this stage, I needed to set up sending two API calls to Telegram: the first one with a notification of the successful completion of the collection running and the second one containing an HTML report. These calls were to go immediately after Newman completed his work. To implement this task, it is certainly possible to use any HTTP client libraries for Node.js which are available as npm packages (Axios, Got, SuperAgent, etc.), but I turned to the Fetch API. The following command was used to install it:

npm i node-fetch

In the code, I used the fetch() method with the usual promise syntax:

const fetch = require('node-fetch')

//Sending an information notification to Telegram:

const url1 = `https://api.telegram.org/bot${process.env.TOKEN}/sendMessage?chat_id=${process.env.CHAT_ID}&text=
Your collection has been successfully run. The results are contained in the attached report below.`

fetch(url1, {
    method: 'GET'
})
    .then((res) => res.json())
    .then((res) => {
        console.log(res);
    })
    .catch((error) => {
        console.log(error);
    });

As you can see, the constant url1 contains a string with the URL to which the request is sent. This is a standard template that is set by the Telegram API. According to this template, I included the token and chat ID inside the URL as environment variables, which I pulled from the .env file using the process.env property.

The next request turned out to be a little more difficult to implement because it had to send a report.html file inside it, which first needed to be converted using the encoding type multipart form-data. Therefore to create readable "multipart/form-data" streams, I needed to use the Form-Data library. In order to install it locally, used the following command:

npm i form-data

Based on the syntax of this library, I created a form with one field containing the file stream of the report.html file and passed it to the fetch() method to submit as the request body. The result was the following block of code:

const FormData = require('form-data');
const fs = require('fs');

//Sending html-report to Telegram:

const url2 = `https://api.telegram.org/bot${process.env.TOKEN}/sendDocument?chat_id=${process.env.CHAT_ID}`

let readStream = fs.createReadStream('report/report.html');
let form = new FormData();

form.append('document', readStream);

fetch(url2, {
        method: 'POST',
        body: form,
    }
)
    .then((res) => res.json())
    .then((res) => {
        console.log(res, 'The report was successfully sent to 
        Telegram');
    })
    .catch((error) => {
        console.log(error);
    })

Well, it's finally time to execute the written script. However, in the package.json file, I previously changed the command to run scripts by writing:

"test": "node runner.js"

When I ran the script using the command:

npm test

immediately in the terminal appeared a special green bar that indicated the progress of running the collection:

After the running was over I received two messages with notification and report.html file directly in my Telegram group "API alerts": ✉️

Great, let's open the resulting report and see what it is:

In addition, in the terminal, I received response bodies from the Telegram API containing detailed information about what was sent and where:

Instead of a conclusion 👋

In this article, I tried to describe the entire process in as much detail as possible, from exporting the Postman collection to receiving a report on its running in Telegram. I hope that repeating the steps I did will help you charge your API skills. 💪

This project with detailed installation and usage instructions is completely available here.