Viktorija Filipov

Posted on Jul 2, 2023

Cypress Workshop Part 11: Reporting - Qase, Github Actions

#cypress #javascript #testing #tutorial

Qase

Qase is a test management tool, used for test documentation, test execution, reporting etc. In this tutorial, you will learn how to connect your test run in Cypress and push results to Qase. This will serve to have one report for manual and for automated testing.

Steps:

Go to Qase website and create account: Qase
Once you do it create a new project called for example Cypress Workshop - Book Store

Inside the project create a new test suite and call it Web for example.
Inside the web suite, create and document all test cases we have automated for book store app. Fill them with all details and steps. Create one additional test case for example for registration, that is NOT part of our automation suite, we want to execute it manually but still needs to be documented here.

Create a new test plan and call it Web Regression for example. Include all our test cases there.

Go to test runs and create a new test run that includes all test cases from the test plan. Start a test run.

Go to project settings → setting and check these two points to allow bulk push of cypress results

Go to api tokens page and create a new token. Save token value somewhere we will need it later.

Go to your cypress project and execute in terminal these 3 commands to install dependencies for Qase:

npm install cypress-qase-reporter

npm install dotenv

npm install prompt

Under root directory of the project create new folder scripts and new file cypress-with-qase.js
Write the following code inside to manage a test run with case (upon each run prompt will ask for Qase specific values of project and run, and it will also ask for which automation suite you want to run. We are also providing basic Cypress config)

require('dotenv').config();
const prompt = require('prompt');
const cypress = require('cypress');

prompt.start();
prompt.get(
    [
        {
            name: 'PROJECT_ID',
            description: 'Provide Qase PROJECT_ID',
            type: 'string',
            required: true,
        },
        {
            name: 'RUN_ID',
            description: 'Provide RUN_ID for the Qase test run',
            type: 'number',
            required: true,
        },
        {
            name: 'SUITE',
            description: 'Provide SUITE folder path for the Qase test run',
            type: 'string',
            required: true,
        },
    ],
    function (err, result) {
        cypress.run({
            spec: `cypress/e2e/${result.SUITE}/*.cy.js`,
            browser: 'chrome',
            reporter: 'cypress-qase-reporter',
            headed: true,
            reporterOptions: {
                apiToken: process.env.QASE_API_KEY,
                projectCode: result.PROJECT_ID,
                runId: result.RUN_ID,
                logging: true,
            },
        });
    }
);

Go to package.json file and add new command for running Cypress with Qase (line 13):

{
  "name": "cypress-workshop",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "cypress-cli-prod": "cypress open --env prod=1",
    "cypress-headed-prod": "cypress run --headed -b chrome --env prod=1",
    "cypress-headless-prod": "cypress run --headless -b chrome --env prod=1",
    "cypress-cli-staging": "cypress open",
    "cypress-headed-staging": "cypress run --headed -b chrome",
    "cypress-headless-staging": "cypress run --headless -b chrome",
    "cypress:run:qase": "QASE_REPORT=1 node scripts/cypress-with-qase.js",
    "eslint": "eslint cypress",
    "eslint-fix": "eslint cypress --fix"
  },
  "author": "",
  "license": "ISC",
  "husky": {
    "hooks": {
      "pre-commit": "npm run eslint-fix"
    }
  },
  "devDependencies": {
    "cypress": "^10.0.0",
    "eslint": "^8.16.0",
    "eslint-config-airbnb": "^19.0.4",
    "eslint-config-prettier": "^8.5.0",
    "eslint-plugin-chai-friendly": "^0.7.2",
    "eslint-plugin-cypress": "^2.12.1",
    "eslint-plugin-import": "^2.26.0",
    "eslint-plugin-jsx-a11y": "^6.5.1",
    "eslint-plugin-prettier": "^4.0.0",
    "eslint-plugin-react": "^7.30.0",
    "husky": "^8.0.1",
    "prettier": "^2.6.2"
  },
  "dependencies": {
    "cypress-file-upload": "^5.0.8",
    "cypress-qase-reporter": "^1.4.2-alpha.2",
    "dotenv": "^16.0.1",
    "prompt": "^1.3.0"
  }
}

Now, go back to Qase app and find out IDs of each test case. It is a number next to project ID (for example, here CWBS is a project ID and 1 is case ID and so forth):

Go back to your automation project and export API key from Qase we received before. This is how you should write it and execute in terminal, but YOUR KEY IS DIFFERENT, PASTE YOUR KEY from step 8 here:

export QASE_API_KEY=46b7d640b6841da28aea575cb6084141661976bcq

Edit each test case to include your IDs from Qase, we are matching documentation with automation here:

addBookToProfile.cy.js:

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { bookActions } from '../../support/bookstore_page_objects/book_store';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Collections: Add Book To Collection', () => {
  // Perform login
  beforeEach('Perform login', () => {
    cy.createUser();
    cy.generateToken();
  });

  // Delete user
  afterEach('Delete user', () => {
    cy.deleteUser();
  });

  qase(
    1,
    it('Check adding book to profile collection', () => {
      // Navigate to book store
      navigateTo.bookStore();
      // Load books fixture
      cy.fixture('books').then((books) => {
        // Add first books to collection
        bookActions.addBookToCollection(books.collection1.Git);
        // Handle alert and verify alert message
        cy.verifyWindowAlertText(`Book added to your collection.`);
        // Navigate to user profile and verify that book is in collection table
        navigateTo.profile();
        cy.get('.rt-tbody').find('.rt-tr-group').first().should('contain', books.collection1.Git);
      });
    })
  );
});

checkBookInfo.cy.js:

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { bookActions } from '../../support/bookstore_page_objects/book_store';
import { profileActions } from '../../support/bookstore_page_objects/profile';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Collections: Check Book Info', () => {
  // Perform login
  beforeEach('Perform login', () => {
    cy.createUser();
    cy.generateToken();
  });

  // Add book to book collection
  beforeEach('Add book to profile collection', () => {
    navigateTo.bookStore();
    cy.fixture('books').then((books) => {
      bookActions.addBookToCollection(books.collection1.DesignPatternsJS);
      cy.verifyWindowAlertText(`Book added to your collection.`);
    });
  });

  // Delete user
  afterEach('Delete user', () => {
    cy.deleteUser();
  });

  qase(
    2,
    it('Check book info from profile table', () => {
      // Navigate to user profile
      navigateTo.profile();
      // Load books fixture
      cy.fixture('books').then((books) => {
        // Click on book in collection to open book info
        profileActions.checkBookData(books.collection1.DesignPatternsJS);
      });
      // Define book info elements
      const bookDataElements = [
        '#ISBN-label',
        '#title-label',
        '#subtitle-label',
        '#author-label',
        '#publisher-label',
        '#pages-label',
        '#description-label',
        '#website-label',
      ];
      // Check book info elements
      cy.elementVisible(bookDataElements);
      // Define data about the book
      const bookData = [
        '9781449331818',
        'Learning JavaScript Design Patterns',
        `A JavaScript and jQuery Developer's Guide`,
        'Addy Osmani',
        `O'Reilly Media`,
        '254',
      ];
      // Check data about the book
      cy.textExists(bookData);
    })
  );
});

deleteBookFromProfile.cy.js:

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { bookActions } from '../../support/bookstore_page_objects/book_store';
import { profileActions } from '../../support/bookstore_page_objects/profile';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Collections: Delete Book From Collection', () => {
  // Perform login
  beforeEach('Perform login', () => {
    cy.createUser();
    cy.generateToken();
  });

  // Add book to collection
  beforeEach('Add book to profile collection', () => {
    navigateTo.bookStore();
    cy.fixture('books').then((books) => {
      bookActions.addBookToCollection(books.collection1.SpeakingJS);
      cy.verifyWindowAlertText(`Book added to your collection.`);
    });
  });

  // Delete user
  afterEach('Delete user', () => {
    cy.deleteUser();
  });

  qase(
    3,
    it('Check deleting book from profile collection - confirm deletion', () => {
      cy.fixture('books').then((books) => {
        // Navigate to user profile
        navigateTo.profile();
        // Check if book is in the collection table
        cy.get('.rt-tbody')
          .find('.rt-tr-group')
          .first()
          .should('contain', books.collection1.SpeakingJS);
        // Delete book from table - confirm deletion
        profileActions.deleteBookFromTable(books.collection1.SpeakingJS, 'ok');
        // Handle delete alert and verify message
        cy.verifyWindowAlertText(`Book deleted.`);
        // Verify that book is no longer in collection table and that table is empty
        cy.get('.rt-tbody').should('not.contain', books.collection1.SpeakingJS);
        cy.get('.rt-noData').should('contain', 'No rows found').should('be.visible');
      });
    })
  );

  qase(
    6,
    it('Check deleting book from profile collection - decline deletion', () => {
      cy.fixture('books').then((books) => {
        // Navigate to user profile
        navigateTo.profile();
        // Check if book is in the collection table
        cy.get('.rt-tbody')
          .find('.rt-tr-group')
          .first()
          .should('contain', books.collection1.SpeakingJS);
        // Cancel book deletion
        profileActions.deleteBookFromTable(books.collection1.SpeakingJS, 'cancel');
        // Verify that book is still in the table
        cy.get('.rt-tbody').should('contain', books.collection1.SpeakingJS);
      });
    })
  );
});

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { auth } from '../../support/bookstore_page_objects/auth';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Auth: Login user', () => {
  // Navigate to login page
  beforeEach('Navigate to Login page', () => {
    navigateTo.login();
  });
  qase(
    7,
    it('Check valid user credentials', () => {
      // Load users fixture
      cy.fixture('users').then((users) => {
        // Perform login
        auth.login(users.user2.username, users.user2.password);
      });
      // Verify that user is redirected to profile page (user is logged in)
      cy.url().should('contain', Cypress.env('profile'));
    })
  );

  qase(
    8,
    it('Check invalid user credentials', () => {
      // Perform login
      auth.login('invalid345', 'invalid345');
      // Verify that user is still on login page (user is not logged in)
      cy.url().should('contain', Cypress.env('login'));
      // Verify that error message is displayed
      cy.get('#output').should('contain', 'Invalid username or password!');
    })
  );

  qase(
    9,
    it('Check login with invalid username and valid password', () => {
      // Load users fixture
      cy.fixture('users').then((users) => {
        // Perform login
        auth.login('invalid345', users.user2.password);
      });
      // Verify that user is still on login page (user is not logged in)
      cy.url().should('contain', Cypress.env('login'));
      // Verify that error message is displayed
      cy.get('#output').should('contain', 'Invalid username or password!');
    })
  );

  qase(
    10,
    it('Check login with valid username and invalid password', () => {
      // Load users fixture
      cy.fixture('users').then((users) => {
        // Perform login
        auth.login(users.user2.username, 'invalid345');
      });
      // Verify that user is still on login page (user is not logged in)
      cy.url().should('contain', Cypress.env('login'));
      // Verify that error message is displayed
      cy.get('#output').should('contain', 'Invalid username or password!');
    })
  );
});

logout.cy.js:

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { auth } from '../../support/bookstore_page_objects/auth';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Auth: Log out user', () => {
  // Perform login
  beforeEach('Perform login', () => {
    cy.createUser();
    cy.generateToken();
  });

  // Delete user
  afterEach('Delete user', () => {
    cy.deleteUser();
  });

  qase(
    11,
    it('Check logging out user', () => {
      // Navigate to user profile
      navigateTo.profile();
      // Perform log out
      auth.logout();
      // Assert that user is on login page
      cy.url().should('contain', Cypress.env('login'));
    })
  );
});

searchBookstore.cy.js:

/// <reference types="Cypress" />

import { qase } from 'cypress-qase-reporter/dist/mocha';
import { bookActions } from '../../support/bookstore_page_objects/book_store';
import { navigateTo } from '../../support/bookstore_page_objects/navigation';

describe('Bookstore: Search For Book', () => {
  // Perform login
  beforeEach('Perform login', () => {
    cy.createUser();
    cy.generateToken();
  });

  // Delete user
  afterEach('Delete user', () => {
    cy.deleteUser();
  });

  qase(
    12,
    it('Check searching for existing book in book store', () => {
      // Navigate to bookstore
      navigateTo.bookStore();
      // Load books fixture
      cy.fixture('books').then((books) => {
        // Perform book search
        bookActions.searchCollection(books.collection1.DesignPatternsJS);
        // Verify that there is a book in filtered table (in search result)
        cy.get('.rt-tbody')
          .find('.rt-tr-group')
          .first()
          .should('contain', books.collection1.DesignPatternsJS);
      });
    })
  );

  qase(
    13,
    it('Check searching for non-existing book in book store', () => {
      // Define invalid book name
      const invalid_book_name = 'Game of Thrones';
      // Navigate to bookstore
      navigateTo.bookStore();
      // Perform book search
      bookActions.searchCollection(invalid_book_name);
      // Assert that there are no search results (no book in the table and table is empty)
      cy.get('.rt-tbody').should('not.contain', invalid_book_name);
      cy.get('.rt-noData').should('contain', 'No rows found').should('be.visible');
    })
  );
});

Finally let’s run the tests. In your terminal write execute command

npm run cypress:run:qase

and answer questions, provide project ID, run ID and suite you want to run, in this case bookstore suite.

You can find run ID only in URL as last number when you navigate to your test run page:

Once the test run is completed, you will be able to see results in terminal and also in Qase app just refresh test run page in Qase.

As you can see all test cases results are uploaded from Cypress to Qase. We have only one manual test case left to be executed and you can execute it manually in Qase itself. That we we combined automated and manual test execution.

Github Actions

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.

GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository. For example, you can run a workflow to automatically add the appropriate labels whenever someone creates a new issue in your repository.

GitHub provides Linux, Windows, and macOS virtual machines to run your workflows, or you can host your own self-hosted runners in your own data center or cloud infrastructure.

ℹ️ Learn more about Github Actions: Github Actions

ℹ️ Github Actions + Cypress: GA and Cypress

You can customize your CI/CD pipelines according to your project needs. In this demo we will create an example github actions which will allow us to do few simple actions:

Schedule cron job to run cypress tests at 10am UTC every Sunday
Run cypress tests on each push to main branch

Cypress test run job will consist of following:

Defining container and machine where tests will run
Installing dependencies
Running all our tests in default headless mode on Chrom

So, how do we write this?

Create a new folder .github in the root directory.
Create a new folder workflows , under .github
Create a new file main.yml

Write following content inside (best to copy paste because this file is sensitive to spaces etc.):

name: Cypress Tests

on:
  schedule:
  #schedule at 10:00 on Sunday
  - cron: '0 10 * * sun'
  push:
    branches:
      - main
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Install dependencies
        run: npm install    

      - name: 'Run all tests'
        uses: cypress-io/github-action@v4
        with:
          # we have already installed all dependencies above
          install: false
          wait-on: 'https://demoqa.com'
          wait-on-timeout: 120
          browser: chrome
          spec: cypress/e2e/**/*

Once you push this file to your remote github repository, github will immediately start a test run. You can see it in my cypress presentation repo that contains all work from previous lessons: link

You can see your test runs github performed under actions. When you open certain flow you will see its log and all of the output of the test run.

Above is the example of the scheduled run I set up to start each Sunday at 10am UTC.

Don’t forget to push everything you did today on Github 😉 Remember git commands?

git add .

git commit -am "add: qase and github actions support"

git push

Thank you for following this workshop. I hope it was useful. For any questions, you know where to find me: My LinkedIn

Completed code for this lesson

If you have learned something new, feel free to support my work by buying me a coffee ☕

Top comments (2)

Matheus-Fonseca • Aug 24 '23

Hello, can u help me please, im having this issue when i'm gonna star the run command

in English means " 'QASE_REPORT' is not recognized as an internal command
or external, an operable program or a batch file."

and i think i setup everything correctly

instaled

npm install cypress-qase-reporter

npm install dotenv

npm install prompt

have any idea whats wrong ?

Viktorija Filipov • Sep 4 '23 • Edited

Hey! Thanks for the question.

Just few things to check, did you execute this command?

Did you install all packages? (Check my package.json file)

Also, I'm not sure that cypress.config.js you wrote is working fine, it is different from mine.

The purpose of that script cypress-with-qase.js is that when you run it, you write your project code etc in terminal. It is not stored anywhere in the code in my version, so it doesn't happen automatically (I guess what you are trying is to read automatically credentials from config file). So basically, you execute that script, it will ask you for project key, you write it in terminal, and answer all other questions prompt asks you (result.PROJECT_ID, etc.).

I hope I helped a bit, let me know. :)

DEV Community

Cypress Workshop Part 11: Reporting - Qase, Github Actions

Qase

Github Actions

Top comments (2)

Read next

Set Git to Recognize Case Changes

Mind Map of the Software Testing Ecosystem

Cypress Cheat Sheet

What is DOM (Document Object Modle) in JavaScript