Hello Everyone, I hope you all are rocking in your digital world. So, Here is my first blog, I will be sharing my experience and what i learned while working on end-to-end testing for Litmus-Portal. we will be going through how to get started with End-to-End testing with Cypress in any of your project and how litmus-Portal is using Cypress for testing different scenarios.Stay tuned till the end, you will get to know many awesome things.
Litmus-Portal provides console and UI experience for managing, monitoring, and events around chaos workflows. Chaos workflows consist of a sequence of experiments run together to achieve the objective of introducing some kind of fault into an application or the Kubernetes platform.Using Litmus-Portal, you can make your projects or products more resilient.
For doing all this and providing resiliency to your product, Litmus-Portal has to be resilient itself.That's where Cypress comes which helps us to test Litmus-Portal in different scenarios and makes it more resilient.
Cypress
Cypress is a modern frontend End-to-End testing tool using which we can write our tests in javascript as well as Typescript. It simplifies how we write our tests, makes our tests less flaky, and helps us in reducing maintenance cost of our project.
Why Cypress?
Well, we could have used some other framework for our purpose but, we wanted the one, which is easy to setup and reliable.There are many advantages of using Cypress -
- Easy to setup, documentation is more than sufficient.
- It helps us in adapting best practices for testing with it's documentation.
- As Cypress shows all logs side by side to AUT (Application Under Test), it is very easy to debug our projects.
- A plugins catalog provided by Cypress and it's community, which is very helpful to test different scenarios.
- It is very easy to adapt, as it build on top of Mocha, Chai, chai-jQuery and many other libraries.
Installing Cypress
Cypress is an NPM package. We can install Cypress as a development dependency like
npm install cypress --save-dev
We can use Cypress in two modes -
- Browser mode
For using Cypress in Browser Mode, we can make use of this command -
npx cypress open
This will open a browser for you, showing different default-test scripts. We can click on different scripts to execute them.
- Headless mode
For using Cypress in Headless Mode, we can make use of this command -
npx cypress run
This will open a terminal for you, and start to execute the tests present in the Test scripts path (By default, Integration directory).
After executing this command,you will observe that some predefined directories and files have been added in your project -
-cypress
|_fixtures
|_integration
|_plugins
|_support
-cypress.json
Here, cypress is the directory that contains everything required for testing using Cypress.
- fixtures/ - This directory contains all the static data (data you want to use for setting your Databases between tests or you want to input on your different screens) to be used while testing in form of JSON files.
An example of User.json (fixture file for users) is given below -
{
"projectname":"litmus",
"AdminName":"John",
"AdminPassword":"admin1234",
"AdminEmail":"admin@gmail.com",
"NewName":"John",
"NewPassword":"John123",
"NewEmail":"John@gmail.com"
}
integration/ - This directory contains all the test scripts. We can configure a different location for storing our test scripts in cypress.json.
plugins/index.js - This file can contain configuration for all plugins installed.
support/commands.js - It will contain all the custom functions that we might need while writing our test so that we don't repeat ourselves.
support/index.js - This file contains any configurations for test cases. for e.g. by default cookies are not preserved between tests. They can be preserved add following code in this file -
// For preserving cookies between tests.
Cypress.Cookies.defaults({
preserve: 'token'
});
cypress.json - This is a configuration file for Cypress.
An example of cypress.json is given below -
{
"baseUrl": "http://localhost:3001",
"experimentalFetchPolyfill": true,
"viewportWidth": 1800,
"viewportHeight": 1200,
"defaultCommandTimeout": 10000,
"chromeWebSecurity": false,
"video": false,
"experimentalNetworkStubbing":true,
"env": {
"authURL" : "http://localhost:3000",
}
}
You will get to know more about them, as we proceed with testing different scenarios.
Let's have something in our pocket before moving on further -
DOM Selector - It is a selector that is used for selecting different objects in DOM for testing or automation. A Selector can be any CSS property, Ids, and Classes. But let me tell you when you make a product your CSS properties, ids, and classes keep changing, which might break our tests.
The best practice is to use a unique identifier which is not much subjected to change also which is unique on a particular page for identifying an element. While using Cypress, we have the support to use data-*
attributes with our elements.
As a best-practice. we enforce everyone working on Litmus-Portal to use data-cy=<selector>
selector to every different component, so that it can targeted easily by Cypress while testing. This practice is also preferred by Cypress test runner
In Cypress, We can query an element on DOM using the command cy.get(<Your_Selector>)
In Cypress, data-* are given high priority while querying, so it is a bit fast as well.
for example, If there is button like this,
<button class="btn1">Save</button>
We can inject a unique identifier like this
<button class="btn1" data-cy="save">Save</button>
Now, we can access this button like this
cy.get('[data-cy=save]');
Okay, now I think we are good with testing and Cypress, we will go deep while working with live scripts, but let's get our hands dirty a bit.
Firstly, for testing an App, We need an endpoint of our WebApp.
Let's say the endpoint of our web app is https://localhost:3001
In Cypress, We can visit this link by using the function visit()
cy.visit("https://localhost:3001");
But being a lazy person, we don't want to write this bigger link again and again in every test or even in different test scripts.
You will be happy to know, we can also set the endpoint in cypress.json for universal use in test scripts.
In cypress.json,
{
"baseUrl":"https://localhost:3001"
}
Now, anywhere in the test script, we want to visit the link, we can just do
cy.visit("/");
Well, this setup will work well in local setup. But when we are working on different CI's, we won't be able to use it because every time we setup the full stack web app in CI, A dynamic Link will be getting generated.
As Litmus-Portal is a cloud native web application, we have to deploy it on kubernetes while testing on different CI's. Everytime we generate a new dynamic link using loadbalancer for accessing the frontend.So, for this we needed a better approach as we can't provide access link before deploying Litmus-Portal.
But Hurray, I have something for you, We can provide the link as an environment variable to cypress while starting testing using command -
CYPRESS_BASE_URL=$link npx cypress run
So, Cypress will use this URL as BaseURL while executing our test scripts.
Now, as we know how to query an element and how to open our web app to be tested, the Next thing is how we write tests for our app.
We will be using Litmus-portal as our web app for Testing.
Starting with login Page for Litmus-Portal.
While writing tests for the Login Page, we have to consider all scenarios including positive and negative tests.
A Positive scenario can be like this -
- Visit the login page.
- Find the input for the name and type our correct name in it.
- Find the input for the password and type our correct password in it.
- Click on the login button.
- Check if we are landing on welcome modal after clicking on the login button.
A Negative scenario can be like this -
- Visit the login page.
- Find the input for the name and type a wrong name in it.
- Find the input for the password and type a wrong password in it.
- Click on the login button.
- Check if we are prompted with the error "Wrong Credentials".
Let me give you a small script for login page testing,
This was what I wrote on my first try. So don't laugh.
describe("Checking functionality of Login Page",()=>{
it("Testing the only single input sign in [ Should not be possible ]",()=>{
cy.visit("/");
cy.get('[data-cy=inputName] input').type(" ");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click();
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with wrong details [ Should not be possible ]",()=>{
cy.visit("/");
cy.get('[data-cy=inputName] input').type("Johnce");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click();
cy.url().should('include','/login');
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
cy.visit("/");
cy.get('[data-cy=inputName] input').type("John");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click(); //Correct Details
cy.contains("Welcome to Portal");
})
})
If you have worked with Mocha before, then you must be knowing that it's BDD interface provide describe(), it(), beforeEach(), afterEach() and many other functions. If you don't, don't worry we will discuss all of them as we use them.
Here, describe() is mainly used for collecting all tests under a single umbrella (Test-suite) which are related to each other or similar in objective.
it() is the function which signifies one test or one scenario.
Also, you must be seeing some new interactive functions as well.
Let me explain what we are doing here.
cy.visit("/");
visits the login page.
cy.get("[data-cy=inputName]")
finds the input field for name andtype(<data_to_be_typed>)
is used to type in the selected input field.
cy.contains(<text>)
is used to search text on page.
You must be seeing that we are visiting the login page again and again and also writing the same functions many times.
Let's refactor it a bit with one more BDD function i.e. beforeEach().
beforeEach() is used when we want to do something before every test scenario or if something is common before every scenario.
describe("Checking functionality of Login Page",()=>{
beforeEach(Visiting the login Page,()=>{
cy.visit("/");
});
it("Testing the only single input sign in [ Should not be possible ]",()=>{
cy.get('[data-cy=inputName] input').type(" ");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click();
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with wrong details [ Should not be possible ]",()=>{
cy.get('[data-cy=inputName] input').type("Johnce");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click();
cy.url().should('include','/login');
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
cy.get('[data-cy=inputName] input').type("John");
cy.get('[data-cy=inputPassword] input').type("John123");
cy.get('[data-cy=loginButton]').click(); //Correct Details
cy.contains("Welcome to Portal");
})
})
One problem is solved, but we are still writing functions for logging in many times which are the same, just the value provided is different.
So here, we take the help of the Custom Commands facility provided by Cypress.
Now, we will create a custom function which will take username and password as arguments and login the user. We can add that function in commands.js inside Support directory.
In your support/commands.js file inside the support folder,
// Custom login function for logging In which takes username and password as parameters.
Cypress.Commands.add('login',(Username,Password)=>{
cy.get('[data-cy=inputName] input').type(Username);
cy.get('[data-cy=inputPassword] input').type(Password);
cy.get('[data-cy=loginButton]').click();
})
Your test script will look like this,
// Here in the script, we can just call the custom login function that we made just by using cy.login(username, password).
describe("Checking functionality of Login Page",()=>{
beforeEach(Visiting the login Page,()=>{
cy.visit("/");
});
it("Testing the only single input sign in [ Should not be possible ]",()=>{
cy.login("John"," ");
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with wrong details [ Should not be possible ]",()=>{
cy.login("Vedant","1234");
cy.url().should('include','/login');
cy.contains("Wrong Credentials").should('be.visible');
})
it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
cy.login("John","John123");
cy.contains("Welcome to Portal");
})
})
Currently, the above script will work fine if we are testing locally, but when we work on a production server or CI, There might be delays in response from backend server, Cypress might get timed out waiting for homepage to load.
For dealing with this situation, we can use one command i.e,
cy.wait(8000);
Here, cy.wait() will wait make the test to wait for constant time given as argument to wait() function.
But this will make our test slower when we wait for constant time.
So, the better approach here is to use aliases for waiting for request to get resolved.
Here is an example -
// Custom login function for logging In which takes username and password as parameters and also waits for data from server.
Cypress.Commands.add('login',(Username,Password)=>{
cy.server();
cy.route("POST", Cypress.env('apiURL')+"/query").as("detailsResponse");
cy.get('[data-cy=inputName] input').type(Username);
cy.get('[data-cy=inputPassword] input').type(Password);
cy.get('[data-cy=loginButton]').click();
cy.wait("@detailsResponse").its("status").should("eq", 200); //Request Done.
})
In above example, cy.server()
will start a mock server and will be intercepting all requests from frontend. cy.route()
will tell Cypress to intercept a request going on a particular route. We can mock response, status and many other parameters while intercepting a request using cy.route()
.
Now, for waiting for a request to get resolved, we have to make alias for that route using as()
. as()
makes alias of any route with given name, which the Cypress will remember for us.
cy.route("POST", Cypress.env('apiURL')+"/query").as("detailsResponse");
Now, we can wait for this request using cy.wait()
by giving alias name to it and checking it's status property using its()
function after executing our steps.
cy.wait("@detailsResponse").its("status").should("eq", 200); //Request Done.
Thanks for staying with me till here, In my next article, we will be discussing more about how to test other scenarios that we face in a project.Till then, you can always check their documentation here
If you are interested in learning more about different test scenarios in Litmus-Portal, Checkout our Litmus-Portal-E2E Repository here
Conclusion
Feel free to check out our ongoing project - Litmus Portal and do let us know if you have any suggestions or feedback regarding the same. You can always submit a PR if you find any required changes.
Make sure to reach out to us if you have any feedback or queries. Hope you found the blog informative!
If chaos engineering is something that excites you or if you want to know more about cloud-native chaos engineering, donโt forget to check out our Litmus website, ChaosHub, and the Litmus repo. Do leave a star if you find it insightful. ๐
I would love to invite you to our community to stay connected with us and get your Chaos Engineering doubts cleared.
To join our slack please follow the following steps!
Step 1: Join the Kubernetes slack using the following link: https://slack.k8s.io/
Step 2: Join the #litmus channel on the Kubernetes slack or use this link after joining the Kubernetes slack: https://slack.litmuschaos.io/
Cheers!
litmuschaos / litmus
Litmus helps SREs and developers practice chaos engineering in a Cloud-native way. Chaos experiments are published at the ChaosHub (https://hub.litmuschaos.io). Community notes is at https://hackmd.io/a4Zu_sH4TZGeih-xCimi3Q
Open Source Chaos Engineering Platform
Read this in other languages.
๐ฐ๐ท ๐จ๐ณ ๐ง๐ท ๐ฎ๐ณ
Overview
LitmusChaos is an open source Chaos Engineering platform that enables teams to identify weaknesses & potential outages in infrastructures by inducing chaos tests in a controlled way. Developers & SREs can practice Chaos Engineering with LitmusChaos as it is easy to use, based on modern Chaos Engineering principles & community collaborated. It is 100% open source & a CNCF project.
LitmusChaos takes a cloud-native approach to create, manage and monitor chaos. The platform itself runs as a set of microservices and uses Kubernetes custom resources (CRs) to define the chaos intent, as well as the steady state hypothesis.
At a high-level, Litmus comprises of:
- Chaos Control Plane: A centralized chaos management tool called chaos-center, which helps construct, schedule and visualize Litmus chaos workflows
- Chaos Execution Plane Services: Made up of aโฆ
Top comments (2)
One thing is missing, implementation of fixtures/
It could be more useful and clear to readers.. ๐
Thanks @sanampakuwal for the feedback!!.
Yes, I will be covering more over the fixtures & the use cases in coming posts along with my long experience.