Collaboration is pivotal for any successful release. Can you imagine going through a sprint without consulting or informing any other team involved in the project about what you did? You can’t right because it is not a pretty picture. Modern SDLCs demand various teams to coordinate as they try to deliver a product as quickly as possible in the market, with assured quality.
Your job as an automation tester doesn’t end up by merely running automation scripts for your web application, you also have to share the test results across different teams such as the development team, or business analysts. Generating and presenting reports requires a lot of effort as your reports need to clearly indicate which modules are working fine and which ones are failing.
This week I added a new tool in my testing checklist to help me share across the results of my Selenium automation testing across my team as I performed automated cross browser testing, and the tool is Calliope. In this article, I am going to help you integrate your LambdaTest account with Calliope, so you could share reports of test automation scripts executed at LambdaTest Selenium Grid across your teammates in a jiffy. Let’s start with the big question.
What Is Calliope?
Calliope is cloud-based tool providing you with a collective dashboard for sharing, and monitoring, and comparing the results of your automation script execution. Offering compatibility with major test automation frameworks, Calliope enables your organization to have a unified view of everything that is going around with your automation test scripts. Your stakeholders can analyze it too for comparing the current state of your test cycles with a historic state.
Calliope ensures that your entire team is on the same page when it comes to analyzing your test results. Here is how:
- Sharing your test results – Your test results are presented in Gherkin syntax, making it easier to understand by your stockholders as well.
- Customize your own dashboard – Invite your colleagues and structure the dashboard in a manner as you structure your team.
- Compare Historical Data – You can compare the current health status of your test builds with historical status. Plus, all your test cases are gathered in a single region allowing even your non-technical stakeholders with a clear picture of the overall health for your automation scripts.
- Easy Regression Check – Calliope stores all your test imports on their clouds making it easier for you to reflect back anytime for validating your regression test cycles.
- API integration – Calliope API will help you run your test suites on third-party tools instantly through the test results dashboard presented in your Calliope instance.
- CI/CD Integration with GitLab – Like LambdaTest, Calliope also offers integration with GitLab CI to help you import your test cases directly from your CI/CD pipeline to Calliope instance.
Now, that we have an idea about the various features offered by Calliope. It is about time we get to know about integrating LambdaTest with Calliope. However, to those of us who are not aware of LambdaTest.
What Is LambdaTest?
We offer a cloud-based cross browser testing platform to help you perform browser compatibility testing by both manual and automation testing with Selenium. LambdaTest offers a Selenium Grid of 2000+ real browsers for both mobile and desktop, running on real operating systems.
Perform Cross Browser Testing with LambdaTest on 2000+ Browsers With A Free Sign-up.
Why Should You Integrate LambdaTest With Calliope?
If not well-orchestrated, cross browser testing can turn out to be a mess. You will require a thorough plan and strategy, design a cross browser compatibility matrix to figure out which browsers are of higher priority and which ones are of the lowest. Depending on the amount of test coverage, and time in-hand, you will need to make a decision to opt for automated cross browser testing. To pace things even faster you would require parallel execution with Selenium, by which you can run multiple test cases simultaneously. And even after following up the right plan, you would need a testing dashboard where you can club all of your test cases in one place, for everyone to have a clear and concise view. This is where LambdaTest integration with Calliope comes into the picture.
Using LambdaTest integration with Calliope you can analyze the test results of automation test scripts being executed in parallel on more than 2000 browsers + OS combinations. Let’s get started.
Import LambdaTest Automation Test Results Into Calliope Dashboard
So, now you are aware of LambdaTest and Calliope’s services, let’s have a look at how you can import your LambdaTest automation test results directly to Calliope dashboard. For fetching LambdaTest automation test results details from scratch we need to call LambdaTest API which provides us our recently executed test session details such as name, status, os, browser, version and all generated logs endpoint. We will be making use of the below API:
Get/sessions/{session_id}
It will provide us with information specific to our test sessions. This would require you to give the session ID for the test session you want to retrieve the details for. You can refer to LambdaTest API Documentation to check the example value or schema.
Below is the test session details that would be fetched through LambdaTest API and will get imported to Calliope dashboard.
{
data:{
name=Calliope-Sample-Test,
duration=24,
platform=win10,
browser=chrome,
browserVersion=74.0,
device=,
statusInd=running,
sessionId=ca7e3b1df8ddc533fc5394141601431f,
buildName=LambdaTest Integration with Calliope,
consoleLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/console,
networkLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/network,
commandLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/command,
seleniumLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/selenium,
videoUrl=https://automation.lambdatest.com/public/video?testID=3K6C3-HBAMX-BUS4B-FWY5Z,
screenshotUrl=https://d15x9hjibri3lt.cloudfront.net/3K6C3-HBAMX-BUS4B-FWY5Z/screenshots.zip
},
message:Retrieve session was successful,
status:success
}
You might be wondering how you can send the above test session details directly to Calliope dashboard after retrieving from LambdaTest API. So, here is the Java-TestNG code for you that can help you setting up the test environment which further includes your username, accesskey, gridURL and test configurations such as browser, browser version, OS etc.
Once you have set the test environment, you can now write your test cases. We have been using SessionId Java class that provides the session ID for running session, we can use the same session ID for calling LambdaTest GET Session API whose URL is:
https://api.lambdatest.com/automation/api/v1/sessions/
For accessing the LambdaTest API, you need to get authorized with your credentials that include your username and access key. We have used Base64 class to encode the credentials which are universally accepted format by web servers and browsers.
Now, we can use our encoded basic authorization with session URL to send the GET request for the test session details.
Since, we are using Java-TestNG here, all our test results and logs get saved in testng-results.xml file. The purpose of this file is to capture the test session data and can further be used with Calliope API in order to transfer the saved test session data to Calliope Dashboard. In the below code, we have also used Reporter.log which would report the test logs and all other session details to the testng-results.xml file.
BaseTest.java
package calliopeIntegration;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Base64;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.SessionId;
import org.testng.Reporter;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import calliopeIntegration.session.SessionResponse;
@Listeners({IntegrationExecution.class})
public class BaseTest {
public String username = "Your_Username"; //LambdaTest Username
public String accesskey = "Your_AccessKey; //LambdaTest AccessKey
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub"; //GridURL
boolean status = false;
@BeforeMethod
public void setUp() {
DesiredCapabilities capabilities = new DesiredCapabilities(); // Setting configurations
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "74.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "LambdaTest Integration with Calliope");
capabilities.setCapability("name", "Calliope-Sample-Test");
capabilities.setCapability("network", true); // To enable network logs
capabilities.setCapability("visual", true); // To enable step by step screenshot
capabilities.setCapability("video", true); // To enable video recording
capabilities.setCapability("console", true); // To capture console logs
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Build and Session Information retrieved from LambdaTest API's",true);
}
// Test cases
@Test
public void BuildSession() throws Exception {
try {
driver.get("https://www.apple.com/");
driver.manage().window().maximize();
Thread.sleep(5000);
driver.findElement(By.xpath("//*[@id=\'ac-globalnav\']/div/ul[2]/li[3]")).click();
Thread.sleep(2000);
driver.findElement(
By.cssSelector("#chapternav > div > ul > li.chapternav-item.chapternav-item-ipad-air > a")).click();
Thread.sleep(2000);
driver.findElement(By.linkText("Why iPad")).click();
Thread.sleep(2000);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
@AfterMethod
public void teardown() throws Exception {
SessionId session = driver.getSessionId();
String sessionId = session.toString();
if (driver != null) {
((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
driver.quit();
}
String usernameColonPassword = username+":"+accesskey ; // API Authorization
String basicAuthPayload = "Basic " + Base64.getEncoder().encodeToString(usernameColonPassword.getBytes());
Reporter.log(basicAuthPayload);
//Session Info
String sessionURL = "https://api.lambdatest.com/automation/api/v1/sessions/"+sessionId;
Reporter.log(sessionURL,true);
String jsonResponse2 = sendGetRequest(sessionURL,basicAuthPayload);
ObjectMapper objectMapper2 = new ObjectMapper();
objectMapper2.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
SessionResponse response2 = objectMapper2.readValue(jsonResponse2, SessionResponse.class);
Reporter.log(response2.toString(),true);
}
public static String sendGetRequest(String url, String basicAuthPayload) throws Exception {
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
// add GET request header
request.addHeader("Content-Type", "application/json");
request.addHeader("Authorization", basicAuthPayload);
HttpResponse response = client.execute(request);
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null)
{
result.append(line);
}
return result.toString();
}
}
Now, after executing the above code, we have our test executed on LambdaTest Selenium Grid along with the test testng-results.xml file saved in our system which includes all our recent test session details.
The next step is to call Calliope API to post the test data from testng-results.xml file to Calliope dashboard. Calliope support different result file format according to different frameworks. For instance, XML for Junit and TestNG, JSON for Cucumber. For more information on this, you can refer to Calliope’s documentation link.
We have used MediaType class to define the file type(XML) for our TestNG framework. Now, we need a testng-results.xml file to GET and POST test session data. So, we need to define the directory where this file is saved to access the file by calling the Calliope API.
We have defined API command as an endpoint URL which includes your profile number along with OS, platform and build name. You can extract your Calliope API from their documentation for API import.
Make sure you have entered the correct API Key, otherwise, your test would fail due to unmatch of API Key with your profile number.
After setting up the endpoint URL and the API Key, it’s time to call Calliope API now by sending a POST request for importing test results to the dashboard.
CalliopeAPI.java
package calliopeIntegration;
import java.io.File;
import java.io.IOException;
import org.testng.annotations.Test;
import okhttp3.MediaType;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
public class CalliopeAPI
{
@Test
public void calliopeAPIcall()
{
final MediaType MEDIA_TYPE_XML = MediaType.parse("application/xml"); // for xml files
final MediaType MEDIA_TYPE_JSON = MediaType.parse("application/json"); // for json files
MultipartBody requestBody = null;
try {
String report_filename = "C:\\Users\\Lenovo-I7\\git\\Calliope-Integration-with-LambdaTest\\CalliopeSample\\test-output\\testng-results.xml"; // Result File Address
requestBody = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("file", report_filename, RequestBody.create(MEDIA_TYPE_XML, new File(report_filename)))
.build();
} catch (Exception e1) {
e1.printStackTrace();
}
OkHttpClient client = new OkHttpClient();
String endpoint_url = "https://app.calliope.pro/api/v2/profile/577/report/import?os=myos&platform=myplatform&build=mybuild"; // Calliope API
final String API_KEY = "ZDk4ZGVhM2VlMzRlYjlkZGI0Y2MxZTA4Yjg1OTYxNjUyMzQzMGZhZmE0NTY0MTk4Y2MyMmM0NGQ3OTlmNTk2N2Jm"; //Calliope AccessKey
Request request = new Request.Builder().url(endpoint_url).post(requestBody).addHeader("x-api-key", API_KEY).build(); // POST request
Response response = null;
try {
response = client.newCall(request).execute();
System.out.println("=============");
System.out.println(response);
String response_body = response.body().string();
System.out.println(response_body);
System.out.println("=============");
if (response.isSuccessful()){
System.out.println("created");
} else {
throw new IOException("Unexpected HTTP code " + response);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
To make sure that CalliopeAPI.java file is getting executed after BaseTest.java file finishes its execution, we have used TestNG Listeners for this. Just for the recap, the BaseTest.java file includes the test configurations, test cases and the procedure for calling the LambdaTest GET session API. So, unless the execution of BaseTest.java doesn’t end up we cannot call CalliopeAPI.java, this is the reason that we have used IExecutionListener which provides two methods and onExecutionFinish().
- Method onExecutionStart() gets invoked before TestNG run starts.
- Method onExecutionFinish() gets invoked when all the suites have been run.
IntegrationExecution.java
package calliopeIntegration;
import org.testng.IExecutionListener;
public class IntegrationExecution implements IExecutionListener {
public void onExecutionStart() {
System.out.println("Fetching LambdaTest Automation Test Data and Sending to Calliope Dashboard");
}
public void onExecutionFinish() {
CalliopeAPI object = new CalliopeAPI();
object.calliopeAPIcall(); // Calling Calliope API after test execution on LambdaTest
}
}
Since we are retrieving data from LambdaTest API whose schema is in JSON format, so we need to set up and organize the test session retrieving data using JsonPropertyOrder.
Below is the code which would show you the hierarchy or list in which the test session data would get organized.
SessionData.java
package calliopeIntegration.session;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonPropertyOrder({
"name",
"duration",
"platform",
"browser",
"browser_version",
"device",
"status_ind",
"session_id",
"build_name",
"console_logs_url",
"network_logs_url",
"command_logs_url",
"selenium_logs_url",
"video_url",
"screenshot_url"
})
public class SessionData {
@JsonProperty("name")
private String name;
@JsonProperty("duration")
private Integer duration;
@JsonProperty("platform")
private String platform;
@JsonProperty("browser")
private String browser;
@JsonProperty("browser_version")
private String browserVersion;
@JsonProperty("device")
private String device;
@JsonProperty("status_ind")
private String statusInd;
@JsonProperty("session_id")
private String sessionId;
@JsonProperty("build_name")
private String buildName;
@JsonProperty("console_logs_url")
private String consoleLogsUrl;
@JsonProperty("network_logs_url")
private String networkLogsUrl;
@JsonProperty("command_logs_url")
private String commandLogsUrl;
@JsonProperty("selenium_logs_url")
private String seleniumLogsUrl;
@JsonProperty("video_url")
private String videoUrl;
@JsonProperty("screenshot_url")
private String screenshotUrl;
@JsonProperty("name")
public String getName() {
return name;
}
@JsonProperty("name")
public void setName(String name) {
this.name = name;
}
@JsonProperty("duration")
public Integer getDuration() {
return duration;
}
@JsonProperty("duration")
public void setDuration(Integer duration) {
this.duration = duration;
}
@JsonProperty("platform")
public String getPlatform() {
return platform;
}
@JsonProperty("platform")
public void setPlatform(String platform) {
this.platform = platform;
}
@JsonProperty("browser")
public String getBrowser() {
return browser;
}
@JsonProperty("browser")
public void setBrowser(String browser) {
this.browser = browser;
}
@JsonProperty("browser_version")
public String getBrowserVersion() {
return browserVersion;
}
@JsonProperty("browser_version")
public void setBrowserVersion(String browserVersion) {
this.browserVersion = browserVersion;
}
@JsonProperty("device")
public String getDevice() {
return device;
}
@JsonProperty("device")
public void setDevice(String device) {
this.device = device;
}
@JsonProperty("status_ind")
public String getStatusInd() {
return statusInd;
}
@JsonProperty("status_ind")
public void setStatusInd(String statusInd) {
this.statusInd = statusInd;
}
@JsonProperty("session_id")
public String getSessionId() {
return sessionId;
}
@JsonProperty("session_id")
public void setSessionId(String sessionId) {
this.sessionId = sessionId;
}
@JsonProperty("build_name")
public String getBuildName() {
return buildName;
}
@JsonProperty("build_name")
public void setBuildName(String buildName) {
this.buildName = buildName;
}
@JsonProperty("console_logs_url")
public String getConsoleLogsUrl() {
return consoleLogsUrl;
}
@JsonProperty("console_logs_url")
public void setConsoleLogsUrl(String consoleLogsUrl) {
this.consoleLogsUrl = consoleLogsUrl;
}
@JsonProperty("network_logs_url")
public String getNetworkLogsUrl() {
return networkLogsUrl;
}
@JsonProperty("network_logs_url")
public void setNetworkLogsUrl(String networkLogsUrl) {
this.networkLogsUrl = networkLogsUrl;
}
@JsonProperty("command_logs_url")
public String getCommandLogsUrl() {
return commandLogsUrl;
}
@JsonProperty("command_logs_url")
public void setCommandLogsUrl(String commandLogsUrl) {
this.commandLogsUrl = commandLogsUrl;
}
@JsonProperty("selenium_logs_url")
public String getSeleniumLogsUrl() {
return seleniumLogsUrl;
}
@JsonProperty("selenium_logs_url")
public void setSeleniumLogsUrl(String seleniumLogsUrl) {
this.seleniumLogsUrl = seleniumLogsUrl;
}
@JsonProperty("video_url")
public String getVideoUrl() {
return videoUrl;
}
@JsonProperty("video_url")
public void setVideoUrl(String videoUrl) {
this.videoUrl = videoUrl;
}
@JsonProperty("screenshot_url")
public String getScreenshotUrl() {
return screenshotUrl;
}
@JsonProperty("screenshot_url")
public void setScreenshotUrl(String screenshotUrl) {
this.screenshotUrl = screenshotUrl;
}
@Override
public String toString() {
StringBuilder builder = new StringBuilder();
builder.append("{\nname=");
builder.append(name);
builder.append(",\nduration=");
builder.append(duration);
builder.append(",\nplatform=");
builder.append(platform);
builder.append(",\nbrowser=");
builder.append(browser);
builder.append(",\nbrowserVersion=");
builder.append(browserVersion);
builder.append(",\ndevice=");
builder.append(device);
builder.append(",\nstatusInd=");
builder.append(statusInd);
builder.append(",\nsessionId=");
builder.append(sessionId);
builder.append(",\nbuildName=");
builder.append(buildName);
builder.append(",\nconsoleLogsUrl=");
builder.append(consoleLogsUrl);
builder.append(",\nnetworkLogsUrl=");
builder.append(networkLogsUrl);
builder.append(",\ncommandLogsUrl=");
builder.append(commandLogsUrl);
builder.append(",\nseleniumLogsUrl=");
builder.append(seleniumLogsUrl);
builder.append(",\nvideoUrl=");
builder.append(videoUrl);
builder.append(",\nscreenshotUrl=");
builder.append(screenshotUrl);
builder.append("\n}");
return builder.toString();
}
}
In SessionData.java only session data variables are declared, though they are defined in a hierarchical manner but they do not contain any values yet. We have now created a new class SessionResponse.java that would set values to the previously defined session data variables.SessionResponse.java class has been called by BaseTest.java class for setting the values for session data variables after retrieving from LambdaTest GET session API.
SessionResponse.java
package calliopeIntegration.session;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
@JsonInclude(JsonInclude.Include.NON_NULL)
public class SessionResponse {
@JsonProperty("data")
private SessionData sessionData;
@JsonProperty("message")
private String message;
@JsonProperty("status")
private String status;
@JsonProperty("data")
public SessionData getData() {
return sessionData;
}
@JsonProperty("data")
public void setData(SessionData sessionData) {
this.sessionData = sessionData;
}
@JsonProperty("message")
public String getMessage() {
return message;
}
@JsonProperty("message")
public void setMessage(String message) {
this.message = message;
}
@JsonProperty("status")
public String getStatus() {
return status;
}
@JsonProperty("status")
public void setStatus(String status) {
this.status = status;
}
@Override
public String toString() {
StringBuilder builder = new StringBuilder();
builder.append("{\ndata:");
builder.append(sessionData);
builder.append(", \nmessage:");
builder.append(message);
builder.append(", \nstatus:");
builder.append(status);
builder.append("\n}");
return builder.toString();
}
}
Now, if you look at the Calliope dashboard you will be able to notice a screenshot below, representing the executions of your automation test script at LambdaTest Selenium Grid for automated cross browser testing.
How Was That?
Kudos! You have successfully imported your automation test results of the Selenium Grid offered by LambdaTest on your Calliope dashboard. Now, you can effortlessly collaborate your teammates while performing cross browser testing with LambdaTest. Let me know your thoughts on this integration and how it helped to fast track your test cycles. Stay tuned for more articles by hitting the notification bell at the bottom. Cheers and happy testing! 🙂
Top comments (0)