Beyond Functional Testing: Devender Sharma [Testμ 2022]

The concept of testing looks pretty straightforward when you begin working with the software. You want to determine the extent of the software where it meets all the requirements, but there is more to software testing than just answering the question, “is it functional?”

In this insightful session of the Testµ conference, Devender Sharma, Head of Quality Engineering at Plentific, explains what’s beyond functional testing and the principles and practices one can adopt.

Devender starts the session by introducing Functional Testing as a testing technique that ensures a product behaves as expected. He highlighted a few key facts, such as:

• Products are built by people for the people.

• People use technology to build products.

• People also create technology itself.

• Processes are made for people.

He presented through these key facts that People, Technology, and Products are the three components of a wheel that support each other.

Devender then asked a question, “Why should we think beyond?”

To answer this, he highlighted that we need to look at the real user perspective in the world. The following is the list of all types of people that might be a user of the product:

• Users on different platforms such as mobile devices, etc.,

• Users on different browsers,

• Users who mine data,

• Users redirected from other platforms,

• Users with special needs,

• Hackers,

• Users from diverse locations.

Devender elaborated that we get to know our customers better by categorizing our users. This includes:

• Demographic

• Behavior

• Technological platforms

• Usage

Beyond Functional Testing

“One of the key things is that it is always good to be customer-centric.”

Devender explains that being customer-centric means asking questions like, “Is the experience for the user engaging and exciting?” or “Is the customer experience consistent?”

These questions led us to look at features like Usability, Reliability, Accessibility, Performance, and Security as the key aspects of how we can easily cater to the customer.

“Quality is everyone’s responsibility”

Devender introduces the Quality and Capabilities framework, which works as an assessment tool to provide a 360 view of your team and how to navigate testing.

Quality Engineering is the discipline of building and delivering software that considers and bakes in quality from the outset. Devender believes that quality is a significant asset when attached to engineering, and it becomes a bigger responsibility to empower everyone with it.

Principles and Practices

According to Devender, there are some principles and practices that he believes are the core values that can help team leaders achieve success in their projects.

• Engineer for Quality.

• Automated tests are part of the delivery.

• Quality embedded teams.

He explained each core value as a Principle and a Practice.

Principle: Engineer for Quality

Ensures you have the appropriate strategies for production and non-production environments. Before development begins, start thinking about starting designing and strategizing about performance, security, compatibility, accessibility, and automation.

Practice: Engineer for Quality

• Ensure you have the right skills.

• Engage quality engineers from the outset.

• Focus on what you need to do to move fast before you move fast.

• Stay Cohesive as a team.

• Aim to reuse tools and knowledge by building reference patterns and propagating.

Principle: Automated tests are part of the delivery

Automated tests should be delivered as part of a new feature or change of production and not treated as an optional opposed release task. The rationale behind this is to reduce cost and increase the delivery speed.

Practice: Automated tests are part of the delivery

• Automate at the lowest layer

• Make tests accessible

• Make tests small and independent

• Integrate automated tests in deployment pipelines

Principle: Quality embedded teams

This principle focuses on creating a culture around embedding quality in teams. This will contribute to increasing the speed of delivery as well as to finding quality-related issues at the first possible opportunity.

Practice: Quality embedded teams

• A whole team approach which includes the BDD approach

• Fast Feedback

• Exploration focuses on doing more exploratory testing within your teams.

  “How can we know progress if we don’t measure it?”

Devender advocated for a principle that should apply to our testing world: having metrics in place for everything possible that gives value. For example, having metrics in areas such as code coverage keeps track of the code quality, which helps you to reflect where you and your team are with the quality of the products.

Finally, to sum it up, Devender emphasizes the importance of collaboration. This explains the act of working together and understanding what is important. It includes an alliance between People, Processes, and Products. In the end, he touches on how everything revolves around these three aspects, which is what engineering quality can define.

Q&A round!

After the session, the attendees had a few questions for Devender. Let’s look at them:

What are your thoughts on doing Non-Functional Testing (Load test, Performance, etc.) in the early phase of Software Development?

Devender: It is tough to do load testing very early in software development when your software is not ready. However, if people can do this very early, your architecture is mature enough, allowing you to run these sorts of testing early in the process.

In your personal experience, are DevOps skills (at least for the deploy process validations) needed for a Quality Engineer?

Devender: DevOps is a skill set that is very concentrated on ensuring developers’ operation side is maintained, created, or evolved in good practices. The quality engineers are people who need to work very closely with DevOps because we need to make sure the way DevOps are building up those operational objectives is also shaped up correctly.

It becomes very important for quality engineers to have that prospective work very closely with DevOps and shape up the process of operations and how those operations are happening, how the releases are happening in the production or non-production environments, and what sort of tests we are running.

What are you testing as a part of analytics?

Devender: For example, you are on a website, and you click on something, and the event that comes up on the console lets you see a Google Analytics event that generates. That event is generated as part of analytics. It is probably fed to some other external systems, so when I have to test it, I’m making sure if I click something that analytics is generated or not.

Now, suppose you already have automation in place. In that case, this is relatively easy to automate, so some people would focus on automating it at the very early stages of the unit test. This unit test will help run something similar to a virtual dom, where you can test the analytics automatically rather than manually testing it at the later stages. It ends up helping you as it makes you future-proof for upcoming releases or other benefits.

Any suggestions on how to get Developer buy-in for Shift Left testing?

Devender: It seems a very hard process because it’s a cultural change. But the good thing is showing them the importance of shifting left is easy if they are keen and passionate about the technology. If you’re developing something, it should have quality in place, and to convince developers to shift left testing, I think, is always a cultural thing, so we have to work closely with them.

When we talk about shift-left, some folks might confuse that with the meaning that testing needs to be done by the Developer (whether Unit or E2E). Is that what Shift Left truly means, or does it mean that devs should focus more on Unit testing?

Devender: It is all about perspective. Shift left, in my opinion, isn’t something related to people and technology; instead, it’s much more perspective related, so what processes or sort of practices are you following that you can shift left in the process?