If you work in software development or are planning to, you've probably encountered environments at some point such as local, dev, test or staging. Yet, the difference between them might be unclear and you could wonder what their purpose is. Additionally your company may refer to them with different names, making the whole thing even more blurry.
Because each environment has its own properties, you need to use them accordingly. To help you do that, we will give you an overview of the different environments that are commonly used and a few best practices.
As microservices are becoming the state of the art, we will mainly illustrate their use with the case of a web app with a microservice architecture (by the way if you don't know what microservices are, have a look at this article!).
What is an environment?
Now, let's begin to dig into the subject! First, let's define what an environment is. An environment is a set of computer resources, real or virtual, that runs your code.
In the cloud era, and in our example, an environment is usually a system of virtual machines that runs your containers and carries the infrastructure needed. Under this paradigm, an environment is mainly defined by:
- The stored data it uses (storage)
- The processing power of the virtual machines (computing)
- The communication between the pieces (network)
Except for the final environment - Production, the goal of an environment is to allow the development teams to test their code and make sure it behaves as expected.
But why are there so many different environments? Because you will test your code step-by-step, each time getting closer to Production. The main idea is to find, at each stage of the development process, the right balance between having an environment that is easy to set up, not too heavy (which would be really expensive), and the ability to perform efficient and realistic tests on it. Of course, this balance will vary greatly from one application to an other.
The local environment
The local environment, sometimes called 'development environment', is where development happens. Each developer has its own. It serves as a sandbox to work and iterate on the copy of the code base that they imported from the source code repository. Most of the times, the local environment consists of the developper's computer. It might also be a virtual one.
This environment doesn't need to be close to the production environment. If you are working on a software for an embedded system with low processing power for example, your local environment could totally be a powerful desktop computer - it's even better. The goal of this environment is to allow developers to develop and test their code through unit tests quickly.
The most challenging task regarding this environment is to design unit tests, especially for microservices. It might seem straightforward as microservices are supposed to perform a single business or infrastructure functionality. Yet, as they mainly rely on interactions with databases or services, the term 'unit test' might seem counterintuitive, as Cindy Sridharan pointed out in this great article. Therefore, a unit test at this stage can consist in testing the interface of the microservice (i.e. how it interacts with others).
Best practices
- The local environment has to be light but with a good processing power Well designed unit tests / interface tests need to be performed, automated whenever possible
- Developers should design the tests themselves and keep them updated Developers should commit frequently - but only once the code succeeded the tests!
The test environment
Test environments, sometimes called 'QA environments' or 'integration environments' are where most of the tests are performed - including integration tests, hence its secondary name.
These environments and tests are usually designed by a dedicated team - the QA engineers. Deployment on this environment and the tests that are associated are usually part of the CI process, and as such, should be automated. The tests performed are either functional or non-functional.
Functional tests in this environment are usually integration testing, i.e. how the microservices perform as a group. In a complex architecture, performing large integration tests (in terms of numbers of microservices included) can prove to be really costly and time-consuming.
Non-functional tests are more about testing the resilience of the system against traffic and charge: load testing, stress testing, endurance testing, etc...
Overall, there is an arbitrage to make on the similarity of the environment with Production, and on the number of tests and speed of integration.
Best practices
- It is of utmost importance for the team responsible for those environments and tests to document all actions. This is key for other users to be able to replicate the environment.
- Automate test and deployment on the test environment with CI.
- To keep your velocity, be realistic on the amount of integration testing you can do.
The staging environment
The last environment before the release to production is usually the staging environment but it can also have different names like 'pre-production' or 'acceptance'.
It is the final step before Production and, as such, is used to test in conditions as close as possible to Production. It can also be used to perform a demo in front of prospects, or to show a prototype to a product manager or a designer.
Usually, end-to-end tests, which are tests of the whole system, are performed in the staging environment. Teams also use the staging environment to test all the installation and configuration scripts before they're applied to a production environment. As it is closer to production than test environnements, some performance testing, particularly load testing, are tested in Staging.
Because staging environments are a wide subject, we wrote a whole article about best practices regarding these environments, but here are a few of them:
- Use anonymized data from production.
- Fake trafic.
- Limit the access to each staging environment. For instance, implements slots per team, to make sure that only one team at a time uses the environment. This prevents people from accidentally screwing with each others' tests.
The production environment
For a long time, the production environment was used only as the user-facing environment.
However, it is now trendy to perform more and more tests in production. It began with tests such as A/B Testing. A/B Testing is not about testing if the code is bug-free, but making design and UX experiments.
These days, companies begin to test the resilience of an application in production. The idea behind it is that there is nothing as close to production than production. It makes sense because in Staging, the size of your cluster will be different, configuration options will be different, there is no monitoring, etc. Also, maintaining a staging environment costs a huge amount of money and energy...
Moreover, if you have the ability to roll-back quickly, is there any risk in testing gradually in production? That's why some companies begin to neglect heavy staging environment and prefer practice such as Canary testing.
Wrap-Up
As said at the beginning, there is no single testing process that fits all. Each project must find its balance.
However, with the rise of distributed systems, the increased complexity of applications and the increased velocity of deployments, some practices seem impossible. You can't expect to test a system as a whole for each change of tens of microservices, you can't expect to fake a huge production environment on staging, and you can't expect several teams to test on the same environment without some interferences.
Therefore, it might make sense to test in pre-prod only specific elements and focus on keeping your developers as agile as possible, and lean back on a strong ability to roll-back and to deploy step by step. At Strio, we try to help companies doing that by allowing developers to spin up environment when needed, giving them agility and independance.
I hope this article made things clearer for you and that you began to grasp the difficulty of pre-prod testing for microservices!
Top comments (0)