Intro
If I told you I am passionate about software testing, you may think I need to get a new hobby. That said, if you never tried writing automated tests, you are truly missing out on a discipline that makes writing software better in every way.
This post is first in a series that will cover the basics of testing, particularly in relation to web-apps, on the front-end (JavaScript) and back-end (python). If you use a different stack, most of the theory will still apply. If you use JavaScript (Node.js based stack) for the back-end, you can use the many of the same tools for front-end and back-end.
This part contains a lot of theory. I still encourage you to read through it - once I add practical examples, it will all make sense. For now, we will cover two types of testing - manual testing vs automated testing.
Why Testing?
Code without tests is broken by design.
~ Jacob Kaplan-Moss
As developers, we often want to deliver software as fast as possible. If we're honest and good developers or engineers, we also want to deliver software that isn't buggy. Writing software isn't just about taking an issue from the issue tracker, writing some code, and closing the issue. If we don't test the code we write, then our customers will have to test it for us, meaning that they will find bugs. That is not fun for your customers, and based on the severity of the bugs they find, they may be put off from using your services altogether.
That is why we should always test before we release software... Catching bugs earlier is always cheaper.
Manual vs Automated Testing
There are two very common approaches to testing: Manual, and Automated. Let us describe them, in case you are not familiar with them.
Manual Testing
Manual testing means that we do everything by hand. We load our website into our browser, click around it, and make sure that it acts as we expect it to.
It is still an important part of testing. (Also, it's a very fun part! Who doesn't like seeing first-hand the results of their labor?)
It used to be very common for companies to spend weeks, or even months working on some feature(s), then passing it off to a QA (Quality Assurance) team for approval. QA engineers had a list of tasks they had to execute manually, and, if they didn't find any bugs, they approved the release. Many companies still do that, but there is a much better way.
Automated Testing
Strange as it may sound, automated testing means we write code that tests our code. Many technological companies have now implemented automated testing into their workflow and, to be honest, you probably do not want to work for a company that has not.
A comparison of manual and automated testing follows, so you can see the benefits for yourself.
Comparison of Manual and Automated Testing
| Property | Manual testing | Automated testing |
|---|---|---|
| Protects from user errors | ❌ | ✅ |
| Fast to execute | ❌ | ✅ |
| Does not add lead time | ✅ | ❌ |
| Prevents repetitive work | ❌ | ✅ |
| Integrates into CI/CD pipelines | ❔ | ✅ |
| Integrates into TDD | ❔ | ✅ |
| Prevents regressions | ❌ | ✅ |
| Is easy to implement | ✅ | ❌ |
Protection from user errors
Humans are not as good at repetitive tasks as machines are.
While I have much respect for QA Engineers, they may easily miss a step when running through the app. When you use automated testing, a machine will run through your tests exactly as you programmed it to. For that reason, test results are more predictable and repeatable.
Rapidity of execution
How long would it take you to run 102 tests manually using the interactive python REPL shell?
In one of my projects, 102 automated tests run in under a second (disclaimer: that's only the pytest run - it takes tox a bit more to set everything up, but it's still really fast.)
Of course, not all tests are created equal. Unit tests, integration tests, end-to-end tests, smoke tests... there are many types of tests (that will be covered in a later article), and some of them are slower than others.
Still - automated testing is generally much much faster than manual testing.
Lead time
On the flip side, a common complaint about automated tests is that they are time consuming to write whereas manual testing does not get in the way of releasing software.
It is true that writing tests takes a bit more time when writing the code. That said, the amount of manual testing you have to do to ensure there are no regressions (i.e., you implement a new feature, and accidentally break an existing feature that was previously working) grows rapidly, which brings us to...
Repetitive work
Remember when I said manual testing was fun? The fun occasionally does get old. If you have to repeat tests manually, it gets really tedious. Worse, if you spend 10 hours a week on manual testing (and you are not a QA engineer, but a developer), that is 10 hours less you have for programming.
On the other hand, the computer can repeat tasks essentially forever. It is great at it, and its time is generally much cheaper than an engineer's.
Integration with CI/CD Pipelines
Continuous Integration (CI) is the practice of merging all developers' working copies to a shared mainline (e.g. the master branch) often - usually several times a day. It makes your merges much smaller, meaning that if there are conflicts in the code of different developers, you will catch the conflict sooner, meaning it is cheaper to fix. Imagine spending a month working on a feature, only to realize that it is incompatible with the work your colleague has done during the same month - you may just have wasted one man-month of work.
On the other hand, with CI, you make small incremental merges, with incomplete features.
Continuous Delivery (CD) is an approach that ensures that software can be reliably released at any time. It uses a deployment pipeline - a set of validations the software must pass in order to be released.
Automated testing helps tremendously - you can configure pipelines to run your tests every time you push to your git repository on GitLab, GitHub, etc. It is good practice to also run manual tests on your software before releasing it to production... You can use a Continuous Deployment pipeline to build a Staging version of your website, and run automated and / or manual tests against it.
That said, manual testing does not fully integrate into CI/CD Pipelines since those are meant to be automated.
Integration with TDD
Test-driven Development (TDD) is a development technique where you write tests first, watch them fail, write the minimal amount of code you need to make the tests pass, then refactor the code.
I originally was going to say you can't do TDD with manual testing, as I have never heard of anyone doing that... But I guess, in a way, we all sort of do it.
- Define the functionality your website should have.
- Create a basic HTML file (or Vue.js, React, Angular, etc. project).
- Open it in your browser.
- Add some functionality, reload the browser, and test if it works as expected.
- Repeat until your website is feature-complete.
So, theoretically, I guess you can integrate TDD and manual testing.
That said, most people who claim to practice TDD do it with automated tests. A practical example of TDD will follow later in this series.
Preventing regressions
A software regression is a bug that breaks functionality that worked previously, e.g. something that works in versions 1.0-1.5, but break in version 1.6.
When using manual testing, you typically focus on testing new functionality, and the most crucial features that were known to work previously. It is very time-consuming to test all functionality manually, so it is very easy to miss a bug that suddenly appears.
With automated testing, if you introduce a new bug, a test should fail, and you should be aware of it before you release. If your tests fail to detect it, you write new tests to cover the case at the same time you fix the bug - then you can be sure the bug will not reappear.
Difficulty
Manual testing is obviously easy. As long as you can use a computer, you can test your website's functionality (although testing all edge cases may prove much more difficult than you expected).
Automated testing is honestly very difficult to get right. You need to learn the following:
- Test tools
- How to write tests that...
- Test the right thing
- Pass for the right reasons
- Fail for the right reasons
- Do not break when you write new code
Conclusion (part 1)
Hopefully, the difficulty will not put you off from adding automated testing to your arsenal - like all things, you will get better at it with practice.
Stay tuned for the next installments in this series, we still have plenty of things to cover, such as:
- Test tools
- Types of tests (unit tests, integration tests, smoke tests, end-to-end tests, regression tests, etc.)
- TDD in detail
- What to test
- etc.
I further hope that by the time this series is complete, you will be as passionate about testing as I am 😄.
Top comments (0)