Improve your project workflow with these bitbucket pipelines/github actions automations - ClickPesa way

Introduction

Bitbucket Pipelines is a continuous delivery service that helps you build and ship software quickly.

You can use Pipelines to automate the building and testing of your software projects, so that it's easier for your team to deliver new features and updates on schedule.

Bitbucket Pipelines are a powerful tool for automations. But if you don't know what they are or how to use them, they can be pretty scary. We've been using pipelines for a while now and have learned some things about how to make them work for us. Now we want to share those learnings with you so that you can improve your project workflow! - ( Learn more about bitbucket pipelines )

Problems

We did not just create deployment branches and automate the workflow, we had a problems first,

1. Single branch deployment allows untested content to be deployed directly to the production. These tests include internal and external testing, causing buggy features to be published to production.
2. Manage a team of developers working on the same project and pushing changes to the same branch.
3. After creating multiple deploy branches when a developer push changes they had to merge those changes manually to each deploy branch increasing human errors in the whole deployment process

After all these problems and with the development team growing, we came up with one solution. Automated the entire development process. This process starts from when developers push changes to the first deployment branch (develop) to preview branch (staging) to production to software release.

Solution

For the first problem we created three different deployment branches which run on different environments. Develop branch deploying to Develop environment for internal testing.

Staging branch deploying to Staging environment for internal and external testing, in this configuration, the Staging environment matches Production as closely as possible. Master/Main branch that deploys to production.

Developers create their own branches and merge changes to develop for internal testing. This solved the second problem as well where it is easier managing developers in this workflow.

After creating deployment branches we employed a solution to automate the entire process from when developers merged their changes to develop branch to publishing those changes to production using bitbucket pipelines/github actions and Javascript.

These automations includes

1. Running automated unit tests on feature branch
2. Running automated end to end tests (e2e tests) on develop branch
3. Automate pull request creation ( Learn more )
4. Running scheduled end to end tests (e2e tests) on staging branch
5. End of the day/scheduled pipelines/actions running everyday at 1800 hours to run e2e tests, merge pull requests to staging branch, deploy staging branch and creating pull requests to master/main branch
6. Automated pull request merging
7. Deploying branches
8. Sending automated slack notifications to reviewers for manual review before production releases
9. Automated software versioning
10. Automated changelogs and release documentation ( Learn more )

Conclusion

After we implemented a number of these actions and tools into our existing workflow and we have seen some great improvements. The main benefits I think they offer are two fold.

Firstly they save us time as developers as we no longer need to manually perform repetitive tasks, like opening pull requests or merging pull requests after a successful build. These tasks can now be automated, so that these things happen in the background without any user interaction required.

And secondly they keep mistakes to a minimum by providing feedback when something goes wrong.