Shift-left is a practice intended to shorten the feedback loop in the lifecycle in order to take decisive action.
The idea is to improve overall quality and security and more by moving tasks to the left as early in the lifecycle as possible.
- Much fewer human errors
- Increased test confidence
- Fewer production issues
- The time between releases can reduce significantly
- The quality of software improves
- Faster delivery of software with less defects is a major benefit for all; from product owners, stakeholders and even customers!
When building API’s for example, the team owns the API product, its design, its code, its quality, in order to ensure a high-quality product is produced for its customers (consumers), the engineering team is responsible for testing its own product. From unit-testing to performance testing and all those in-between, the list above is not an exhaustive one, and there are many other testing practices you can put in place.
I do hope many of the items listed above are being carried out by an engineering team who owns the product already, for those who are not, the next step is how? How do we, “shift-left”? Well, we need to automate, if you can automate these items, and let’s be honest, who hasn’t automated some of these before in a CI (Continuous Integration) pipeline? - but not everyone has. If you haven’t, these should be an essential items in your shift-left shopping basket!
Security is paramount these days, and a lot of the time, InfoSec (Information Security) or security teams are on the right. Quite often, these teams are not large teams, and adopt a decentralised model, which puts the responsibility on engineering teams. This means the pressure is already on the engineering team to shift-left and bake security practices into the lifecycle, early and often.
Through automation, we can achieve fantastic results, as part of CI, it is easy to achieve static / dynamic code analysis, security scanning of internal and open source libraries, and even container scanning. This all means as an engineer, you’ll know very early on if something isn’t quite right, instead of waiting later, and possibly relying on another team.
As the world of tech continues its love affair with containerisation, security is always at the forefront of security engineers, and with shift-left, it now should also be at the forefront of every software engineer. With containers specifically, container scanning tools are abundant, and some container registries also scan images that are stored. Scanning before pushing a container and when it lands in the registry, is a typical practice to follow.
With great power comes great responsibility, security should never be an afterthought.
Infrastructure is something we simply can’t avoid, even with serverless; serverless may have integration on vnets (virtual networks). Usually there will be a centralised Operations or Platform team who owns, designs, builds and operates all infrastructure. This can become a big blocker for engineering teams, teams who cannot deploy their own API’s for example, are reliant on another team, and this will in-turn break team autonomy.
So what do we do, we shift-left? Now, each group of teams will be different, each infrastructure will be different, it is likely you may not shift-left 100%, but you can easily shift-left quite a lot in most setups.
A centralised Operations / Platform team can design, build, and operate baseline, foundational, bedrock infrastructure. Thereby, product teams can build on top of this, and come together when needed, for instance, virtual networks, firewalls, a SEIM as examples.
As a product team, what can you now do? Well, now the engineering team can shift-left; they can build on top of the foundational infrastructure. This improves team autonomy, a sense of ownership, and responsibility, furthermore, you breakdown that barrier to release frequently - Deployment frequency is a key DevOps metric.
One way to achieve this is through infrastructure as code (IaC), and we treat it like any other code, it lives in git. We create PR’s (Pull Requests) and go through CI and CD (Continuous Deployment), it can all be automated, checks can be executed, linters, most tools such as Terraform and Pulumi can preview the infrastructure changes before they are applied.
DevOps is not a role, it’s a culture!
Once an API, or a server-side rendered web application, or even a statically generated web application is in production, it needs to be monitored; I wonder who should do that….? A centralised team…. or the team that designed it, built it and deployed it….? It’s of course the engineering team that designed it, built it etc.
In many organisations just like infrastructure above, the Operations or Platform teams operate the web applications and API’s etc., this is rather right-heavy so-to-speak. As with infrastructure, the same problems occur with operating services, they define monitoring and alerting, are on-call and more. This can work great for smaller organisations, however, if you are in an organisation that is growing and growing as rapidly as NewDay for instance, this model starts to become a hinderance. Now, what do we do, that’s right, we shift left!
As part of our normal engineering lifecycle, we define performance metrics, degradation metrics, any kind of metric we need for our apps and API’s, we go further and define alerts, anything that is perhaps close to breaching our metrics where we have clear and concise SLA’s (service level agreements) or SLO’s (service level objectives). As part of IaC, these can be baked in as code, we can test early and often in our engineering lifecycle. Once we get to production, it’s the job of the engineering team to monitor there apps and API’s, as they are the ones who designed it, built it and deployed it, the team will be the one to operate it. The engineering team will most likely be on-call if your organisation requires 24/7 operations, because if something goes wrong, the best people to fix it is the team that built it!
If you've reached this far, thank you, I'm sure you're now wondering what's next? Well, we will be showcasing practical examples as part of a series soon.
Here at NewDay, we are on our journey to shift-left as we scale, it’s not a short journey, but it will be an enjoyable one, one where we come together not just as individual teams, but also as a community of engineers to help drive forward-thinking, innovative change.
The shift-left philosophy will bring unprecedented value to engineers and our customers lives.
NewDay embraces a culture of team autonomy, ownership, responsibility and collaboration. We always aim to do the right thing and pull together, taking on the challenge to aspire to the extraordinary and adapting as we build for the future.