What do companies like Amazon, Target, Esty, Netflix, Google, and Walmart all have in common? Apart from the fact that they are wildly successful companies, they all use a method known as DevOps in their day to day processes to increase efficiency and improve delivery time. In this DevOps guide, we will try to explain what is DevOps and how it can be a boon for your business.

What is DevOps and Why is It So Widely Used?

So what exactly is DevOps? Let’s take a small hypothetical example to illustrate. Let’s say there is a small startup that builds AI-enabled cleaning robots. There are 3 developers (let's be lazy and just call them Team D) who write and execute the code to create the robots and two operations people (Team O of course) who maintain the robot infrastructure in the real world environment and provide support for the robot users.

Team D has just spent 8 months creating the latest robot. It can recognize people, take commands from Alexa devices and of course clean like a boss. Team D has spent time creating this robot in their controlled dev environment and everything seems to be working smoothly. They couldn’t be prouder.

They hand over their creation to Team O which promptly takes it out into the real world. That’s when the problems start. It turns out that the perfect cleaning robot is not so perfect after all. It doesn’t recognize everyone, it’s capacity to follow Alexa commands breaks down when they are given by different people and it can’t reach and vacuum tricky shelves.

Team O is angry and frustrated. They have been waiting for this robot for so long and they can’t believe it’s an unmitigated disaster. Team D, on the other hand, is now on the defensive. They believe that everything was working perfectly during controlled testing conditions and the royal mess now must be because of Team O’s poor execution.

In conclusion, there is a sub-optimal product in the market, it seems like corrections and improvements will now take almost as long as product roll-out did, and two very competent teams now hate each other and their jobs.

That, in a nutshell, is why DevOps was created and why more than 70% of SMBs (Small and Medium Businesses) were adopting DevOps Services in their companies in 2016. A lot of heartache, frustration and inefficiency could have been avoided had Team D and Team O worked with each other right from conception and execution to delivery and support. Instead, they worked in silos with an imaginary wall between them.

Team O had no involvement when the code was being written and the robot was actually being built while Team D was completely out of the picture when the robot went out into the real world and cleaned many different houses. The result was a robot that was not ready for the market and a dev team that still doesn’t quite know how to resolve things.

As you can imagine, a lot of time will go by before the robot is finally market-ready. There will be a number of iterations as Team D makes some changes and Team O sends the robot out into the real world. And not just during product development, this same issue will continue when the robot needs maintenance and upgrades. As a result, even a small startup ends up becoming inefficient and slow and there’s a good chance that it loses out to other companies that get superior products to market faster.

This was the problem that companies started seeing especially as technology became more advanced. Keeping their operations and development teams isolated from each other was resulting in slower delivery and less efficient products and services. Moreover, many processes in the company operations that could have been easily automated to increase efficiency were not being automated because the developers were simply unaware of them.

That’s when the concept of DevOps came to the fore and began being adopted widely. DevOps is nothing but a set of philosophies, practices, and tools that help an organization to deliver better products faster by facilitating the integration of the development and operations functions. This enables companies like the one in our example to serve their customers and markets in a better way and have a competitive edge.

This starts from the design to the entire development process right up to production support.

How Has DevOps Evolved?

DevOps is now in its 10th year. DevOps, like most philosophies and tools that have practical application, has had a journey from being just a bunch of ideas and principles thrown together into a full-fledged discipline with its own processes and tools.

Back in 2007, a project manager named Patrick Debois was working with the Belgian government to help with data center migrations. He found the whole process extremely frustrating because of the wall between the developers and the operations team which made his job much harder and delivery much slower.

Debois was a big believer in the agile methodology which promotes continuous iteration of development and testing throughout the development lifecycle which helps dev teams ship better products faster. He believed that similar principles should apply to the development and operations teams working together in sync.

In 2008, Andrew Schafer (who later became known as a DevOps evangeliser), and Debois got together at a conference to discuss initial ideas and principles around what they then referred to as “agile systems administration”. They also formed an agile administrator group on google which is where the real beginnings of DevOps lie.

Another landmark event in the history of DevOps’ evolution was the now famous presentation by Flickr employees John Allspaw(then VP of technical operations) and Paul Hammond(then Director of Engineering) at the O’Reilly Velocity conference in 2009. With a hilarious yet hard-hitting role-play, Hammond and Allspaw brought home the fact that there was major business loss due to the wall between development and operations and the only way out was a seamless integration between the two.

This presentation came to be known as “the defining moment” for DevOps as the tech world quickly woke up to the need of such integration. The presentation inspired Debois to organize a DevOps conference called Devopsdays in Belgium, and the rest, as they say, is history.

Another important moment in the evolution of DevOps was the first DevOps conference in the United States. It was held in 2010 at Mountain View, California, the Mecca of technology. This was the definitive signal that DevOps had arrived and was here to stay. In 2018 in fact, there were more than 30 DevOps conferences across the world.

How Does Using DevOps Benefit an Organisation?

The reason DevOps saw such rapid adoption is that it truly makes a massive difference to how a tech company operates at a very fundamental level. Let’s take a look at some of the benefits that accrue when a company adopts the DevOps approach.

Accelerated Innovation

This is the major reason that DevOps came into existence. Using DevOps allows companies to develop and deploy products much faster. As we saw in our earlier example, cycle times become significantly longer when there is a wall between development and operations. When the two are integrated, on the other hand, changesets are smaller and problems to be solved each time are less complex. Moreover, team members can make software changes easily since they only need to look at the latest code added and not at all of it. Things like microservices and continuous delivery allow teams to take complete ownership of projects and deliver them faster.

Collaboration

As our example has shown, a wall between development and operations often results in an environment where the two teams don’t trust each other and each is walking around a little blindly. This has long-term repercussions on the morale of the team and how motivated they are to work towards their goals. A DevOps approach results in a collaboration between the two teams where they work with a shared passion to achieve common goals. This creates a much more positive work  environment where outcomes can be reached much faster and more efficiently. This also has other positive outcomes like enhanced job satisfaction and lower attrition.

Reliability

Before DevOps updating an application to meet changing user needs was a nightmare. There was always a chance that updating the application would compromise the quality required by the user.
With DevOps tools like continuous integration and delivery, it is now easy to test the functionality of the software and keep security and quality in mind. Other processes like monitoring and logging help keep track of real-time performance metrics which help maintain the reliability of the software.

Security

Without DevOps, you have to often make a tradeoff between speed and security which results in delivery time becoming a lot more. With DevOps, you can use automated compliance policies, fine-grained controls, and configuration management techniques to maintain speed without compromising on security.

Scalability

DevOps started growing in prominence as companies like Google, Amazon, and Youtube started finding it harder to manage their technology at scale with a wall separating development and operations. The automation and consistency that comes with DevOps allow you to manage and change complex systems more efficiently.

What are Some of the Best Practices for Effective DevOps

While DevOps still means different things to different people, there has emerged a core of best practices that should be incorporated by companies looking at adopting DevOps.

• Active Stakeholder Participation

This is the fundamental guiding principle of DevOps. DevOps can succeed only if both the developers and the operations and support staff are truly committed to collaborating and using an integrated approach to achieve goals.

• Automated Testing

Automated regression testing is something agile teams adopt very often as it helps to fix problems right away and ship higher quality code. This works well in DevOps too as a dire need of operations staff is that the code shipped should meet a certain quality standard.

• Integrated Configuration Management

In a DevOps environment, configuration management applies not only to the current solution being worked on but also on the configuration issues between the solution and the rest of the organization infrastructure. Integrated Configuration management helps operations teams see the potential impact of a new release more clearly which helps in making better decisions regarding when the release should be made.

• Integrated Change Management

With integrated change management, operations and development teams work together to understand how using different technologies will impact the organisation as a whole and then work toward managing that.

• Continuous Integration

With continuous integration, the code is tested and analysed whenever updated code is checked into the version control system. This provides immediate feedback on code defects which allows developers to build a high-quality solution with little risk.

• Integrated Deployment Planning

A DevOps approach means operations engineers will be closely involved with the developers when it comes to planning the deployment of products as per an organizational deployment schedule.

• Continuous Deployment

With continuous deployment, when integration is successful in one sandbox it is automatically promoted to the next sandbox and integration begins there. This continues until it reaches the point where it requires human verification. This usually occurs at the point of transition from dev to operations.

• Production Support

With DevOps, not only do developers work on new releases, but they also work on addressing critical problems with a solution that is already in production. Although they are the third and last team to get involved in solving production issues, it is a fairly common occurrence and gives them insights on production problems that help them design better solutions in the first place.

• Application Monitoring

This refers to the practice of monitoring and logging solutions real-time once they are in production. This gives us performance metrics that improve the reliability of the solution and prevent failures.

• Automated Dashboards

DevOps allows us to create automated dashboards for several key metrics. All metrics cannot be automated of course, but several key metrics can be seen real-time using automated dashboards and they provide critical business intelligence.

What are the DevOps Tools?

In order to implement DevOps best practices described above, certain tools have been developed to automate and facilitate different DevOps processes. While the right tools play a key role in effective DevOps implementation, simply using the tools does not mean DevOps adoption. Tools are only relevant when they are used as the last stage- after the organization has already adopted the philosophy of DevOps and there is a commitment to executing its best practices.

Although DevOps was not supposed to be about tools, with its evolution in the last few years, a number of technologies that were not part of the original concept have now become an integral part of DevOps. According to research firm Gartner, a linked toolchain of technologies has now become critical if DevOps is to bring about the change it’s meant to. In recent years there has been an explosion of DevOps tools for different DevOps practices. Here are just a few examples.

Release Tools

• Jenkins
• Travis
• TeamCity
• Bamboo

Configuration Management Tools

• Puppet
• Chef
• Ansible
• Cfengine
• Saltstack

Orchestration Tools

• Zookeeper
• Noah
• Mesos

Monitoring, Virtualization and Containerization Tools

• AWS
• OpenStack
• Vagrant
• Docker
• New Relic
• Sensu
• Spunk
• Nagios

Coding Tools

• Jira
• Git
• Eclipse

Testing Tools

• JUnit
• Zephyr
• Selenium
• Vagrant
• SoapUI

The Key to Effective Adoption of DevOps

Adoption of organisation wide DevOps is a slippery slope because it requires a philosophical and cultural change combined with a more practical implementation of tools and best practices. If an organisation simply aspires to the philosophy of collaboration and efficiency behind DevOps without doing the hard work of actually executing it on the ground, DevOps will remain a philosophy and nothing more.

At the same time, simply adopting DevOps practices and tools without the philosophy and DevOps culture permeating across the organization is also futile. The starting point of successful adoption of DevOps within your organization should be getting your development and operations teams fully committed to the cause. It is only after they are fully onboard that the best practices and DevOps tools should come into the picture.

Originally published on Cuelogic Blog