TotalCloud’s easy-to-use Workflow Editor just got easier.

Original post: https://blog.totalcloud.io/post/the-easy-to-use-workflow-editor-just-got-easier?utm_source=dev-to&utm_medium=syndication&utm_campaign=editor

Our Workflow editor remains one of our most unique offerings, that can be thought of as a DevOps engineer’s personal drawing board to automate AWS actions. This release, we worked on enhancing its capabilities. We built custom workflows to make it really easy for any DevOps engineer to create and deploy workflows, eliminating the need to code or do it manually - thereby saving a lot of engineering time that could be spent on innovation. With the latest update, we bring an enhanced level of flexibility and customization in the workflow creation process that enables you to solve complex use-cases.

One of the biggest measurable benefits you acquire from this is the ability to create workflows for extremely complex use-cases. Actions that would previously require multiple workflows to execute through the old editor, can now be setup with a single workflow. For instance, a use-case that identifies unused EC2 AMIs, deregisters them and deletes the associated snapshot can be performed using a single workflow, with two action nodes.

So here’s what changed

A workflow is created by combining a bunch of building blocks called ‘nodes’. Earlier, the output of each node was translated as the input of the subsequent node, which meant that the workflow could only be created as a unilateral logical sequence of events. With the new engine update, you consequence actions based on your needs, opening up a whole range of complex possibilities with respect to creating workflows. Here’s how:

Node Referencing

On the new engine, each node is given an ID, which can be referenced while setting up actions. This essentially means that you can perform multiple actions on multiple resources, filtered or otherwise. The editor asks you to pick the specific resource on which the action is to be performed, and the applicable ID can be picked, as shown below.

Resource Referencing

Node referencing solves the problems of complexity, while resource referencing allows customization at the granular level. Apart from the basic configurations, where you pick the resource you want to operate on, advanced parameters can be used to select additional qualities/attributes of resources, like Security Groups of EC2 instances, or Policy Information of Buckets. Like Node IDs, each resource attribute is given an ID; this enables you to perform actions on a specific attribute of a resource, along with actions on a filtered resource.

The following examples depict how complicated use-cases can be solved with the new TotalCloud editor:

Use-case 1 - EC2 Auto Availability Zone Rebalancing

It is considered an AWS best practice to ensure EC2 instances are evenly spread across Availability Zones (AZs) within a particular AWS region. This isolates the impact of any data center outages and improves the reliability of your instances. TotalCloud enables you to perform a reliability check to determine unevenly balanced AZs. But the process of remediating this check, and migrating instances from one AZ to another to rebalance distribution is a complicated process, if done manually and can take from days to weeks, depending on your scale. In addition to that, AZs can continue to get unbalanced over time. This means that remediation will become a repetitive task. With the new workflow engine, TotalCloud enables you to automate the rebalancing of instances across AZs through a single workflow, and ensures that they always remain balanced, eliminating the tedious repetition of manual remediation.

Manually: The rebalancing can be done manually, through the AWS console or CLI with the following steps:

• Determine the instances that need to be migrated to various AZs
• Create AMIs of these instances
• Re-create or launch the instances using the AMI in new AZs
• Configure Security Groups for the newly launched instance
• Reassign the Elastic IP address from the instance you are moving to the new image
• Set up required configurations for the newly deployed instance
• Attach required ELB and ASG to the new instances
• After ensuring the new instance is running on the new AZ, shut down the old instances

Each step of this process requires you to manually configure AWS settings, which constitutes a huge waste of time and effort.

With TotalCloud: The workflow editor can generate a single workflow to rebalance your AZs

A workflow can be created to filter out the instances that need to be migrated and perform a series of actions on the filtered instances. Four action nodes can be set up to:

• Take AMIs of the instances
• Bring up instances in the new AZs
• Other configurations (like attach to ELB)
• Delete the migrated instances

Node and Resource referencing allow you to select the output of the filter node (through the ID) as the input for each action node, ensuring that all actions are performed on the instances that are to be migrated.
The ‘Custom Node’ can be used to define which instances need to be migrated, and across which AZs they must move; so the filter will pick out the correct instances. The custom node is a particularly unique node that lets you enter almost any configuration, directly through a javascript; and it happens to be popular amongst our devops audience (because it practically lets you do anything you want!).

Use-case 2 - Find and attach missing CloudWatch alarms to resources

Another complex use-case, finding and attaching missing CloudWatch alarms to resources, is a unique one that we created and deployed for a customer! If a resource is missing a CloudWatch alarm, that was set to notify the user of a certain threshold, it can lead to production or system outages, and additionally, uncontrolled costs.

The new update performs this entire use-case in a single workflow. Multiple resources can be selected and filtered out to find which resources are missing alarms, and what those alarms are. In order to remediate this, action nodes can be configured to attach the missing alarms to each individual resource. For example, if 10 out of 100 instances are missing different sets of alarms, a single workflow can be used to attach a different alarm to a different instance, as per the report. With the introduction of node referencing, every action node will reference a particular filtered resource node that it’s performing the action on. So the number of individual resources will determine the number of resource nodes, filter nodes and action nodes; all connected through the IDs.

Conclusion

Scalability comes in conjunction with complexity, which is why the updated Workflow Editor performs complex cloud actions in a matter of minutes. You can build out a workflow to solve almost every use-case imaginable, whether simple or complex. Every workflow you create and deploy automates an action, preventing the need to constantly create scripts for repetitive tasks; so your devops team can focus on adding value through innovation. You can get started with creating a workflow right away with a TotalCloud account. If you have any feedback or ideas, let us know in the comments section!