Cloud Build Dependency Graph

A couple of months ago, my coworker Mark Mandel asked if anyone had a way of visualizing dependencies in Cloud Build. As you’d expect, several of us jumped at the problem. I wrote my solution up in Ruby using the graph gem.

Cloud Build configuration files are in yaml. Each build step lists the container to run and some configuration options like arguments to pass and directories to use. Each step also has a waitFor field to specify dependencies between the steps. This is all the information you need to parse and understand to build the dependency graph.

Since I’d used it before I started with Zenspider’s graph gem. The graph gem uses a DSL to output the dot file format used by Graphviz. Using the graph gem, you define edges with the syntax edge "A", "B". An edge is just an arrow. In this case, the arrow is between A and B. If either A or B doesn’t exist yet, the gem automatically adds it.

The DSL makes creating the dependency graph pretty straightforward. All I had to do was parse the build config file and then use the waitFor field to draw edges between dependent steps. To make the graph easy to read, I chose to use the ID as the node label. If there was no ID field, I used the name. The only tricky bit is that Cloud Build config files don’t have to state all their dependencies using waitFor. If no dependency is given for a step, it is assumed to depend on all previous steps. This meant I had to keep track of the previous steps to get the edges correct. Putting it all together resulted in this code:

require 'graph'
require 'psych'

steps = Psych.load(File.read(ARGV[0]))["steps"]

previous_steps = []

digraph do
  steps.each do |step|
    id = step["id"] || step["name"]

    deps = step["waitFor"] || previous_steps
    deps.each do |dep|
      break if dep == "-"
      edge dep, id
    end

    previous_steps << id

  end

  save "buildgraph", "png"

end

Gist here

Running that code against the build file Mark shared with the team leads to this dependency graph:

All told it took about 30 minutes at hack night to code this up. Most of that time was reading about the Cloud Build configuration file format. If you’re curious, the dependency graph shown above is for Agones an open source game server scaling and orchestration platform built on top of Kubernetes.