Danger on Rails: make robots do some code review for you!

This article is a b-side of my recent RailsConf talk "Terraforming legacy Rails applications" (video, slides).

We, developers, spend an essential part of our work time doing code reviews. Sometimes much more time than writing code. And even spending hours couldn't save you from missing some potential problems in pull requests.

We're just people, we do make mistakes, especially when dealing with routine tasks.

Who loves doing boring things, and (almost) does not make mistakes? Robots! 🤖

In this article, I want to introduce a robot which can help you with code reviews. Its name is Danger (or more precisely, her name).

Danger runs during your CI process, and gives teams the chance to automate common code review chores.

Adding Danger to the project as simple as adding danger to your Gemfile and running the interactive bundle exec danger init command, which:

• generates an example Dangerfile—that's where you define the review rules using a Ruby DSL;
• navigates you through the GitHub integration setup;
• finally, helps you to add a Danger task to CI.

The danger gem itself provides only a "small core":

• Git integration to allow accessing the repo information (commits, diffs, etc.).
• Git hosting services integrations (GitHub, BitBucket, Gitlab) to access PRs data and add comments.
• (internal) CI services integrations to correctly match CI builds to PRs.

All the specific functionality (e.g., linters integrations) is implemented via plugins (checkout the awesome-danger).

That could be the end of this post: install Danger, select plugins, add Dangerfile, and you're done. If that were all I want to share, I would probably just tweet about it. Adding Danger to a Rails project turned out to be a bit more tricky and much more interesting...

GitHub re-integration

Danger assumes the following GitHub workflow:

• you create a new GitHub user;
• add it to the repo as a collaborator (for closed-source projects);
• generate an access token for this user and add it to CI as DANGER_GITHUB_API_TOKEN.

Wait, create a new GitHub user? With email and password? Isn't there a better way to add bots? Yes, there is—GitHub Apps.

GitHub Apps can do everything we need: read repos/PRs data, add comments, and, what is more important, have more granular permissions*.

* Recently, GitHub introduced new permissions levels for collaborators, Triage and Maintain. The first one is a good fit for a Danger user.

The main difference is the way GitHub Apps access tokens work: they have a very short lifetime (10 minutes maximum). Thus, we need to generate an access token every time we want to run Danger.

Despite that everything else stays the same: we pass this token as DANGER_GITHUB_API_TOKEN and Danger uses just like a user's access token.

Continue reading to see how to create a GitHub App and make it serve as Danger bot on CircleCI.

Step 1. Creating a new GitHub App.

You can create a personal GitHub App or an organization-level one. We recommend using the latter one if you have an organization.

Follow these instructions to create a new app. We only care about the "Permissions" section:

• Select "Read" for "Repository contents".
• Select "Read-only" for "Issues" (issues and PRs are "closely related").
• Select "Read-write" for "Pull Requests".

In the newly created app, go to the "Install App" section and install it to your account. There you will be able to select the list of repositories to use with this app.

And don't forget to choose an avatar for your robot)

Step 2. Adding a token generator.

To solve the token expiration problem, we wrote a simple Ruby script (github-token) that requests a new token using the "Authentication as an installation" flow. This is how we use it in our CircleCI configuration:

danger:
  executor: ruby
  steps:
    - attach_workspace:
        at: .
    - run:
        name: Add github.com to known_hosts
        command: mkdir -p ~/.ssh && ssh-keyscan -H github.com > ~/.ssh/known_hosts
    - run:
        name: Danger review
        command: |
          DANGER_GITHUB_API_TOKEN=$(bundle exec .circleci/github-token) bundle exec danger
    - store_artifacts:
        path: tmp/brakeman

The source code of the github-token script could be found in the evilmartians/terraforming-rails repo.

The script itself requires the following information:

• a private key—go to the app's profile and generate one;
• your app ID—you can find it on the app's page as well;
• your installation ID—go to https://github.com/organizations/MY-ORG/settings/installations (or https://github.com/settings/installations for personal installations), click "Configure" for the app, and check the URL: it should have a form of .../installations/ID.

Put the information above to CircleCI environment using the following names respectively: GITHUB_APP_PRIVATE_KEY*, GITHUB_APP_ID, GITHUB_INSTALLATION_ID.

*CircleCI doesn't support multiline env values, so you can do, for example, the following to copy the flattened value:

$ cat private-key.pem | perl -p -e 's/\n/\\n/g' | pbcopy

The github-token script will take care of multilining the key back.

Modulirizing Dangerfile

By default, Dangerfile is the only entry point for all your review checks. Keeping everything in one file works great until it becomes too large, which usually happens very quickly.

On the other hand, most checks do not depend on each other. So, why not keeping each in a separate file?

That's exactly what we do:

• All checks are stored in Ruby files in the .danger/ directory.
• Dangerfile is responsible for executing all the checks and also contain some shared logic.

This is our Dangerfile:

# Shared consts
CHANGED_FILES = (git.added_files + git.modified_files).freeze
ADDED_FILES = git.added_files.freeze

Dir[File.join(__dir__, ".danger/*.rb")].each do |danger_rule_file|
  danger_rule = danger_rule_file.gsub(%r{(^./.danger/|.rb)}, "")
  $stdout.print "- #{danger_rule} "
  # execute each check using `eval`
  eval File.read(danger_rule_file), binding, File.expand_path(danger_rule_file)
  $stdout.puts "✅"
# allow a single check to fail without breaking others
rescue Exception => e
  $stdout.puts "💥"

  # make sure the result is a failure if some check failed to execute
  fail "Danger rule :#{danger_rule} failed with exception: #{e.message}\n" \
       "Backtrace: \n#{e.backtrace.join("\n")}"
end

The output looks like this:

$ bundle exec danger

- rails_credentials ✅
- missing_labels ✅
- brakeman ✅
- ruby_deps_inconsistency ✅
- updated_deps ✅
- missing_tests ✅
- merge_commits ✅
- db_schema_inconsistency ✅
- outdated_seeds ✅

Warnings:
- [ ] Are you sure we don't need to add/update tests for the main app?

Bonus: Danger meets Brakeman

One of the reasons I started to experiment with Danger was an idea to automate security checks with Brakeman. Simply running it on CI didn't fit my requirements:

• I needed to have a better way to see the results of the scanner run, preferably right on Github
• I didn't want Brakeman failures to make my builds "red" (it's not 100% accurate and could produce false negatives, especially when you're using a lot of metaprogramming).

Danger seemed to be the right tool for the job. It just didn't have a ready-made Brakeman integration (or plugin).

So, I wrote it by myself, and you can grab it here.

This check runs Brakeman programmatically (i.e., without CLI), generates HTML report and adds a link* to it along with the report summary to the final Danger result:

* We generate a link to the CircleCI build "Artifacts" tab, where the report could be found. The code is available in the Dangerfile.

In Conclusion

Danger proved to be a good code review companion. From now on I'm going to use in every project.

What do you think about Danger? Do you have some tips? Feel free to share in the comments!

P.S. Although we've open-sourced the code described above in the terraforming-rails repo, the better way to share it would be adding PRs to Danger itself and/or its plugins. So, if you want to contribute to OSS—you know what to do 😉.

Read more dev articles on https://evilmartians.com/chronicles!