AWS + Gitlab — Leveling up security of your CICD platform.

AWS + Gitlab — Leveling up security of your CICD platform.

For eons there have only been three ways to “secure” your build platform or servers. All of them have been historically bad, for different reasons:

• On prem  — Running the on premise solution. With the whole deployment platform and conceivably your production software running also on premise. Obviously no one wants to do this, that’s why we invented the cloud in the first place.
• Use your own runners  — The runners can be locked inside your production cloud and call out to your build server to fetch waiting jobs. Your pipeline gets to use an instance profile which defines which roles it should be allowed to assume, and which permissions come with it.
• Use secure environment variables  — Inject environment variables into the runners by saving in your CI/CD platform access key and secret for a user. You can’t easily rotate this, and it can give far too much access. Still further you have an additional vulnerability for anyone who can get access to the variables, print them out, etc… Also you can’t really tell if they have been compromised.

Now you can stop using access key and secret today!

The solution

The perfect solution is if GitLab could directly authenticate with AWS and you give gitlab access directly to the resources it needs, in the context of your job. And now it exists.

Gitlab generates signed JWTs that you can use with AWS to get temporary access tokens. (You don’t need any nonsense with Hashicorp’s Vault, you’ll notice we can get this working without it!)

Gitlab’s tokens look like this:

Now it’s just a matter of setting up AWS to accept that token and allow it to generate an STS token.

Setup

We’ll be using AWS IAM’s AssumeRoleWithWebIdentity to convert the token into an identity

Create Identity Provider:

2. Create a role for that identity provider

Assign Role:

3. Call AWS STS at build time to get your credentials

AssumeRole CLI Docs

And it works?

Okay it almost works, but there is one very small problem. The aud property of the token isn’t valid! So you’ll actually get this error:
An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Missing a required claim: aud!!!
(Relevant Gitlab open issue)

So what can we do about it?

The complete solution

We just need to convert the JWT we get to one with the aud claim. It isn’t hard to do this by running a public service which accepts gitlab JWTs, there is a slightly easier way.

Jump through the quick set up in Authress, add a custom OAuth connection, and then swap the the incoming JWT with one that AWS will accept.

Create the connection

• Set the Authorization URL to be https://gitlab.com/oauth/authorize
• Make sure to set the Audience Identifier and the User ID Resolution , as the gitlab token has neither a sub nor an audience:

Create a new identity provider in AWS

Create a new connection in AWS that matches the Authress connection. The provider url will be your account domain from the connection configuration and the AWS Audience will be the connectionId from Authress.

Important: Update the Trust Policy to restrict the valid token sources:

Update your login script
Just need to swap out a couple of lines, also using js here as it is much simpler to execute that raw bash.

Originally published at https://authress.io on June 14, 2021.

---