DEV Community

codecowboydotio
codecowboydotio

Posted on

Pulumi - creating cloud infrastructure using yaml

#pulumi #iac #programming #devops

Intro

I've been fooling around with pulumi for a bit now, and thought I would write about it here.

What is it?

Pulumi is an infrastructure as code framework with a twist. You can use any language you like in order to write your code (within reason).

The pulumi tagline is Your cloud. Your language. Your way.

I'm here to say it's true!

Why is it cool?

Pulumi is super cool because it means as a developer I can write infrastructure as code for my cloud based infrastructure in a language I am comfortable with.

This is a HUGE DIFFERENCE to learning an opinionated hybrid language for IAC.

Interestingly it also means I can write tests for my IAC in the same language I'm used to writing in. This is also a massive benefit.

I do not cover it here, but look out for another post soon.

Installation

Installation is simple, I am using AWS and will follow the docs at https://www.pulumi.com/docs/get-started/aws/begin/.

Install pulumi

In order to install pulumi, just as the docs say, use the following command:

curl -fsSL https://get.pulumi.com | sh
Enter fullscreen mode Exit fullscreen mode

You will also need to install your language runtime - good news - for YAML, there is no language to install.

Next configure your AWS account in the normal way that you would

export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>
Enter fullscreen mode Exit fullscreen mode

Create a new project

In order to get up and going once pulumi is installed you will need to install a new project.

To do this you use the commands:

mkdir quickstart
cd quickstart
pulumi new aws-yaml
Enter fullscreen mode Exit fullscreen mode

By default this will create a project that has two files:

  • Pulumi.yaml: This contains the program that manages your resources 
name: quickstart
runtime: yaml
description: A minimal AWS Pulumi YAML program

resources:
  # Create an AWS resource (S3 Bucket)
  my-bucket:
    type: aws:s3:Bucket

outputs:
  # Export the name of the bucket
  bucketName: ${my-bucket.id}
Enter fullscreen mode Exit fullscreen mode
  • Pulumi.dev.yaml: This contains project level configuration 
config:
  aws:region: ap-southeast-2
Enter fullscreen mode Exit fullscreen mode

Default Project

The default project is fairly simple to understand and creates a bucket with an output.
Let's look at what else we can do that's useful!

A more complex example

A more complex example requires a more complex explanation.
What if I want to create something more akin to a real world use case in AWS?

What if I want to create a VPC, subnet, internet gateway, instance and security groups as a separate stack?

The image below is exactly what we are going to create.

Image description

AWS VPC

To create an AWS VPC we have the following code:

name: aws-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program

configuration:
  project_name:
    type: String
    default: pulumi-yaml
  vpc_name:
    type: String
    default: ${project_name}-vpc
resources:
  main_vpc:
    type: aws:ec2:Vpc
    properties:
      cidrBlock: 10.100.0.0/16
      tags:
        Name: ${vpc_name}
Enter fullscreen mode Exit fullscreen mode

The code above does a few things that are interesting.

It sets up some variables that are used throughout the rest of the codebase.
The variable project_name is set as a type string, and used defaults to pulumi-yaml. This is a standard yaml way of setting a variable.

The variable vpc_name is set as a type string, but uses the previously defined variable, project_name.
This is a good example of how to use variables in yaml if you aren't familiar.

The resources section builds the AWS resources for me. Each resource has a type and properties. These are defined as part of the API documention in pulumi.

In my case, I am creating a VPC, with a CIDR block and tagging it.

To run this code we use the command:

pulumi up -y
Enter fullscreen mode Exit fullscreen mode

This produces:

     Type                 Name          Status
 +   pulumi:pulumi:Stack  aws-yaml-dev  created
 +   └─ aws:ec2:Vpc       main_vpc      created
Enter fullscreen mode Exit fullscreen mode

AWS Subnet

The next step is to create an AWS subnet. This allows me to further carve up the VPC supernet.

The code for the next portion is as follows:

  main_subnet:
    type: aws:ec2:Subnet
    properties:
      vpcId: ${main_vpc.id}
      cidrBlock: 10.100.1.0/24
      mapPublicIpOnLaunch: True
      tags:
        Name: ${project_name}-subnet
Enter fullscreen mode Exit fullscreen mode

Note that the type is different - the object is again a different API type.
The vpcId is important as well. I am using the vpc object name from my previous piece of code, and the calulated id. This allows pulumi to calculate the vpc ID that was created in the previous step.

The output from running this looks like this:

     Type                 Name          Status
     pulumi:pulumi:Stack  aws-yaml-dev
 +   └─ aws:ec2:Subnet    main_subnet   created
Enter fullscreen mode Exit fullscreen mode

Internet Gateway

The next step is to create an internet gateway for my new VPC. This will allow internet connectivity in and out of my VPC.

  internet_gateway:
    type: aws:ec2:InternetGateway
    properties:
      vpcId: ${main_vpc.id}
      tags:
        Name: ${project_name}-igw
Enter fullscreen mode Exit fullscreen mode

Again I use the vpc ID and tag the asset with a tag that includes my project name that I defined above.

When I run this the output is as follows:

     Type                        Name              Status
     pulumi:pulumi:Stack         aws-yaml-dev
 +   └─ aws:ec2:InternetGateway  internet_gateway  created
Enter fullscreen mode Exit fullscreen mode

Route table and Association

The next steps are to create both a route table and associate that route table with the internet gateway created above, as well as a subnet.

  route_table:
    type: aws:ec2:RouteTable
    properties:
      vpcId: ${main_vpc.id}
      routes:
        - cidrBlock: 0.0.0.0/0
          gatewayId: ${internet_gateway.id}
  route_table_assoc:
    type: aws:ec2:RouteTableAssociation
    properties:
      subnetId: ${main_subnet.id}
      routeTableId: ${route_table.id}
Enter fullscreen mode Exit fullscreen mode

Again, the object identifier of vpc ID in object notation is used. I also use the internet gateway ID as well. For the route table association, I use the subnet identifier created above and the route table id.

When I run this, the output is as follows:

     Type                              Name               Status
     pulumi:pulumi:Stack               aws-yaml-dev
 +   ├─ aws:ec2:RouteTable             route_table        created
 +   └─ aws:ec2:RouteTableAssociation  route_table_assoc  created
Enter fullscreen mode Exit fullscreen mode

Security Group

The next step is to create a security group and associate that security group with a VPC.

  security_group:
    type: aws:ec2:SecurityGroup
    properties:
      vpcId: ${main_vpc.id}
      ingress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      egress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      tags:
        Name: ${project_name}-sg
Enter fullscreen mode Exit fullscreen mode

The output from this step when I run it is:

     Type                      Name            Status
     pulumi:pulumi:Stack       aws-yaml-dev
 +   └─ aws:ec2:SecurityGroup  security_group  created
Enter fullscreen mode Exit fullscreen mode

Instance

The final step here is to create an instance within the newly created VPC and security groups.

  web:
    type: aws:ec2:Instance
    properties:
      ami: ${ubuntu.id}
      instanceType: t3.micro
      tags:
        Name: ${project_name}-instance
variables:
  ubuntu:
    Fn::Invoke:
      Function: aws:ec2:getAmi
      Arguments:
        mostRecent: true
        filters:
          - name: name
            values:
              - ubuntu/images/hvm-ssd/ubuntu-focal-20.04-*
          - name: virtualization-type
            values:
              - hvm
          - name: architecture
            values:
              - x86_64
Enter fullscreen mode Exit fullscreen mode

The first part of the YAML creates an instance, and has the instance identifier and size as well as providing tags for the instance.

The instance identifier is a variable.
The variable is updated with the output of a function.

This occurs dynamically.

The function calls the AWS API and gets the AMI based on the filters provided. This is set as a variable which can then be queried and walked through. Note that I use the identified ubuntu.id. This is the returned value from the function that is placed in the variable.

When I run this, I get the following output:

     Type                 Name          Status
     pulumi:pulumi:Stack  aws-yaml-dev
 +   └─ aws:ec2:Instance  web           created
Enter fullscreen mode Exit fullscreen mode

End to end

End to end the entire yaml file looks like this:

name: aws-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program




configuration:
  project_name:
    type: String
    default: pulumi-yaml
  vpc_name:
    type: String
    default: ${project_name}-vpc
resources:
  main_vpc:
    type: aws:ec2:Vpc
    properties:
      cidrBlock: 10.100.0.0/16
      tags:
        Name: ${vpc_name}
  main_subnet:
    type: aws:ec2:Subnet
    properties:
      vpcId: ${main_vpc.id}
      cidrBlock: 10.100.1.0/24
      mapPublicIpOnLaunch: True
      tags:
        Name: ${project_name}-subnet
  internet_gateway:
    type: aws:ec2:InternetGateway
    properties:
      vpcId: ${main_vpc.id}
      tags:
        Name: ${project_name}-igw
  route_table:
    type: aws:ec2:RouteTable
    properties:
      vpcId: ${main_vpc.id}
      routes:
        - cidrBlock: 0.0.0.0/0
          gatewayId: ${internet_gateway.id}
  route_table_assoc:
    type: aws:ec2:RouteTableAssociation
    properties:
      subnetId: ${main_subnet.id}
      routeTableId: ${route_table.id}
  security_group:
    type: aws:ec2:SecurityGroup
    properties:
      vpcId: ${main_vpc.id}
      ingress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      egress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      tags:
        Name: ${project_name}-sg
  web:
    type: aws:ec2:Instance
    properties:
      ami: ${ubuntu.id}
      instanceType: t3.micro
      tags:
        Name: ${project_name}-instance
variables:
  ubuntu:
    Fn::Invoke:
      Function: aws:ec2:getAmi
      Arguments:
        mostRecent: true
        filters:
          - name: name
            values:
              - ubuntu/images/hvm-ssd/ubuntu-focal-20.04-*
          - name: virtualization-type
            values:
              - hvm
          - name: architecture
            values:
              - x86_64
Enter fullscreen mode Exit fullscreen mode

When I run it from the beginning, I get the following output:

     Type                              Name               Status
 +   pulumi:pulumi:Stack               aws-yaml-dev       created
 +   ├─ aws:ec2:Vpc                    main_vpc           created
 +   ├─ aws:ec2:Instance               web                created
 +   ├─ aws:ec2:SecurityGroup          security_group     created
 +   ├─ aws:ec2:Subnet                 main_subnet        created
 +   ├─ aws:ec2:InternetGateway        internet_gateway   created
 +   ├─ aws:ec2:RouteTable             route_table        created
 +   └─ aws:ec2:RouteTableAssociation  route_table_assoc  created

Resources:
    + 8 created

Duration: 39s
Enter fullscreen mode Exit fullscreen mode

Cleanup

Cleanup is just as easy.

pulumi delete -y
Enter fullscreen mode Exit fullscreen mode

This deletes all of the resources that have been created.

     Type                              Name               Status
 -   pulumi:pulumi:Stack               aws-yaml-dev       deleted
 -   ├─ aws:ec2:RouteTableAssociation  route_table_assoc  deleted
 -   ├─ aws:ec2:RouteTable             route_table        deleted
 -   ├─ aws:ec2:Subnet                 main_subnet        deleted
 -   ├─ aws:ec2:SecurityGroup          security_group     deleted
 -   ├─ aws:ec2:InternetGateway        internet_gateway   deleted
 -   ├─ aws:ec2:Instance               web                deleted
 -   └─ aws:ec2:Vpc                    main_vpc           deleted

Resources:
    - 8 deleted

Duration: 1m2s
Enter fullscreen mode Exit fullscreen mode

Conclusion

Pulumi rocks!

I don't know how else to say it.

I can use a multitude of languages to create my cloud resources - languages that I'm familiar with and like.

This example is YAML - which isn't quite a language, but has some characteristics of a language. I learned a few things about YAML and syntax through this process, and hopefully you did too.

Look out for my next post where I will walk through pulumi and python.

Top comments (0)

Read next

karadza profile image

DevOps vs Platform Engineering

Juraj -

cordlesswool profile image

How to use generics in pipe-and-combine

Wolfgang Rathgeb -

kaankaya profile image

Dirty Code: Simple Rules to Avoid It

Kaan Kaya -

colindickson_78 profile image

Using Weak Pointers in Go

Colin Dickson -