Building a Development Environment for Cloud Engineering

Starting can be daunting. Before you take your first step, there’s a lot to consider, but you can prepare your development environment ahead of time to make your first steps in cloud engineering smooth and productive. In this article, we’ll cover how to set up your development environment to work across cloud providers, multiple languages, and different operating systems.

Building your toolbox

You might have heard of the French culinary term “mise-en-place,” which means laying out all the ingredients and cookware before starting to cook. That’s what we’re going to do here; layout all the accounts, authorizations, and software that you need to be a successful cloud engineer. We are going to need the following:

• A package manager: Every operating system has a package manager for installing software. Unlike binary installers, a package manager lets you manage all software packages, including updates. Package managers can help resolve dependencies, saving you from frustration.
• Cloud provider accounts: You can choose to set up one or multiple accounts. The important thing is how to configure credentials in your development environment,
• Programming languages: You can choose one or many, but they all have different versions and dependencies.
• Code editor: This is personal preference but make sure that it can perform code completion, error checking, and use enums. These features can be the difference between hitting Tab and searching through online documentation for a function.
• Pulumi: You can install Pulumi with a package manager and configure

Decisions

Before proceeding, you need to answer three questions:

• Which operating system to use for building cloud resources: macOS, Windows, or Linux?
• Which cloud provider are you using? This guide covers AWS, Azure, and Google Cloud.
• Which programming language will you use? Pulumi supports Node.js (JavaScript and Typescript), Python 3.6 or higher, Golang, and .NET (C#, F#, and VB).

Once you’ve made these choices, you can follow this guide in a choose-your-own-adventure style.

Package manager

Let’s start with the package manager. We’ll use it to install and manage all the software we need, including cloud provider CLIs, programming languages, editors, and Pulumi. Here's an example using macOS, but you can find examples for Windows and Linux here.

MACOS

Homebrew is the most popular package manager for macOS. The Command Line Tools (CLT) for XCode is required to install and build Homebrew. Install the XCode Tools first, then install Homebrew from the command line.

xcode-select --install
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

If you are uncomfortable with downloading and running an online shell script (with good reason), Homebrew provides an alternate installation method.

Setting up a cloud account

The first task is signing up for an account. Once you have that out of the way, the next steps are installing the CLI and configuring your credentials. We'll show examples using AWS here, but visit the original blog to see Azure and GCP.

macOS

We’ll use brew to install AWS CLI version 2 and verify if it’s installed by checking the version.

brew update && brew install awscli
aws --version

The next step is to create and download your AWS access keys and configure your environment to make them available to both the AWS CLI. To create your access keys, follow these directions for programmatic access.

Pulumi uses the AWS SDK to authenticate requests. Your AWS access keys are never sent to pulumi.com.

To configure the AWS CLI with your credentials, use aws configure:

aws configure
AWS Access Key ID [None]: <YOUR_ACCESS_KEY_ID>
AWS Secret Access Key [None]: <YOUR_SECRET_ACCESS_KEY>
Default region name [None]:
Default output format [None]:

This creates a ~/.aws/credentials file used by the AWS CLI to authenticate requests.

Windows

We’ll use chocolatey to install AWS CLI version 2 and check if it’s installed. Open a cmd Command Prompt as Administrator:

> choco install awscli
> aws --version

The install adds the AWS CLI client to the $PATH, so either open a new cmd window or use the refreshenv command to update the window’s environment variables and use the AWS CLI.

The next step is to create and download your AWS access keys and configure your environment to make them available to both the AWS CLI. To create your access keys, follow these directions for programmatic access.

Pulumi uses the AWS SDK to authenticate requests. Your AWS access keys are never sent to pulumi.com.

To configure the AWS CLI with your credentials, use aws configure:

> aws configure
AWS Access Key ID [None]: <YOUR_ACCESS_KEY_ID>
AWS Secret Access Key [None]: <YOUR_SECRET_ACCESS_KEY>
Default region name [None]:
Default output format [None]:

This will create the ~/.aws/credentials file used by the AWS CLI to authenticate requests.

Linux

We’ll use brew to install AWS CLI version 2 and check if it’s installed.

brew update && brew install awscli
aws --version

The next step is to create and download your AWS access keys and configure your environment to make them available to both the AWS CLI. To create your access keys, follow these directions for programmatic access.

Pulumi uses the AWS SDK to authenticate requests. Your AWS access keys are never sent to pulumi.com.

To configure the AWS CLI with your credentials, use aws configure:

aws configure
AWS Access Key ID [None]: <YOUR_ACCESS_KEY_ID>
AWS Secret Access Key [None]: <YOUR_SECRET_ACCESS_KEY>
Default region name [None]:
Default output format [None]:

This will create the ~/.aws/credentials file used by the AWS CLI to authenticate requests.

Choose a Programming Language

Pulumi supports Node.js (JavaScript and Typescript), Python 3, Golang, and .NET Core (C#, VB, and F#) languages. Examples below will be for macOS, visit the original post for examples on other operating systems.

Node.js

Use brew to install Node.js for JavaScript and Typescript.

brew install node

Python

MacOS includes Python; however, versions shipped before December 2019 have Python 2.7 installed, which is deprecated. Pulumi requires Python 3.6 or higher. Use brew to install Python, which installs it at /usr/bin/local/python3.

brew install python

In macOS versions with Python 2 installed, calling python uses the 2.7 binary. To ensure that you will always use Python 3, you can add an alias to your .bash_profile.

alias python=/usr/local/bin/python3

It is a best practice to create a virtual environment and activate it for a Python project. Pulumi creates a new venv environment when you start a new Project with pulumi new.

Golang

Use brew to install golang.

brew install golang

It is a best practice to create a local programming environment and set GOPATH as an environment variable, although it is not required for golang versions after 1.8. Pulumi creates a new local environment and go.mod to work with modules each time you start a new Project with pulumi new.

.NET

Install .NET with brew, Pulumi requires .NET Core 3.1 or higher.

brew install dotnet

Get a code editor

While knowing how to use vim or similar text editors is a worthwhile skill, you should use a modern code editor. Here’s a non-exhaustive list why you should use a code editor.

• Quickly navigating to a type
• Autocompletion when you can’t remember the names of all members by heart
• Automatic code generation
• Refactoring
• Organise imports
• Warnings as you type.
• Hovering over something to see the docs
• Keeping a view of files, errors/warnings/console/unit-tests and source code on the screen
• Running unit tests from the same window
• Integrated debugging
• Integrated source control
• Navigating to where a compile-time error or run-time exception occurred directly from the error details.

You can use an IDE (Integrated Development Environment) such as Microsoft Visual Studio, Xcode, or any one of JetBrains’ language-specific IDEs. Alternatively, you can use a lightweight solution with many of the features of an IDE. Popular code editors include Visual Studio Code, Sublime Text, and Atom.

You can install some editors with brew.

brew install --cask visual-studio-code
brew install --cask sublime-text
brew install --cask atom
brew install --cask pycharm-ce
brew install --cask goland

Install Pulumi

macOS Sierra (10.12) or later is required.

Homebrew

You can install Pulumi through the Homebrew package manager:

brew install pulumi

This will install the pulumi CLI to the usual place (often /usr/local/bin/pulumi) and add it to your path.

Subsequent updates can be installed in the usual way:

brew upgrade pulumi

Verifying your Installation

After installing Pulumi, verify everything is in working order by running the pulumi CLI:

pulumi version
v2.22.0

Next steps

Congratulations! You have a fully configured environment, and you’re ready to jump into cloud engineering. You can maintain your development environment using the package manager to add or update your toolset. Your code editor provides a modern development platform that takes advantage of all the advances in software engineering. Your coding experience will be more productive and less frustrating.

What are the next steps? Begin with Pulumi’s Getting Started. You can skip the configuration sections and jump straight into your first project. Once you’re done with your first project, try out example projects on Github. You can start with simple projects using the pulumi CLI, such as deploying a web server on AWS with python.

pulumi new https://github.com/pulumi/examples/tree/master/aws-py-webserver

This command will download the project from Github, create a virtual environment and activate it, and download all the python package dependencies. You’re ready to go and deploy with pulumi up. Want more? How about deploying Kubernetes on Azure with python?

pulumi new https://github.com/pulumi/examples/tree/master/azure-py-aks

You can use the examples as a starting point for building your cloud infrastructure and add resources documented on Pulumi’s API reference and Guides.