In this podcast episode, Krish explores the topics of platform engineering and Terraform. He starts by discussing Humanitech, a tool that streamlines platform engineering processes. Krish then dives into Terraform, an infrastructure as code tool, and demonstrates how to get started with it. He covers the basics of Terraform configuration, provisioning and destroying infrastructure, using variables, and working with the AWS provider. Krish also explores the resources available in Terraform and compares it to AWS SAM, another infrastructure as code framework. In this podcast episode, the host continues reviewing the build infrastructure section of Terraform. They explore resource blocks and the unique ID for resources. The host emphasizes a pragmatic approach to learning and highlights the benefits of hands-on experience. They also provide a recap of the progress made so far, including modifying the Terraform file and working with variables. The episode concludes with a promotion of the host's APIs and a call to action for listeners to try them out.
Takeaways
- Humanitech is a tool that helps streamline platform engineering processes.
- Terraform is an infrastructure as code tool that allows you to define, provision, and manage infrastructure resources.
- Terraform configuration consists of providers, resources, and variables.
- You can use Terraform to provision and destroy infrastructure, and manage resources across different cloud providers.
- AWS API Gateway is a service that allows you to create, deploy, and manage APIs. Resource blocks in Terraform consist of a resource type and name, which form a unique ID for the resource.
- Taking a pragmatic approach to learning, such as hands-on experience, can be more effective than solely relying on documentation and theory.
- Using variables in Terraform files allows for more flexibility and avoids hard coding values.
- Promoting APIs and encouraging listeners to try them out can help streamline backend system development.
Chapters
00:00 Introduction and Background
02:18 Exploring Humanitech
06:07 Introduction to Terraform
07:06 Getting Started with Terraform
13:37 Understanding Terraform Configuration
19:05 Provisioning and Destroying Infrastructure
23:16 Using Variables in Terraform
28:08 Working with AWS Provider
38:42 Exploring Terraform Providers and Resources
45:46 Understanding AWS API Gateway
49:30 Comparing Terraform and AWS SAM
51:43 Reviewing the Build Infrastructure Section
53:05 Taking a Pragmatic Approach to Learning
54:32 Recap of Progress So Far
55:02 Using Variables in Terraform Files
56:03 Ending the Podcast and Promoting APIs
Transcript
Krish (saas.snowpal.com) (00:01.282)
Hey there, I hope you’re doing well. Welcome to Snowpals Software Development and Architecture podcast. In the podcast I published, we published yesterday, I had a conversation, a very interesting conversation with James about platform engineering. We skimmed the surface and learned a fair number of things about platform engineering. After the podcast, I was wanting to check some of it out for myself, you know, do a bit more hands-on.
a follow up to that podcast. So I looked at something and before I spend any time digging deeper into it, I figured it might just be worth exploring this together in another podcast. So I went to the page, spent like two or three minutes and then before I could, you know, before I tried to understand any more about it, I told myself, you know what, let me do that in a call, which would be a good follow up to the really interesting conversation that I had with James yesterday.
So shout out to James for taking the time to have the chat that gave me the opportunity to sort of keep chugging along from where we actually left off. Without further ado, let me share my screen and we can actually get started here.
Krish (saas.snowpal.com) (01:20.878)
and share my whole desktop.
Krish (saas.snowpal.com) (01:27.988)
Okay, for some reason it’s taking a few seconds here. Let me actually try, let me stop sharing and let me reshare.
Krish (saas.snowpal.com) (01:40.882)
Okay, I think it works now. So let’s go here. So make sure, yep.
Krish (saas.snowpal.com) (01:50.562)
Okay. One of the things when I was searching for it, I stumbled into this product, Humanitech. As it turns out, it’s probably very related to the platform engineering conversation. I don’t know this company, the product. When I did the search, it popped up and before I spent more time or any time, like I mentioned earlier, I figured I’ll just do it on the podcast here.
So let’s learn together. So this tool, it says build a perfect platform and it says standardized with the platform orchestrator. And this is what, let’s go scroll down here. It says drop hours of manual ticket operations and overwhelm developers. Overwhelm developers, standardize all your configs, reduce waiting times and error rates, slash your lead time, right? So if we looked at this, it says today, how you might possibly be doing things.
And it says with Humanitech, it becomes a bit more streamlined. I’m not going into the details of these boxes here, but you know, just looking at the picture, as a layman, you can tell that this looks more complicated than this does. This looks a lot more streamlined. You have the developer, you have the platform engineer. It’s, let’s see what this is. It says define workloads and resource dependencies with the score workload specification.
The platform engineer define how resources are provisioned with resource definitions and criteria specifying where they apply. It goes through the orchestrator, and then you have the config creator infrastructure matched, and then the workload deployed. I’m reading that. Don’t exactly know what each of those things mean, except it does resonate with the conversation again. In the chat, we talked about how the roles
how the role of a developer might change, going from them doing some DevOps work to doing, you know, integrating with the platform engineering team. So clearly they have support for a number of different platforms and here’s another diagram here. Maybe we can take a closer look at this one. So are there any differences between these providers? It’s probably the same architecture. So internal dev platform, I guess on…
Krish (saas.snowpal.com) (04:14.754)
I don’t know if there’s any variations when I click through, they look quite the same except for the heading here. Maybe there are some differences. Service catalog and API catalog developer portal, and it has GitHub score. I don’t know what score is. I wonder if it’s specific to Humanitech. I’m not sure. Yep, it does that product looks like. It says design sensible abstractions, okay. What?
And then we have the integration and delivery plane here, monitoring and logging plane, security plane, the resource plane, okay. This all looks cool, except that I, now I’m not quite sure I’m understanding it because we haven’t done anything hands-on. So let’s try to do it. And then I looked up the price also, as you can see it’s $10,000 a year. I don’t know if they have a free trial for us to check out, but after I did that,
actually followed the trail and it took me to reference architecture for AWS. And it says, spin up your own Humanitech AWS reference architecture implementation. When I clicked that, it brought me here. And again, they have the same diagram here. Don’t know if there’s a free trial variation, but if we notice one of the first things is we need the AWS CLI.
given that all our APIs are needed, please API gateway uses CLI all the time. So I have that actually running. Terraform, no, I don’t have it. And I haven’t done any work on Terraform, which is where I was like, you know what, maybe this podcast, we could start exploring Terraform. You may have heard of infrastructure as code and Terraform, if I’m not wrong, is probably one of the biggest players if not the biggest player in that space.
It says Terraform is an infrastructure as code tool that lets you build, change and version infrastructure safely and efficiently. This includes low level components like compute instances, storage and networking and high level components like DNS entries and SaaS features. You know what you’d have done using a user interface with something like Terraform, you can do it through configuration. So that’s fundamentally, I understand where this fits in the big picture.
Krish (saas.snowpal.com) (06:36.022)
but I have not actually used it. So maybe we can take this opportunity to kind of get started and see what we need to do to get this working. So let’s say I was here and I Googled and I think I went into developer, Terraform, tutorials, state and import. And it says there’s some instructions here.
Why don’t we just get started? I’m gonna pull my machine closer. We are like what, what seven minutes in for that super high level intro. So now we can actually do something that’s a bit more hands-on. First things first, let’s say I picked the community edition. It says, I assume that you’re familiar with the workflow. Unfortunately, I’m not. If you’re new to Terraform, complete the get started tutorials.
First, uh…
There it is, okay. We should go through this. AWS, I wanna play that video there. It says IAC allows you to manage the infrastructure with config files rather than through a graphical user interface. If you remember this kind of what I said like a few minutes ago, right? That’s where I knew that this would be, this would fall in that category. It allows you to build, change, and manage your infrastructure in a safe, consistent, repeatable way by defining resource configurations that you can version, reuse, and share.
Now, between DevOps and platform engineering, I don’t know where Terraform falls. I don’t know if you’ll have more to do with Terraform directly if you’re a DevOps engineer versus if you’re a developer integrating with a platform engineering team. I reckon it’s the former, but I’m not sure. I’m speculating at this point. It says Terraform. Oh, so hash. E-carb. Isn’t that? OK, now.
Krish (saas.snowpal.com) (08:37.89)
for some reason I thought it was Humana techno. So Terraform is a HashiCorp’s infrastructure as code tool. It lets you define blah, blah. Can manage infrastructure on multiple cloud platforms. That’s all great. To deploy infrastructure Terraform, identify the infrastructure for your project, write the configuration, initialize the plugins, preview the changes and make the plan changes. I mean, let’s get started and see how far we can go.
Let’s actually start with installing this. I’m gonna open a window. I’m gonna use the same sort of terminals that we had. Actually, you know what, let me go, you tab.
Krish (saas.snowpal.com) (09:24.002)
Simply because that was what we use for the Salesforce series. I don’t just want to pick a different terminal. Just to there, when we go back to that. Okay, podcast.
Krish (saas.snowpal.com) (09:40.073)
Let’s call it platform engineering. Let’s call this Terraform.
Krish (saas.snowpal.com) (09:52.926)
Okay, not that it matters as to where we ran the brew tap, but it’s all right. I just want to be in a more appropriate directory.
Krish (saas.snowpal.com) (10:10.063)
Okay, now that we’ve done the brew tab, let’s do the brew install.
Krish (saas.snowpal.com) (10:20.658)
Updates your Xcode is outdated. Hopefully.
Krish (saas.snowpal.com) (10:31.118)
I don’t know if it’s going to require that we have the latest version of Xcode.
Krish (saas.snowpal.com) (10:45.102)
because Xcode install will take a fair bit of time. So I hope I don’t have to do it right.
Krish (saas.snowpal.com) (11:00.31)
Um, it’s hanging there. Yeah, maybe it just took some time. So probably I don’t have to do the Xcode update right now. Okay, so we’ve done the brew tab, hasher corp tab and then brew install. To update to the latest version, first update homebrew. Think I was on the latest version, but maybe not. Okay, but that’s what I thought.
Krish (saas.snowpal.com) (11:32.224)
Since we picked up the latest version, I guess we are good. Now we have Terraform.
Krish (saas.snowpal.com) (11:41.714)
Okay. And if you don’t know what, you know, you’ve never used it, then we’re good because I’ve never used this as well. So we are at the same point.
Krish (saas.snowpal.com) (11:56.1)
It says enable tab completion.
Krish (saas.snowpal.com) (12:09.154)
Docker, I actually have Docker installed because we use Docker at Snowpile. Reinstall, stand Docker desktop, okay.
Krish (saas.snowpal.com) (12:40.871)
Okay, let’s create this directory. Learn Terraform Docker container. See you to that. Create a file called main.tf and paste the following, okay.
Krish (saas.snowpal.com) (13:00.267)
istf or terraform.
Krish (saas.snowpal.com) (13:05.742)
Copy this.
Krish (saas.snowpal.com) (13:13.642)
required providers, Docker, Docker image engine X, Docker container, internal external ports. Okay. Initialize a project which downloads a plugin called a provider that lets Terraform interact with Docker. Initialize it.
Krish (saas.snowpal.com) (13:37.674)
Terraform has been successfully initialized. You may now begin working with Terraform.
Krish (saas.snowpal.com) (13:45.546)
That’s actually.
go into that directory here as well.
Krish (saas.snowpal.com) (13:54.242)
Okay, let’s go to Docker container here.
Let’s see. This is stuff we did, I think, in one of the other podcasts. Just saying, because I killed Docker before I started this session.
Krish (saas.snowpal.com) (14:26.466)
I’m gonna run this, it’s not related to…
Krish (saas.snowpal.com) (14:33.31)
not necessarily related to what we’re doing here, but yeah, okay. Let’s go back here.
Krish (saas.snowpal.com) (14:44.578)
provision the Nginx Server container with apply when Terraform asks you to confirm type as enter. Okay, Terraform apply.
Krish (saas.snowpal.com) (14:57.814)
Let’s see what this says here. Docker container engine links will be created. Blah, blah, blah. Docker image engine links will be created. And yes.
Krish (saas.snowpal.com) (15:15.59)
Okay, let’s go back here. Okay, now we see Nginx. So that’s as part of the command that we just ran. So if you remember from like two minutes ago, we did not have this. Okay. Verify that it’s running by going, okay, welcome to Nginx. Try to see if I should pull it open a different window. That’s all right. Okay. Now we have, I think we are, actually, let me do this.
a separate window then we’ll go back here okay moving to the last one that’s that was not the window
Krish (saas.snowpal.com) (16:09.146)
I’m wondering where, I forgot which tab we were on.
It’s not this one.
Certainly not that one Not that one either
Krish (saas.snowpal.com) (16:37.646)
Hang on, do I have another window here? That’s the window I’m recording.
Krish (saas.snowpal.com) (16:48.554)
Yep, it was that render. Okay, it’s a little bit confusing. Okay.
Krish (saas.snowpal.com) (16:55.118)
Copy this go here. I think this is the last tab so we can follow the trail
Krish (saas.snowpal.com) (17:05.467)
So far what we’ve done, just a quick recap, is we brew installed, where are those two brew install commands?
Krish (saas.snowpal.com) (17:26.354)
Okay, maybe it was on another page, I’m not sure. So we did, let me actually do a history here. So we did a brew tap, hash carb tap, and we brew install terraform, and then we ran this command, install autocomplete. We opened Docker, we created a new directory, seeded into that, created a main.tfmpt file, and then stuck the contents of the file, copy pasted from their example. And then we did a terraform, init and apply.
at which point…
Krish (saas.snowpal.com) (18:03.194)
After we hit Terraform apply, we went to port 8000 and we see that the Nginx web server is successfully installed and running. And now let’s do Docker, oops.
Krish (saas.snowpal.com) (18:21.678)
Dynamo is, I mean, that’s, you know, we have, I have a local Dynamo to be running on a Docker container for the Snowpile API gateway work. That’s, you can ignore that one. What we’re interested in is this line item here, right? And where is the command?
Krish (saas.snowpal.com) (18:51.538)
should make this okay um don’t see the whole command here but that’s what this one is to stop that we’ll do a terraform destroy
Krish (saas.snowpal.com) (19:05.506)
Let me see, oops, just right, add a typo there. Now if I run Docker PS, we see that it’s gone, right? We don’t have that line item does not actually show up. So I’m gonna do a Terraform, apply one more time. Let’s say yes.
Krish (saas.snowpal.com) (19:27.726)
Now we run the previous command and we see that back here. Okay. We have that container running now. You’ve provisioned and destroyed engineering server with Terraform. Yep, okay. So we didn’t run the engineering commands to start or stop separately. We provisioned the engineering web server via Terraform essentially, right? That’s what it’s saying here. Now let’s look at that file, the cat.
Let’s look at all of these files actually, terraform, tf.state.
Krish (saas.snowpal.com) (20:05.25)
So when we destroy it, I wonder if…
Krish (saas.snowpal.com) (20:16.146)
Okay, that’s what it looks like when it’s destroyed. And now when we apply
Krish (saas.snowpal.com) (20:26.47)
Outputs empty, resources empty.
Krish (saas.snowpal.com) (20:35.038)
resources, outputs empty resources is not empty. It says Docker container, name engine X. It’s got a bunch of different attributes. It’s all the defaults because we didn’t make any changes there. Just trying to see if there’s anything interesting. So there are two elements. Let’s actually copy this. I have to read this here.
Krish (saas.snowpal.com) (21:00.77)
Let’s actually take that to a JSON formatter.
Krish (saas.snowpal.com) (21:14.062)
Okay, there are two resources. First one’s Nginx. So it’s type is Docker image. The first one is a Docker container and the second one is a Docker image. So in the Docker container, it’s got a bunch of these attributes. It’s got an entry point slash Docker entry point.sh. Okay, we can learn the specifics here as we go. Second one is a Docker image.
Um…
Okay, so I want to quickly see that to see what it had in there. Okay, there’s another file as well. That’s also a JSON file, Terraform.
Krish (saas.snowpal.com) (22:04.558)
Oh, backup interesting. So let’s say we destroy it. I wonder if the backup will now have the larger file with everything else. Yep, it sure does, right? So it just has the previous one, the backup version. So if you want to recover or something along those lines, I reckon, okay. Now the backup is going to…
Krish (saas.snowpal.com) (22:28.374)
Change, yep, to that empty version. And then the non-backup file is gonna have everything, okay? Also wanna see the main.tf. We’ll learn what this Terraform syntax is, the configuration, but I see Terraform.
One, two, three, provider Docker resource, resource doc. So we have two resources, Docker image and Docker container. And that’s probably why when we restarted, we saw both of them. Why don’t we actually marker on with this for a tiny bit? Go here.
I’m gonna actually terraform commenting, terraform config add comments.
It’s probably the typical slash star, but let’s make sure. Yep, that’s what it did. Okay, let’s just do this.
Krish (saas.snowpal.com) (23:44.135)
Now if we destroy…
Krish (saas.snowpal.com) (23:52.718)
Okay. We have Docker image, we commented out Docker container. Now I’m trying, I wanna see when we do the cat backup.
Krish (saas.snowpal.com) (24:10.446)
got Docker image and Docker container and then the Terraform state, this is actually empty. Now let’s say if we wanted to re-initialize, let’s do apply.
Krish (saas.snowpal.com) (24:28.15)
See it says only one resource added. And if you do this, yep, we see only Docker image and not the Docker container. Right, I just want to, so that’s what this is driven by our configuration here in main.tf in the terraform.tf file. So resources, whatever resources we need, we need a Docker image, we need a Docker container. And then each of them has more specifics. So from what I can tell here,
It has provider and resource. So I would go to Terraform, config provider versus resource or something along those lines. A provider is a plugin that offers a collection of resource types, blah, blah.
Krish (saas.snowpal.com) (25:21.714)
Just trying to go look at the documentation here.
Krish (saas.snowpal.com) (25:28.879)
syntax configuration syntax
Krish (saas.snowpal.com) (25:34.758)
Block is a container for other content, identifiers. So this is their page. So if you notice, you know, watch the other podcasts, this is typically how we approach problems at Snowpal. We’re not going starting from the documentation because you know, a lot of these tools are brilliantly documented, but it’s a double-edged sword. What it also means is you’re gonna have to spend a number of hours actually getting up to speed. And sometimes there’s a whole lot more than what you might need to just crash the surface.
So this is, again, there’s no right or wrong way, but this is our preference. Now that we’ve kind of understood the main.tf file, we come back here, search for it, and see what constitutes that file. So here, there are examples I’m gonna speak to. You know, as we understand Terraform, as we get to understand it more, we’ll know that there is a lot that file would comprise. But before we get deeper, at least we now know
Looking at this basic, basic version, there is a provider and there are resources and we have two resources essentially. Let’s see, even if I search for something like keep local.
Krish (saas.snowpal.com) (26:46.542)
I’m surprised it doesn’t.
Krish (saas.snowpal.com) (26:52.234)
I’m trying to see how to get to the rest of the document. Let’s say I want to say Docker container.
Krish (saas.snowpal.com) (27:01.422)
documentation, define input variables. Yep, this one looks exactly, we copied from this page, not really, right? But it looks quite the same. Docker image and Docker container. It says variable container name. Instead of it being tutorial.
you change it to variable dot container name. Let’s try that it’s actually got a little bit more create a new file called variables dot tf. Okay, let’s do that.
Krish (saas.snowpal.com) (27:44.654)
So we have a variable called container name. It’s got a description, default and a type.
Okay, we create a container name. Now we can actually go in and do variable substitution. So we can go into main.tf, change that name.
Krish (saas.snowpal.com) (28:08.287)
variable that container needs.
Krish (saas.snowpal.com) (28:13.046)
Now let’s apply the configuration.
Krish (saas.snowpal.com) (28:22.098)
one resource ads because there’s only one modified essentially. What was our variable name? Sorry.
Krish (saas.snowpal.com) (28:34.722)
container name example nginx container. If we go here.
Krish (saas.snowpal.com) (28:49.502)
Just trying to see where this would…
Krish (saas.snowpal.com) (28:56.922)
Oh, here. Okay. You see here, the example engine container, right? So we’ve given a custom name here by modifying the main.tf, but we didn’t modify directly. We have a variables.tf file. And then we had, you know, we defined declared it. Let’s say cat variables. Oops.
Krish (saas.snowpal.com) (29:19.114)
We have the variable defined here with a description type and a default value and it gets picked up. It does a variable substitution, right? Okay, that doesn’t hurt to know.
Krish (saas.snowpal.com) (29:33.822)
So if you wanted to override it, probably you could do something like this.
Krish (saas.snowpal.com) (29:43.51)
yet another name. So what we did is we actually overrode it by providing a value in the command line essentially, right? So there’s some more, if I could call it fancy stuff that you could potentially do there. So quick recap, right? We got Terraform installed, we were able to apply. We just, we looked at the file, we added the main.tf file and then the variables.terraform file. We saw the Terraform state file that was generated.
And then the backup version, every time we click apply and destroy, it puts the previous one in backup and it’s cyclical, right? So that gives us a way for us to recover. So we know what the previous configuration was. So we are actually moving along.
Krish (saas.snowpal.com) (30:35.886)
Okay, Nginx is running, it’s running through Terraform. Next is you’ll create real infrastructure in the cloud of your choice. Cloud of our choice, add Snowpile is typically AWS. Okay, you need the Terraform CLI installed, which we believe.
Krish (saas.snowpal.com) (30:57.61)
Isn’t that what we did? We did install the CLI. AWS, I happen to have it. AWS account and associate credentials. It says, set the environment variable. Now set your secret access key.
I have to do it after stopping my screen share.
set the AWS access key environment variable, set your secret key. So you need to set the access key and the secret key. Says this will provision resources that qualify under the free tier. Okay. Okay, let me do that. Let me set, let me do a stop share.
for a minute, then go back and set the AWS.
Krish (saas.snowpal.com) (31:56.394)
I’m going to, you know, AWS credentials directory on my machine, because we use AWS, we have AWS CLI all the time for our work. I have the credentials. I’m just going to set those two exports. That’s the only thing I’m going to be doing here. Okay. Let me do that really quickly.
Krish (saas.snowpal.com) (32:23.298)
Setting the AWS key ID.
Krish (saas.snowpal.com) (32:27.719)
in the secret.
Krish (saas.snowpal.com) (32:31.795)
Secret access key.
Krish (saas.snowpal.com) (32:36.878)
Okay, I’ve said those two things. Now let me do a screen share again.
Krish (saas.snowpal.com) (32:51.954)
Okay, now that I’ve said that, I’m going to see the set of files used to describe infrastructure and Terraform is known as Terraform config. Okay, we did. We already have this file. So we’re just going to modify, just going to modify the file. Maybe I should just open this in VS code might be easier.
Krish (saas.snowpal.com) (33:22.966)
Close all that. I’m just going to open Coto, broadcast, platform engineering, Terraform. OK, let’s open main.
I guess I’m sure there is like an extension that we could possibly, a plugin, yeah, let’s install the.
bunch of terraform plugins let’s see if there’s anything for AWS there’s an autocomplete
snippets blah okay
Krish (saas.snowpal.com) (34:07.278)
And now it recognizes the syntax. So this is what we had so far. Now it’s saying, my user is specific to the US West, blah, blah. Let’s copy this. Let’s go here.
Krish (saas.snowpal.com) (34:32.182)
Okay, I’m going to merge these two. So required providers, AWS. Okay, we only have the required provider sections. I’m just going to copy AWS.
Krish (saas.snowpal.com) (34:50.83)
Let’s take it here. OK.
Krish (saas.snowpal.com) (35:00.126)
So anywhere it starts. Okay, required version of Terraform. We don’t have a required version here, but let’s just add it.
Provider AWS, yeah Docker, we’re gonna add the AWS provider.
Krish (saas.snowpal.com) (35:23.662)
Okay, instead of West2, maybe AWS regions. Maybe I’ll just pick maybe East1. But hang on, I thought it said something to the effect here. EMI ID is specific to us West2. You know what, let me not change anything. Let me just stick to what’s here. Just copy the provider.
We put it there, that’s done. And then the resource, AWS instance.
Krish (saas.snowpal.com) (36:00.77)
We had.
I thought it looked a little bit different when I saw it. No, I think it is resource Docker image engine resource Docker container engine. I’m trying to understand the resource syntax here. A little bit curious because it’s Terraform resource attribute.
Krish (saas.snowpal.com) (36:34.942)
resource type dot name, but there is no
Krish (saas.snowpal.com) (36:48.46)
I’m trying to understand the difference between the first double code and the second one here.
Krish (saas.snowpal.com) (37:07.214)
do we have here? We have Docker image, and then Docker container, those were the two and when we saw the other terraform.tf state, Docker, that’s the type. Maybe it’s the type and the name, I bet it’s the type and the name. So if we go to this one here,
The type is AWS instance, maybe the name is example. That’s what I can actually think of here, okay. Okay, let’s go down here. Type and maybe the name is app server, okay. Tag name and then there is an AMI and then it picks up a micro instance, okay. So let’s go with this. Let’s go under the provider. Yeah, we have a third resource actually right here.
Now I can delete the rest of this.
Krish (saas.snowpal.com) (38:10.834)
Okay, so we updated the.
Krish (saas.snowpal.com) (38:19.246)
We’ll close the ones we don’t need for now, okay. This is a complete configuration that you can deploy Terraform, the following sections review each block in more detail. The block contains a set things. I think we have understood most of it. The provider configures a specified provider, in this case AWS. So let’s see.
Krish (saas.snowpal.com) (38:42.222)
Provider is AWS. So let, what if I want to know the Terraform list of available providers.
Krish (saas.snowpal.com) (38:55.242)
There is a registry.
We have AWS, what was the other one was Docker, right? So there is a lot of providers here, but Docker has to be somewhere. It is Docker.
Krish (saas.snowpal.com) (39:24.849)
So…
Krish (saas.snowpal.com) (39:28.048)
AWS is the provider. I’m just looking to see the other one.
Krish (saas.snowpal.com) (39:40.007)
There’s like 3800 of them. Okay, this is interesting. We’ll come back and these providers are a logical abstraction of an upstream API. The response of understanding API interactions and exposing resources. So I’m looking to see if I pick database and filter. Let’s see, just looking.
Mark logic mongo
Krish (saas.snowpal.com) (40:11.27)
Okay cool I think this is going to be pretty interesting okay let’s actually call it add folder we’ll call it podcast move that salesforce here because we’ll come back to that one add a folder terraform move it here
Drop the registry into Terraform and then AWS.
Krish (saas.snowpal.com) (40:46.914)
Downloads this week 13 million. Wow. Okay use providers No, actually I want to see
Krish (saas.snowpal.com) (40:56.614)
I’m just pulling it out. I was just wondering whether there was any example right here. OK, so let’s go.
Krish (saas.snowpal.com) (41:10.054)
Use the AWS provider and try to do the many resources supported by AWS, okay. This is resource. Our resource was AWS instance, and this example is AWS VPC. So I wanna see a list of resources for a given provider. Where would we go in Terraform? Oh, there is documentation here. These are the providers.
Krish (saas.snowpal.com) (41:48.702)
use VPC
Krish (saas.snowpal.com) (42:02.243)
it be in this list at all.
Krish (saas.snowpal.com) (42:23.446)
AWS provider guides. Wonder if these are all actual resources. Yeah, it is. That’s what I thought. So it should have the AWS instance also here.
Krish (saas.snowpal.com) (42:46.026)
AWS Connect Instance. Oh, there we go. AWS Instance. And then if you go to AWS Instance, AMI and Instance Type and all the tags. This is how we would discover. So you first go to the provider at the resource level, right? First you define the provider and for the provider you have the resource. It’s kind of interesting because you know, we have a list of providers defined and then the resources, I would have thought that this would be nested.
because I’m curious to know what would happen if you don’t have the provider and just have the resource is gonna fail with an error. It’d be nice to interesting to see what kind of error that is. Or you could have a list of providers predefined but you still haven’t actually leveraged those resources. I’d say use Terraform Cloud for free. We should certainly check that out. This is, you know, space I haven’t, we haven’t spent much time at, you know, deployment is almost like a necessary evil, if you will.
for a startup because you spend most of the energies in building the software, building the functionality. And in the conversation with again, James yesterday, we touched upon those items. Pre-develops, how was the developer experience, with DevOps, how a developer experience was, and moving to something like platform engineering, how the developer experience could potentially change, and what adaptations would you have to make? And how much of this would you actually deal with directly? So I think it’s…
actually very interesting. As someone who spent, on an average week, we spend the least amount of time on deployments because that’s what we need to get out there. And we have a sort of a process and we’ve become a well-oiled machine of some sort, if you know what I mean. So we just follow the process like a machine, but it also, but we recognize that we have to improve the process as things change. And part of one of the things we are…
expecting and will definitely be doing in 2024, is changing some of our deployment methodologies at Snowpals. You can expect to see a lot more of these, of such podcasts. Let me make sure I’m still recording. Yep, I am. Okay, so back here, just to recap, we went to Terraform, we downloaded the CLI, we installed it, and then we’ve kind of understood that there are providers.
Krish (saas.snowpal.com) (45:06.854)
And within providers, you have these resources and we can kind of spend maybe a little bit of time, a few minutes understanding some of these resources, at least the ones that could be of interest, API gateway and gateway version two.
Krish (saas.snowpal.com) (45:27.158)
manages Gateway version 2 API.
used to create and deploy web socket HTTP APIs. Let’s check for AWS API gateway version one versus version two.
Krish (saas.snowpal.com) (45:46.238)
In v1, there are only REST gateways. In v2, there are two types, HTTP and WebSocket gateways. They can coexist, but they’re incompatible with each other and have very different feature sets. Okay, so let’s go to… This is a bit of a distraction, but that’s I think a useful one, hopefully. Critical production applications, blah, API gateway, HTTP API. Is there, what is the time? This is from…
Three years ago, right? New flavor.
Krish (saas.snowpal.com) (46:21.71)
says it’s in preview, but certainly it’s not in preview after like four years or so. Yep, the V1 and V2, the V2 is for web sockets and HTTP APIs, okay, cool. The V1 represents REST APIs. So let’s say we go with, let’s look at actually V1 from a REST standpoint. You have, what are some of the resources? Usage plan.
Okay so this has a resource, the path.
REST API ID, stage name, API stages, quota settings, throttle settings.
Krish (saas.snowpal.com) (47:07.23)
Amazon gateway integration.
Krish (saas.snowpal.com) (47:19.087)
usage plan.
name, product code, and then the stages. So if we go to, are we looking at, yeah, we’re looking at usage plan here, okay. So I think what I can do as we go make progress with this, we have all of this configured on AWS API gateway, except we’ve done it not through Terraform, which I can already see how this can be, this can help save us a ton of time.
Right now we’re doing it through a non Terraform mechanism, which is a combination of many things. One is we, some of it was created through the AWS console and for the others we use the SAM template. But doing it this way, I wonder, does this mean you use Terraform alongside the SAM templates or does it actually replace the SAM templates? We need to, I need to figure that out. I suspect it replaces the Terraform.
Replace template.
Krish (saas.snowpal.com) (48:31.614)
And it is announcing the public preview support for local development. This is a year ago using Hashtag of Terraform configuration open source frameworks for building applications using infrastructure as code. So I guess Sam is also comes on infrastructure as code. So let me take some of it back. We are doing it predominantly almost exclusively through Sam. So it is infrastructure as code but perhaps it’s not as powerful. I mean, Terraform is you can do it for.
a variety of different cloud providers. And if you’re a large organization, it’s likely you’re using many cloud providers. Maybe I’ve seen really large companies have a vendor lock into a single provider. We as a startup are not like that. Right now we are primarily dependent, that’s the right word on AWS. We do a majority of the stuff on AWS and some a little bit, very little on like GCP and Azure. So with Terraform you can obviously do, you can centralize your scripts.
I can, I am imagining this, I haven’t seen it because we haven’t gone that far, where we can manage all of our deployments across the cloud providers using the Terraform configs. With SAM, obviously it’s, it literally is AWS specific. So looks like they are, let’s see, let’s read a bit more. Previously you could use a SAM CLI to build, test, blah, blah. With this preview release, you can also use, you can also use SAM to test and debug. So we’ll just…
defined using Terraform configurations. Previously, you could use the SAM CLI to build test and debug apps defined by AWS, which is exactly how we do it at Snowpal, or through the AWS CDK, which is where we were making the switch to internally. With this preview release, you can also use SAM CLI to test and debug serverless apps defined.
using the Terraform config. So it looks to me like it is actually a replacement. Let’s look at this diagram. This is AWS Cloud Gateway Lambda, install the HashiCorp, SAM CLI. Okay, there’s a different example here. We could try this one as well. Let me just bookmark it so we can come back and revisit it to see. But it looks to me like it could well be a replacement, right?
Krish (saas.snowpal.com) (50:54.246)
So which means you could manage all of this to Terraform and it’s pretty well beautifully documented. No wonder it’s popular because, even without knowing anything about this other than just being able to spell Terraform and just knowing fundamentals of IAC, we started this podcast, but now I think we’ve found our way into the Terraform, the config we’ve created. We’ve added a couple of providers, Docker and AWS, have two resources.
for Docker, Docker image and Docker container and one for AWS, AWS instance. We were on this, let’s see.
Krish (saas.snowpal.com) (51:43.502)
a few too many tabs open here.
Krish (saas.snowpal.com) (51:50.606)
Maybe I should even close. I don’t wanna close any of them just as yet. Trying to see which tab we were on.
Krish (saas.snowpal.com) (52:00.298)
This looks like it, right? Okay, let me actually add this to the bookmark.
Krish (saas.snowpal.com) (52:08.694)
build infrastructure, that’s the one we’re on. We’ve gotten as far as, I don’t know, like what, 20% into the page, 25%. This is a complete config and deploy the following sections review and we started reviewing it. We looked at providers and that’s where I got distracted. We understand providers, resource blocks, we already seen that. It says it might be a physical or virtual component such as easy to instance or it can be a logical resource such as a Heroku application, okay.
resource blocks have two strings, the resource type and name. I think if you had read the documentation, we’d not have discovered, but I figured it was always nice to discover. But I think we said the exact same thing like 20 minutes ago, it’s a type and the resource name. And again, you could be the type of person who is able to follow through large pieces of documentation and theory. And if that approach works for you, more power to you. It simply doesn’t work for a majority of us at the company, so we take this more.
pragmatic approach, which is how our courses are published as well. It’s an hour, 30 minutes, 45 minutes of trying things out, much like this, that you can watch and learn. And in that window of time, you’ll come out, hopefully, with some element of learning. The prefix, wait, I want the example. Config manages the AWS instance resource of the AWS provider. We tied those two together even without reading the documentation. Together, the type name, type.
resource type and name form a unique ID for the resource. For example, the ID for EC2 instance is AWS instance dot app server. Aha, okay. So you couldn’t have two of them with the same combination. I’m just curious to know why they’re defined as two different strings as opposed to it being just being a, you know, being supported as a dot. Does that mean there is a
there could be more such strings, like there’s a type, there’s a name, and is there more in terms of this? Maybe there’s a schema that we can possibly look at to understand this slightly better. Okay, resources. I think we’ve gotten as far as resources. I wish I can, let me make a mental note. Let’s see how far into this podcast we are. We are at the, almost at the top of the hour. Maybe it is a good time to pause here and just do a quick recap.
Krish (saas.snowpal.com) (54:32.886)
I think we’ve done some iterative recaps progressively along the way. But just so what we’ve done is we’ve created this main.tf file, the Terraform file, we modified it by adding a new AWS provider and two more resources. And then we saw the Terraform state and the backup. We also did the variables and we can probably play with the variables a bit here as well. Because we added the variable for, what was it?
Yeah, for the container name for the Docker container, but you can see that we could add a variable for the AMI, for the instance type and whatnot as well. That way we don’t have these hard coded strings in the actual Terraform files. We just have them as variable substitutions and we can further override those variables as well.
Krish (saas.snowpal.com) (55:35.83)
I think we’ve kind of hit the ground running. Shared, you know, I mean, set up Terraform, the CLI. We’ve created the basic configuration and we’ve added the entries for AWS. As we pick this up.
Let’s call this Terraform series one of N. The subsequent ones we’ll just go pick up from where we left off. Before I end the podcast here, go to sas.snowpile.com, S-A-A-S dot snowpile.com. Check out our APIs. If you’re building, designing, architecting, maintaining, managing, scaling back end systems, there’s a high chance that you shouldn’t have to be doing any of that because our APIs are pretty broad and they help you solve a majority of your more
should I say mundane problems. That way you can focus your energies and your team’s energies and your dollar, your money on building, solving unique problems, your customer problems. So check those out. Check out our postman collections. Go to AWS Marketplace or go to sas.snowball.com and you’ll find those links to aws.snowball.com. You can check out the APIs. We have a free trial for a couple of weeks. Play with it. Have any questions and hit us up.
The idea is so you can go to market much sooner. Let me end this podcast here. I’ll talk to you in the next one. Thanks.
Snowpal Products
- Backends as Services on AWS Marketplace
- Mobile Apps on App Store and Play Store
- Web App
- Education Platform for Learners and Course Creators
Top comments (0)