As my second month in DevOps comes to a close it's time to reflect on a month of AWS, Vagrant and Jenkins (plus perhaps another project!)
An introduction to virtual machine management came this week. Focussing mainly on Vagrant and virtual box we looked at methods at which we can rapidly deploy virtual machines and apps from multiple sources.
We were also told about the catchy buzzphrase of "Go global in minutes!" and the four key pillars of DevOps:
- Ease of use
That sounds pretty good!
The end goal essentially was to be able to have Vagrant set up an operating system and deploy an app for us automatically. But the first roadblock actually came from somewhere I hadn't considered... my computer:
So... what's wrong with this image? Well, my Mac is one of the new M1 based Macs. This hasn't been a problem so far but it does present a problem when it comes to virtualisation because as of now neither VirtualBox nor VMWare have ARM versions of their software available to download and use. Not to worry though, I have another PC ready to go that I use mainly for gaming which I have Ubuntu installed on. Then all was fine in the world!
So without going in to too much detail, I had my Vagrantfile set up to create two machines:
The plan was to have a node.js app automatically deployed and connect to the MongoDB database to pull data from it and the end goal was to have it ready to go just by typing in
You can check out the full configuration here:
Setting up development environemnt
Installation of Vagrant, Virtual box and Ruby
vagrant upto launch the VM
vagrant destroyto delete everything
vagrant reloadto, well, reload
vagrant haltto stop the vm
Getting in to Vagrant
sshinto our VM and launch nginx web-server
apt-getpackage manager in Linux - for mac
apt-get-is used to install and uninstall any packages needed
- To use the command in
adminmode we can use
sudobefore the command
sudo apt-get upgrade -y
sudo apt-get update -y
- To work in
admin modeall the time (don't do this)
- We will install nginx in our guest machine
- Launch the default nginx page in host machine's browser
- To come out of your vm
- Install nginx
sudo apt-get install nginx -y
- Checking status of nginx
systemctl status nginx
- Or restart
systemctl restart nginxor just…
I will admit that there was a couple of things that I couldn't seem to automate in the provisioning file such as setting up the environment variable but in later configurations (that for some reason I didn't push to GitHub yet) so I have another bash file that I run afterward that does the last steps of configuration.
Still haven't figured out exactly why it doesn't work but it hasn't stopped me yet!
Our sixth week was an interesting one, being set a project so close after the previous one but at least it allowed for us to now bring in the our use of APIs with the virtual machines.
We were tasked to create a job searching website which would scrape a website for entries and then let the user manipulate the data in a way that they'd like (sort by alphabetical, salary, etc.) and then provide an easy to download CSV that the user may then take and manipulate the data at their leisure.
Above was our workflow for manipulating the data as drawn by my team mate who goes by twilliams9397 on GitHub
Well, we did it, and we got it to be automatically deployed and running through Vagrant too. I won't go into too much detail over the app itself as you're free to check it out for yourself over here:
But I'm really happy how this one turned out. Most importantly (I hope much to the pleasure of @genei09 ) I think we nailed Git. We only had one merge conflict and we got it resolved very quickly so I'm calling that a win. Plus I didn't realise that suggestions in code reviews were a thing, and we found that really useful for quickly changing stuff on the fly.
Establishing a good system early on was great. Plus merging changes felt so satisfying. I never realised how good it felt to have a whole days work just merged right in... uhhh, anyway.
Here is the site:
This really was the crux of the whole project, this was the main deliverable and what was the most work to get set up. Scraping and displaying the information about the various roles on ITJobsWatch. We then went a little further and added a search box, as well as individual pages with more details about each role:
We then also scraped some of the vacancies on the site and gave people a way to add their own:
I'm really proud of the work we did not only with building on top of what we had done previously but also with really nailing our collaboration tools!
We started the week learning about EC2, a service to host virtual machines in the cloud, as well as S3, an online storage service, plus the AWS CLI, which we actually didn't end up using much.
Setting up both of these and using them is pretty much as simple as it sounds, since we had experience with vagrant already it was just an extension of that. I really like, however, being able to just give it the provisioning file and have my app available worldwide in minutes is actually really impressive. Although it may not officially be for this purpose, I feel that for rapid prototyping and previewing of code will be very useful using both this and Jenkins since I can ask people to test code before pushing it to production servers. Really going to look forward to using this kind of technology.
Jenkins was our last port of call and although we were using it with pre-existing test applications we still were able to get a grasp of what it can do. Testing and deploying of apps automatically? Pretty nice!
I'm planning on experimenting with GitHub actions on my own since it seems similar but I don't have to run my own Jenkins server (I'm cheap, alright?). But automation is something I'm definitely going to do more research in. If you've got any tips I'd love to hear them!
As my instructor said, you need to learn how to do things manually before you can do it automatically and honestly, automation does add a layer of complexity to setups because it makes it harder to know where an error is occurring and for experimentation. But I'm seeing it as an investment, spending a few extra minutes now to save hours later!
Week 8 was thrown a little off kilter, this was due to our instructor not being available due to unforseen circumstances so a stand in was brought in meaning that instead of IAC and Ansible we instead learned about setting up VPCs, security groups, internet gateways, subnets and so on.
Bastion was actually a really interesting concept and was something I hadn't really thought of doing before. In our example we used our demo app from week 5 which was comprised of a node.js app and a MongoDB database. We want the app to be publicly viewable via a web browser and for the app to also pull data from the datanase, but we don't want the database to be publicly available for obvious reasons.
Having a bastion pretty much is like a gate keeper so we can only SSH in to the database through the bastion. So if it's off, access to the database is off too. I like the idea a lot and it does seem more robust than using security groups or network ACLs.
The potential for scaling testing and production up and down basically dynamically using cloud services is something I'd be interested in trying out for myself, but not right now as I've seen that it can get expensive.
I guess if there was a blocker so far it'd probably be AWS' interface, it's a little confusing as a lot of menus look the same and sometimes it seems to hide some options behind menus I wouldn't expect them to be in. But once I figured it out I was fine, plus it looks like they're still taking feedback for the redesign so things may change yet!
Next month is my last on the course, we will be covering Ansible, Terraform as well as completing a big group project where all 15 of us will be working on one project!
Thank you again for reading and I really appreciate the comments I got last time! See you again in a month's time!