(Feel free to skip to the tutorial)
This post is the first in a planned series about rapid software development in the modern world.
Rapid developer onboarding is something we've been dealing a lot with at Otomato software. But what really got us started on diving deeper into this - was my conversation with Elad Meidar a few months back.
Elad was talking to me about how non-trivial it has become to choose a stack in today's world. And let's say you've chosen your tools - only the most experienced developers really know how to set up CI, testing, deployment, instrumentation, monitoring, security, etc. correctly. A lot of Ops knowledge is involved in even getting things initially running.
Our current goal is to take that knowledge that we've accumulated and make it available as a service. And while we're doing that - we're exploring the tooling that's currently available.
Buffalo is a framework, (or a tool) for rapid web development in Go. Cloud Native DevOps folks (and that's what we are at Otomato) have a soft spot for Golang and that's why I'm starting this series with Buffalo.
The official getting started section of Buffalo documentation is great but as I ran through it I noticed it lacks some operational details that I'm planning on exposing. Again there's Ops knowledge lurking in the dark!
Quite naturally one would need to install Go.
On a Mac:
brew install golang
sudo apt update && sudo apt install golang
Note: on older systems (such as Ubuntu 20.04) you'll get a very old version of Go (1.13) by default when installing with
apt. So instead - choose the download-n-extract option here.
For additional installation options go here
On a Mac:
brew install nodejs
sudo apt update sudo apt install nodejs npm
Buffalo makes quite a few educated assumptions when generating your project. One of them is that you'll want to wrap your app in a container. You can, of course opt out, but why? So if you're going with the flow and enjoying the benefits of containerization - you probably already have Docker installed. If not - please do so now - we'll need it further along the tutorial.
On a Mac:
brew install gobuffalo/tap/buffalo
wget https://github.com/gobuffalo/cli/releases/download/v0.18.8/buffalo_0.18.8_Linux_x86_64.tar.gz tar -xvzf buffalo_0.18.8_Linux_x86_64.tar.gz sudo mv buffalo /usr/local/bin/buffalo
Buffalo has a project scaffolding feature that allows us to generate a new app complete with:
- a local git repository
- a backend api
- a db integration
- a frontend
- a Dockerfile
- a CI pipeline.
Let's create a webapp called
testr. It will be used to manage test assignments for new and existing trainees. (Did I mention we do technical training at Otomato too?)
The command to create a new project is
The default DB backend used by Buffalo is PostgreSQL.
We will be using Github for SCM, so we'll choose Github Actions as our CI provider.
buffalo new testr --ci-provider github
After buffalo shows us what it's bringing in and generating (quite a bunch of stuff really) it will say:
Initialized empty Git repository in /Users/username/git/testr/.git/ DEBU[2022-08-28T23:37:54+03:00] Exec: git add . DEBU[2022-08-28T23:37:54+03:00] Exec: git commit -q -m Initial Commit INFO[2022-08-28T23:37:54+03:00] Congratulations! Your application, testr, has been successfully generated! INFO[2022-08-28T23:37:54+03:00] You can find your new application at: /Users/antweiss/git/testr INFO[2022-08-28T23:37:54+03:00] Please read the README.md file in your new application for next steps on running your application.
So we 'll do just what it tells us to:
cd testr git add . git commit -q -m "Initial Commit"
Before we can actually start developing our code there's
a need to spin up a database. We could, of course, use a managed DB but it would probably cost us a few bucks. So for local development it makes much more sense to run the DB in a container.
Let's run PostgreSQL (the default Buffalo DB backend):
docker run --name buffalo-postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres
Note: we're running PostgreSQL with a very naive password here , which is fine for local development but not fit for anything production-like.
We're also exposing it on
localhost:5432 - which is where a Buffalo app is configured to look for it by default.
These configurations are defined in a buffalo-generated file
database.yml which we'll use shortly.
Running the DB container isn't enough. We also need to create a database for our app.
This can be done by entering the container and running good old SQL commands. But Buffalo creators recommend the use of Soda - a small and useful CLI utility that makes managing DBs easier.
go install github.com/gobuffalo/pop/v6/soda@latest
And create a DB:
soda create -a
Soda creates all the databases configured in the file
database.yaml that Buffalo has generated for us.
Note: By default Buffalo uses its own ORM library called
pop for DB integrations. Pop provides a wrapper for
soda - so we can also run
soda commands through buffalo aliases:
buffalo pop create -a or
buffalo db create -a.
Buffalo delivers even more useful DB stuff with the help of
pop - like model generation. But we'll cover that in the follow-up post.
Buffalo provides us with a
buffalo dev command which allows running our app with live reloading - i.e restarting the application server each time we change the code.
Now we can visit http://localhost:3000 in browser and see our app running!
And - we're live!
The web UI we see is generated from
testr/templates/home/index.plush.html using the plush templating engine. Also to be covered in a separate post.
As the final step of this walkthrough - let's push our code to Github and verify the generated CI pipeline works.
I heartily recommend using Github's
gh cli tool:
testr directory run:
gh repo create --public <your-user-or-org>/buffalo-testr --push --source .
This will create the repo and immediately push the code to it, which in turn starts the workflow defined in
name: test on: push: pull_request: permissions: contents: read jobs: test: runs-on: ubuntu-latest services: postgres: image: postgres:9.6-alpine env: POSTGRES_DB: testr_test POSTGRES_PASSWORD: postgres options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 ports: - 5432:5432 steps: - uses: actions/checkout@v3 - uses: actions/setup-go@v3 with: go-version: ~1.18 cache: true - name: setup run: | go install github.com/gobuffalo/cli/cmd/buffalo@latest - name: test env: TEST_DATABASE_URL: postgres://postgres:postgres@localhost:5432/testr_test?sslmode=disable run: | buffalo test
We can see that this workflow:
- spins up a PostgreSQL service container
- installs buffalo
buffalo test- which in turn creates the DB in the container and runs some tests:
[POP] 2022/09/12 15:24:04 info - dropped database testr_test [POP] 2022/09/12 15:24:05 info - created database testr_test pg_dump: error: connection to server at "127.0.0.1", port 5432 failed: FATAL: database "testr_development" does not exist [POP] 2022/09/12 15:24:05 info - Migrations already up to date, nothing to apply [POP] 2022/09/12 15:24:05 info - 0.0102 seconds [POP] 2022/09/12 15:24:05 warn - Migrator: unable to dump schema: open migrations/schema.sql: no such file or directory time="2022-09-12T15:24:06Z" level=info msg="go test -p 1 -tags development testr/actions testr/cmd/app testr/grifts testr/locales testr/models testr/public testr/templates" go: downloading github.com/gobuffalo/suite/v4 v4.0.3 go: downloading github.com/gobuffalo/httptest v1.5.1 go: downloading github.com/stretchr/testify v1.8.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/pmezard/go-difflib v1.0.0 ok testr/actions 0.019s ? testr/cmd/app [no test files] ? testr/grifts [no test files] ? testr/locales [no test files] ok testr/models 0.012s ? testr/public [no test files] ? testr/templates [no test files]
Voila! The workflow works. It doesn't create a Docker image for us (so no artifacts) - but it does run some basic integration testing.
Tests for Buffalo apps is another topic for yet another post.
Buffalo is a well thought-out rapid development framework for full-stack apps or standalone backend APIs. It does pack quite a lot to get us started, but it still leaves the developer in the playground - without any clear guidelines regarding where and how to deploy their code for production.
And what are you using for rapid bootstrapping of new services?
What other rapid development frameworks would you like us to cover?
Let us know in comments - our research is only starting!