This is a translation of bmf-techを支える技術.
This article writes about my blog, bmf-tech.com.
Please take a look if you like. :D
Scratch and mange the my blog
Write about the tech stack behind this blog bmf-tech.com.
Configuration of old bmf-tech
First of all, the current bmf-tech is about the configuration one generation before.
- The application is a monolithic configuration based on Laravel
- API was PHP, management screen was building SPA with React
- I just vaguely adopted the technology I was familiar with at the time
- Hosted on Sakura VPS
- I didn't use any configuration management tools (such as Ansible), and I was manually installing middleware and adjusting the configuration.
- "warm" composition
- No monitoring tools available
- If you want to see the log, ssh to the server and go directly to see it
- not using containers
- Is Vagrant mainstream? was it popular? it was an era
- Deployment was configured to do something good with git hooks
Rubel is the first self-made CMS that was used in the previous generation.
I don't remember how many years this was in operation, but I think it's probably 3-5 years.
Before using Rubel, I created an original theme with Wordpress and operated a blog.
Wordpress (original theme 1) → Wordpress (original theme 2) → Rubel → I feel like I'm still here.
I checked the domain age of bmf-tech.com and it was acquired on November 2nd, 2015.
I forgot when I started blogging, but based on the domain age, I've been operating for nearly 7 years.
new bmf-tech configuration
Regarding the current bmf-tech configuration.
Design
There were several reasons for wanting to replace the system.
- I wanted to create an opportunity to experience new technology
- At that time, I was mainly working with PHP, so I wanted an opportunity to work with other languages.
- I didn't have the energy to maintain Rubel
- Laravel's update cycle is fast and needs frequent support
- I want to focus on business logic as much as possible rather than framework updates
- React is too much for me
- cried to Redux
- FLUX wasn't at the scale we needed
- I wanted to cut out the API and make it easy to throw away the front
- I wanted more control over the source code
- Overly relying on frameworks, I felt a sense of crisis about framework dependencies
- Hard to catch system bugs -etc...
For these reasons, I roughly considered the design policy of the new system.
- Build a system that is easy to maintain in the long term
- Server configuration management
- Do IaC properly
- Ensure idempotency
- Easy to switch even if you have to switch servers
- Application design
- I want to avoid framework dependencies
- Rely only on standard libraries and self-made libraries other than the front end
- Build a monitoring environment
- Ability to monitor system metrics, logs, etc. and set alerts
Architecture configuration
The architecture configuration built based on the design policy is as follows.
- Server is ConoHaVPS instead of Sakura VPS.
- spec -CPU 2Core
- Memory 1GB
- SSD 100GB
- Image type Ubuntu
- ConoHa supports OpenStack, so it is easy to manage code with Terraform for instance construction
- Cheap and easy to use is also a good point
- Server configuration management uses Ansible
- SSL uses Let's Encrypt
- Use go-acme/lego for certificate acquisition/renewal
- cf. legoでLet's encryptのSSL証明書をDNS-01方式で取得する
- At first, I tried to get a DNS-01 certificate with my own script without using lego, but I gave up because sometimes I couldn't get it.
- cf. k2snow/letsencrypt-dns-conoha
- Adopt Docker for virtualization
- Managing multiple containers using Docker Compose
- There was also a verification to try using Kubernetes
- TerraformとAnsibleを使ってKubernetes環境構築
- I wanted to operate my own Kubernetes cluster, but gave up because I couldn't handle the bare metal load balancer well
- Nginx is a web server that delivers management screen (SPA), API (headless CMS. Go), Client (user side screen. Go) screen
- The Nginx image is customized to include the management screen image in a multi-stage build, so the pre-built source code of the management screen is included.
- API and Client are images that include binaries built with Go
- Monitoring uses Grafana as UI
- Prometheus collects various server metrics, and Promtail and Loki collect container logs
- Grafana for visualization of collected
- At the beginning of the release, the EFK stack was built around container log collection, but with a 1GB server spec, the resources were too tight, so I changed the configuration to use Loki and Promtail.
- Functionally sounded good enough and simple and easy to use, seemed good enough for my requirements
Deploy
The deployment isn't particularly complicated.
- We have a private repository (bmf-tech) that manages container configurations
- gobel-example is used as a template
- gobel-example is an EFK stack, but the private repository (bmf-tech) uses promtail x Loki
- Containers can be run in the local environment with the same container configuration as the production environment
- Doesn't contain any application source code
- All environment variables are configured to be injected externally
- For deployment, ssh→pull bmf-tech→rsync to upload files of environment variables→docker-compose build & up Just create a script and run it
- No version control
- Momentary downtime is a disadvantage, but considering the current traffic, there is no big problem
- It affects availability..
- At first, I verified and tried the deployment method using docker context, but it was rejected (I forgot the reason...)
Source code control
Below is a diagram showing how the source code management is based on the container configuration.
- bmf-tech
- A private repository with gobel-example as a template to manage the container configuration
- bmf-tech-ops
- A private repository with gobel-ops-example as a template for construction and operation operations such as server configuration management and deployment script management. I manage the code involved
- bmf-tech-client
- A private repository with gobel-client-example as a template to manage user-side source code
- Push the image to DockerHub and manage it as a private image
- gobel-api
- gobel-api
- Manage the source code of headless CMS
- Push the image to Dockerhub and manage it as a public image
- gobel-admin-client
- gobel-admin-client-example
- Manages the source code of the management screen
- Push the image to Dockerhub and manage it as a public image
Application design
About each application of API, Client (user side screen), Admin (management screen).
APIs
- Build your API as a headless CMS
- Uses Go
- The language specification is simple, the backward compatibility is firmly guaranteed, the compilation is fast, the portability that runs in binary when built is good compatibility with containers, the type is available, and the standard library is substantial etc...
- Adopted Clearen Architecture
- I think there are pros and cons about compatibility between Go and Clean Architecture, but I tried
- We believe that adhering to the "separation of concerns" in Clean Architecture is one way to keep your application maintainable in the long run.
- Few dependencies other than standard packages
- cf.go.mod
- API protocol is REST
- If you are going to make it from now on, gRPC might be good
Clients
- Web server that responds the screen
- Implement API client in Go
- The template file (html) is included in the binary with embed
- The design uses a self-made CSS framework
- cf.sea.css
Admin
- Management screen
- Authentication is token authentication by JWT
- Redis for session management
- Adopted Vue.js
- There is an impression that it is simpler and easier to work on than React
- I haven't touched React recently, so I don't know, but I've been using Vue here and there, so I adopted it as well as gaining knowledge.
- I tried Atomic Design
- but I don't know if it works
- Updated from 2 series to 3 series during development
- I thought about introducing TypeScript, but I'm behind
- It seems that the management screen does not add functions frequently, so it is assumed that maintenance will only be done to update the library
- I thought that front-end trends would change over the course of several years of operation, and that the next framework might come out.
- SPA delivery is supported by Nginx
DB design
Basically, it inherits the DB design of Rubel as it is, but there are some parts that have been redesigned by reviewing logical and physical deletion.
After that, it feels like reviewing the data type and size of the column.
Migration work
For data migration, I wrote my own data migration tool.
Since there was no big difference in DB design, we were able to implement the migration tool in about 2-3 days.
As for the server migration, I didn't do much.
I prepared the server environment of the migration destination, acquired a domain for verification to check the operation, and performed various operation checks in the server environment of the migration destination.
I made and broke repeatedly to check whether there were any problems with IaC.
During release work, migration to the new environment is completed simply by switching DNS.
After confirming that there were no problems after operating the new environment for about a month, the old environment was deleted and the contract was terminated.
Summary of what I made
I will summarize what I made before releasing the new bmf-tech.
- application
- Library
- goblin
- tri-tree based router
- I spent a lot of time writing various blogs
- Simple JSON logger. You can specify the log level.
- goemon
- Go dotEnv
- I made it but never used it
- tool
- migrate-rubel-to-gobel
- Data migration tool
- Boilerplate (created during the validation process)
- go-clean-architecture-web-application-boilerplate
- A repository created to prepare the base for Clean Architecture in Go
- go-production-boilerplate
- A repository that verifies how to deploy using docker context
- docker-based-monitoring-stack-boilerplate
- A repository created so that it can be reused later when creating a monitoring environment
- terraform-ansible-openstack-boilerplate
- Verified repository for using terraform with ansible
- setup-kubernetes-cluster-on-vps-boilerplate
- Repositories verified for building Kubernetes clusters on VPS. Abandoned because it could not handle bare metal load balancer.
- vue-js-boilerplate
- A repository created to catch up on what Vue.js is like these days
While making the above various things, I was writing on my blog, doing LT, and doing other things, so I spent a lot of time on the release of the new bmf-tech.
Future outlook
There were many times when I stopped working on development, strayed from the path, and hesitated about switching to Wordpress or another existing system several times, but I was able to operate it safely and it was a burden on my shoulders. went down.
I have a lot of issues about what I haven't done and what I want to do, so I'd like to deal with it as a hobby in my spare time.
Rather than just creating, I would like to refine how to create and how to operate, so I am thinking of investing through this blog system for that purpose.
The reason why I run my own blog is that there is a strong element of learning.
- for my own learning
- Write articles to organize your learning
- I think it will be an opportunity to learn again if I make my own blog system and host it myself and use it myself.
- Acquisition of system construction and operation experience
- I think there is something I can learn from operating a system where I have a grasp of everything.
- earn pocket money
- I'm joking that I want to live off advertising revenue, but I'd like to earn some pocket money, at least server operation costs.
- I don't try to monetize because I only think about it as a bonus
- For the past 1-2 years, we have been profitable at a percentage of annual operating costs, so we are conveniently thinking that there may be some value in continuing.
We believe that we can continue to operate the current system for the time being, so we will continue to do so patiently.
Top comments (0)