Create a log service with PHP and Clickhouse [Part 1]

I would like to share my experience configuring a very simple logging service with PHP and Clickhouse. I would like to stress that this is by no mean an optimal setup, I simply want to share my experience and I would love to hear feedbacks to improve.

Requirements

1. A very simple logging service to track all kinds of events such as nginx error or php error internally and externally. I find ELK too complicated to properly setup and guard, and too resource intensive (and thus costly) for my needs.
2. Performant enough to handle tens of millions of logs per day on an average server (I pick the Vultr instance below which has 2 CPU and 4 GB of RAM). I'm handling over tens of million logs per day with this setup and still have resources to spare.

Tech stacks

1. PHP
2. Clickhouse
3. Vector

Why PHP?
Because I'm familiar with it and because why not? I don't want to spend days and weeks trying to learn a new thing just for something really quick that I can do for a 1-2 hour with PHP.

Why Clickhouse?
Clickhouse is a very performant, easy-to-setup database designed and optimized for writing data. I have great experience using it to track views and clicks on my website.

Why Vector?
I looked at many log collectors such as syslog, syslog-ng, filebeat etc... I'm not a system guy and their documents are terrible, terrible. Filebeat is not so bad but I was not entirely impressed with it (I used it in the past). Then I found Vector and I was blown away by the website (so clean), the documents, the options, and the support.

Preparation

First, we need to collect the logs. Vector is super simple for collecting logs, you can find the website here with all the documents:

https://vector.dev/

I'm going to share some of my current configs, please do note that I'm a Vector virgin, so please do review before use and please let me know if I can improve anything:

https://gist.github.com/yellow1912/782d53cb97b9de86d4fdb2848cc2e520

You will notice that I'm opting to send the logs to a remote server via http/https protocol. You can choose any other protocol you want. I also pass along an authorization token so my log service can check.

Next time, I will cover Clickhouse database setup.