If you work in the open-source world, this usually means deploying code to some Linux server or working on a Linux/Mac machine.
You will inevitably need to write some BASH code or run terminal commands. In this article, I'll cover some of the essential concepts you need to know to be productive on Linux servers.
Cron jobs
In most Linux systems, you can place cronjob tasks in the following folder:
/etc/cron.d/
Any standard Linux file name is fine, and the format of crons entered in this file should be:
*/15 * * * * root /the/script/to/run
The format of specifying the schedule:
.---------------- minute (0 - 59)
| .------------- hour (0 - 23)
| | .---------- day of month (1 - 31)
| | | .------- month (1 - 12) OR jan,feb,mar,apr ...
| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon ...
| | | | |
* * * * *
bash
You can also use it to manage the crons for the current user session:
crontab -e
Systemd
On Ubuntu systems, you can set up a "supervisor" service that will startup and manage your application as a daemon process in the background.
Let's say for example you have a Golang binary that you want to start on system startup, and you want to be able to ensure that if it crashes for whatever reason, the server will automatically restart the application; you can easily achieve this with systemd.
Simply place a config file in:
/etc/systemd/system
# Example
/etc/systemd/system/myapp.service
# Whenever you make updates to this folder - run the following:
sudo systemctl daemon-reload
# to enable your service
sudo systemctl enable myapp.service
# to disable your service
sudo systemctl disable myapp.service
Once your service is enabled, you can then view its status or start/stop the process. Simply replace "enable" above with "status", "start", "stop" or "restart".
Here is a basic config example:
[Unit]
Description=A microservice to handle payments
After=network.target
[Service]
User=www-data
Group=www-data
Restart=always
RestartSec=3
WorkingDirectory=/var/www/wallet/
ExecStart=/var/www/wallet/server
StandardOutput=/var/log/wallet.log
StandardError=/var/log/wallet_error.log
[Install]
WantedBy=multi-user.target
Essentials bash commands
# search a file for a string.
# -i: ignore case
# -B: show x number of lines that appear before the found result.
# -A: show x number of lines that appear after the found result.
grep -i -A 10 -B 10 ports docker-compose.yml
# Stream the output of a log file to the terminal.
# tail will stream the last line as
# it's being written to the log.
tail -f /var/log/nginx/error.log
# List all running processes
ps aux
# Find a particular running process
ps aux | grep google-chrome
# Kill a process
kill -9 1234
# Kill by name
pkill -9 firefox
# Show headers for URL
curl -I 'www.website.co.za'
# copy a file to a remote server
scp somefile -i ~/.ssh/auth.pem developer@192.168.1.1:/home/developer/
# Start terminal session with remote server
ssh -i ~/.ssh/auth.pem developer@192.168.11
# Lookup commands you ran in the past
history | grep scp
# Post JSON via curl
curl --location --request POST https://myapi.com/status/save \
--header 'Content-Type: application/json' \
--data-binary @- << EOF
{
"Host": "$HOSTNAME",
"Status": "$STATUS",
}
EOF
Useful docker commands
# Bring up docker containers
docker-compose up -d --build
# Take down containers
docker-compose down --remove-orphans
# Bring up containers using a different .yml file
docker-compose -f the/file/path up -d --build
# Read logs for a container
docker logs -f container
# List docker volumes
docker volume ls
# Remove a volume
docker volume rm someVolume
# Remove docker image
docker images
docker rmi 98045bb148f1
# Clean dangling containers and images
docker system prune
# Create a docker network
docker network create myapp-network
# Docker run and expose host via to container
# -v: binds a local folder as a volume
# -e: Specify envs
# -d: run as a daemon
# --rm: delete container when stopping the container
docker run --rm -d --add-host host.docker.internal:host-gateway -v ${PWD}:/app -e VIRTUAL_HOST=myapp.dev --name devbox myapp/image
Resource usage
# Visual system resources viewer
htop
# View disk space usage
df -h
# Size of files and folders
du -h
# Find top three biggest files in folder
sudo du -h /home/developer | sort -rh | head -n 3
# Show memory stats in gigabytes
free -g
# Find percentage utilisation
free | grep Mem | awk '{print $3/$2 * 100.0}'
Run processes without interruption:
# Open a separate bash session in a virtual screen
# This key combination will keep the screen open and take
# you back to your previous session
# ctrl + a + d
screen -S runSomeJob
# Re-open a running screen - this will list open screens
# or open your screen if you only have one active.
screen -r
screen -r 97930.RunSomeJob
# use ctrl + d in the screen session to terminate
# Run a command in the background
# will output logs to ~/nohup.out
nohup bash run.sh &
Finally, some neat bash tricks
# Shortcuts, instead of typing this SSH command all the time
# Simply enter srv01 in your terminal to run the full command.
# add to your ~/.bashrc file
alias srv01="ssh root@192.168.1.1"
# Pre-load an environment when your terminal starts
source ~/.venv/python3/bin/activate
# Extend your terminal with themes and syntax completions
https://starship.rs/guide/#%F0%9F%9A%80-installation
curl -sS https://starship.rs/install.sh | sh
Top comments (2)
Hi Kevin. This looks like a great cheat-sheet!
When defining cronjobs, there are many online utility websites that can help too, e.g. sites like this can give instant feedback, which can be useful.
Thanks, Yes agreed, I use this site often - makes it so much easier.