What I built
I built a gist to install airflow (workflow + scheduler) in Linode
Category Submission:
Smooth Shifters
App Link
http://143.42.142.188:8080 (you won't be able to log in but you will see the airflow instance running)
Screenshots
Description
Airflow is running in linode. A daemon was set as well to enable accessibility to the tool.
Link to Source Code
https://gist.github.com/mamontesp/2d2fb5aa7ab880f257cbed25951da5f1
Permissive License
MIT License
Background
I need airflow in my work, but I have some security restrictions to create an airflow instance in the cloud provider the company uses. Linode offers a straightforward way to fast testing a tool like airflow
How I built it
I used the PyPI version of Airflow, created a virtual environment with python and in there, I did the airflow install. Then, I started the airflow webserver. All the instructions are in the gist file provided :)
I wanted to use my own user in Linode, but got several issues granting permissions, I figured some out, but at the end I just needed to test airflow, so I switched to root and complete the install. It would be good to have a tutorial to set new users.
Also when trying to use my own new user, shell command navigation wasn't working (pressing arrows wasn't navigating across my command history), I was able to solve that out running this.
sudo chsh -s /bin/bash <username>
Finally, I needed to create a daemon to keep up airflow, I also included the instructions that I followed to accomplished that.
Additional Resources
- Airflow: https://airflow.apache.org/docs/apache-airflow/stable/index.html
- YouTube Course in Linux TV to get more confident with Linux general commands: https://www.youtube.com/watch?v=2733cRPudvI&list=PLT98CRl2KxKGj-VKtApD8-zCqSaN2mD4w
- Daemon creation process: https://baykara.medium.com/how-to-daemonize-a-process-or-service-with-systemd-c34501e646c9
Let's connect
Mastodon: @mamontesp@fosstodon.org
LinkedIn: https://www.linkedin.com/in/andrea-montes-5a9a25b1/
Top comments (0)