startupsystemd
I’d like to startup an Apache Spark cluster after boot using the following command:
sudo ./path/to/spark/sbin/start-all.sh
Then run this command when the system prepares to reboot/shutdown:
sudo ./path/to/spark/sbin/stop-all.sh
How can I get started? Is there a basic template I can build on?
I’ve tried to use an extremely simple (file: /lib/systemd/system/spark.service
):
[Unit]
Description=Spark service
[Service]
ExecStart=sudo ./path/to/spark/sbin/start-all.sh
Which doesn’t work.
Accepted Answer
Your .service
file should look like this:
[Unit]
Description=Spark service
[Service]
ExecStart=/path/to/spark/sbin/start-all.sh
[Install]
WantedBy=multi-user.target
Now, take a few more steps to enable and use the .service
file:
- Place it in
/etc/systemd/system
folder with say a name ofmyfirst.service
- Make sure that your script executable with:
chmod u+x /path/to/spark/sbin/start-all.sh
- Start it:
sudo systemctl start myfirst
- Enable it to run at boot:
sudo systemctl enable myfirst
- Stop it:
sudo systemctl stop myfirst
Notes
- You don’t need to launch Spark with
sudo
in your service, as the default service user is already root. - Look at the links below for more
systemd
options.
Moreover
Now what we have above is just rudimentary, here is a complete setup for spark:
[Unit]
Description=Apache Spark Master and Slave Servers
After=network.target
After=systemd-user-sessions.service
After=network-online.target
[Service]
User=spark
Type=forking
ExecStart=/opt/spark-1.6.1-bin-hadoop2.6/sbin/start-all.sh
ExecStop=/opt/spark-1.6.1-bin-hadoop2.6/sbin/stop-all.sh
TimeoutSec=30
Restart=on-failure
RestartSec=30
StartLimitInterval=350
StartLimitBurst=10
[Install]
WantedBy=multi-user.target
To setup the service:
sudo systemctl start spark.service
sudo systemctl stop spark.service
sudo systemctl enable spark.service
Further reading
Please read through the following links. Spark is a complex setup, so you should understand how it integrates with Ubuntu’s init service.
- https://datasciencenovice.wordpress.com/2016/11/30/spark-stand-alone-cluster-as-a-systemd-service-ubuntu-16-04centos-7/
- https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files
- https://www.freedesktop.org/software/systemd/man/systemd.unit.html
The post How to Run a Single Command at Startup Using Systemd in Ubuntu? appeared first on Stack All Flow.
Top comments (0)