DEV Community

loading...

Python: Compile standalone executable with nuitka

Kamal Mustafa
Python/Django Developer at GetOTP (otp.dev), fast and simple to integrate OTP API with ready made UI
・2 min read

So nuitka compile python code into an executable. I have always impress with Go ability to generate single binary and even cross compile for different OS. I wish we can have the same thing with python.

As always, let's start with just a simple script first:-

import requests

resp = requests.get("https://httpbin.org/get")
print(resp.content)
Enter fullscreen mode Exit fullscreen mode

Actually the first I try is with just a simple print("hello world") but that just too simple. I want to see if nuitka can handle more significant code like the requests library above. To compile the code:-

python3 -mnuitka --follow-imports hello.py
Enter fullscreen mode Exit fullscreen mode

It will generate a file called hello.bin in the same directory you run the command above. Execute the file (./hello.bin) and it works! But if you copy the file to different system (I compiled it on ubuntu 18.04 and try to run it on my laptop running Manjaro), I got this error:-

./hello.bin: error while loading shared libraries: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory
Enter fullscreen mode Exit fullscreen mode

So it still need the same python libraries that it get compiled with. And since my manjaro system using python 3.7, hence the error. Fortunately there's second option:-

python3 -mnuitka --follow-imports --standalone hello.py
Enter fullscreen mode Exit fullscreen mode

This time it will generate a folder instead named hello.dist. Inside the folder you'll see various .so files and a file named hello. This time when I copied the folder hello.dist from Ubuntu 18.04 to my Manjaro's laptop, the command works!

Unfortunately the reverse didn't work though. When compiling on my Manjaro and try to run it on Ubuntu 18.04:-

./hello: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/kamal/hello.dist/libpython3.7m.so.1.0)
./hello: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/kamal/hello.dist/libpython3.7m.so.1.0)
Enter fullscreen mode Exit fullscreen mode

This is because manjaro is using more recent glibc than ubuntu 18.04. We can check this with ldd:-

ldd --version
ldd (GNU libc) 2.30
Enter fullscreen mode Exit fullscreen mode

And the only workaround for this is to compile is on the oldest system you want to support. I try this with ubuntu 14.04, it work for simple script but then failed with missing ssl certs when compiling my tgcli script.

So a standalone distribution of your python program kind of work. For my tgcli, the installation look like:-

tar xzf tgcli.dist-glibc-2.27.tar.gz
sudo mv tgcli.dist /usr/local/
sudo ln -s /usr/local/tgcli.dist/tgcli /usr/local/bin/tgcli
Enter fullscreen mode Exit fullscreen mode

For more fun story on getting standalone python program, can also read this sharing from syclladb.

Discussion (6)

Collapse
k1dv5 profile image
Kidus Adugna

I tried this on Windows with nuitka --standalone --recurse-all --recurse-stdlib --plugin-enable=tk-inter script.py, hoping it would reduce the size of the bundle compared to Pyinstaller but the opposite happened (51.7MB vs 20.8MB). Is there anything I don't know?

Collapse
k4ml profile image
Kamal Mustafa Author

Have you checked this?

nuitka.net/doc/user-manual.html#wi...

Collapse
k1dv5 profile image
Kidus Adugna

That worked, removed a good chunk and reduced it to about 28MB, its more than Pyinstaller's 20MB but still. Thanks!

Collapse
k4ml profile image
Kamal Mustafa Author

I've just bumped into this old HN discussion and someone posted GvR comment on nuitka:-

This project looks completely misguided. The talk focused on the trivialities of mapping Python to C++ rather than on the interesting problems to be encountered when trying to optimize Python while maintaining its extremely dynamic semantics. Also the benchmarking effort is laughable; pystone is not to be taken seriously (only exercises a tiny part of the language) and pybench does microbenchmarks, which are optimized away. You should try the "real-world" benchmarks from the PyPy and Unladen Swallow projects. And what is the size of the generated code? (E.g. how big would the binary for the entire standard library be?) In your blog, please use less boring subjects than "version x.y.z released". ~ Guido van Rossum

news.ycombinator.com/item?id=8771925

Collapse
perigk profile image
Periklis Gkolias

Didn't know the tool. Thanks for sharing Kamal

Collapse
k4ml profile image
Kamal Mustafa Author

A review on 3 other tools - pyinstaller, cython and pyoxidizer.

tryexceptpass.org/article/package-...