Running python from node and stream output

My current side project is RPiCon - a desktop application to develop python for the raspberry pi.

The main feature is that you will be able to run code on your desktop/laptop before deploying to the pi taking advantage of a virtual GPIO. Since I already had decided to write it as an electron application I needed a way to run and interact with python code from nodejs.

When deciding how to run the code I had two requirements:

1. Being able to start and stop the python application easily
2. Streaming the output so that I could display it to the user in real time

The obvious choice

As part of node's out-of-the-box api they have something called Child Process which is well documented and there are plenty of tutorials for it. So I added this snippet to my app:

const { spawn } = require("child_process");

var pyProcess = spawn("python", "PATHTOFILE.py");

pyProcess.stdout.setEncoding("utf8");
pyProcess.stdout.on("data", data => {
  console.log(data);
});

pyProcess.stdout.on("end", data => {
  console.log("Token " + token + ": closing connection.");
});

This seemed to do the trick until I realized that the "data" event was not being emitted as soon as the code was printing to stdout but it was being buffered. Although buffering is probably better for some use cases I wanted an IDE-like experience where the output feels like a terminal output.

The unmaintained choice

While I'm sure there is a way to persuade the node child process to not buffer the output my research actually lead me to pty.js which addressed both my requirements. As an added bonus the code didn't have to change much:

const { spawn } = require("pty.js");

var pyProcess = spawn("python", [scriptPath]);

pyProcess.on("data", data => {
  console.log(data);
});

pyProcess.on("exit", exitCode => {
  console.log("Exiting with code " + exitCode);
});

However, when jumping from macos to linux and back I noticed that the behavior when the process completed was not consistent. More specifically the "exit" event did not seem to emitted on macos. Soon I realized that in linux the event was also not getting triggered for the right reasons - when the python code finished it would throw an error and cause the "exit" event to be kicked off.

After tinkering for a while with the lib locally and going through its github issues I was sure this was an issue with pty.js. I also realized that the project seemed to be abandoned even though there were quite a few pull requests.

Fork this!

Fortunately, a kind soul has a forked this project to NodePty and has actively maintained it. Funnily enough I found this fork because there is a pending PR on pty.js to declare it unmaintained and point to NodePty.

Everything I wanted just worked by switching the dependency and replacing the import with:

const { spawn } = require("node-pty");

It's amazing how many amazing tools are being being built and maintained by the OSS community out there. These are just a fraction of the options I had to build what I want and that's just awesome!