DEV Community

Cover image for Python + JavaScript = 🔥🔥🔥
Shuvo
Shuvo

Posted on

Python + JavaScript = 🔥🔥🔥

When in comes to Web Development nothing beats JavaScript. But sometimes we have to do a bit more demanding task, for example analyzing big pile of data. In that case Python might be a superior option. But that's just one function of our website. Do we want to switch to Python just because of that one feature? Probably not.

So what if we could build our backend mostly using NodeJS and only use Python when we have to.

That would be awesome right? We can use child process in Node.JS to run a python script when needed.

const spawn = require('child_process').spawn
app.get("process_data", (req, res) => {
    spawn('python3', ['script.py'])
})
Enter fullscreen mode Exit fullscreen mode
# script.py
doSometing()
Enter fullscreen mode Exit fullscreen mode

And if we want we can pass data to our python script also.

const spawn = require('child_process').spawn
app.get("process_data", (req, res) => {
    const msg = "Hello"
    spawn('python3', ['script.py', msg])
})
Enter fullscreen mode Exit fullscreen mode

In Python in order to be able to read the data you must import the sys module.

import sys, json

def main():
    msg = sys.argv[1]
    doSometing(msg)

if __name__ == '__main__':
    main()
Enter fullscreen mode Exit fullscreen mode

Now instead on passing data while spawning the Python process, lets send data in stream.

const spawn = require('child_process').spawn,
const py = spawn('python3', ['script.py'])
const data = {
    msg: "Hello"
}

py.stdin.write(JSON.stringify(data)) //we have to send data as a string, so we are using JSON.stringify
py.stdin.end()
Enter fullscreen mode Exit fullscreen mode
import sys, json

def main():
    lines = sys.stdin.readlines()
    data = json.loads(lines)
    doSometing(data['msg'])

if __name__ == '__main__':
    main()
Enter fullscreen mode Exit fullscreen mode

Finally we can send response back to our nodejs from the python script

const spawn = require('child_process').spawn
const py = spawn('python3', ['cscript.py'])

py.stdout.on('data', function(res){
   let data = JSON.parse(res.toString())
   console.log(data)
})
Enter fullscreen mode Exit fullscreen mode
import sys

# You will have your own implementation of get data. In this case lets assume it returns a dict/json
res = getData()
print(json.dumps(data))

sys.stdout.flush()
Enter fullscreen mode Exit fullscreen mode

So this article has come to an end. But make sure you check out my other articles.

0shuvo0 image

Discussion (14)

Collapse
aymanbenali profile image
aymanbenali

i was working on a similar project that i needed to pass data from node js to a python script, i think you may use flask better and create an endpoint to send and receive data between them, it can help you in using asynchronous functions

Collapse
0shuvo0 profile image
Shuvo Author

great approach

Collapse
tugrul profile image
Tuğrul Topuz

You have to make async process queue in Python side for this approach. In other case it will work wrong because node.js has shared context among clients and Python instance going to be shared because you execute it in global context. Other option is execute Python in callback or use messaging queue like rabbitmq.

Collapse
akatasonov profile image
Andrew Katasonov

Or just use Python and Flask :) A much more pleasant way to develop web apps than Node :) Everything web doesn't have to be JavaScript

Collapse
jovannyukraine profile image
Jovanny Ukraine

Or aiohttp which is faster and has native asyncio support.

Collapse
olavidps profile image
olavidps

I agree partially with the other comments, an easier and maybe more elegant approach could be to use just python with a framework in the backemd, but I find your article very useful for existing legacy project in nodejs where you need to run something in python (i.e. data processing, ml flows, etc), because sometimes it is more convenient and faster than to create a microservice. Thanks for the article, keep writing 🙌.

Collapse
0shuvo0 profile image
Shuvo Author

Many thanks 💓

Collapse
webreflection profile image
Andrea Giammarchi

related to this topic ... filebus uses a file watcher to notify each-other when either NodeJS sent something to process, or Python did ... have a look 👋

Collapse
jovannyukraine profile image
Jovanny Ukraine

Or you can run apache kafka and get an extremely fast, scalable and stable communication tunnel between your processes instead of shit described above.

Collapse
tomerl101 profile image
Tomer

Kafka is much more harder to maintain, and can increase the product cost.
So maybe the "shit" described above is can fit better than your "shit"

Collapse
mandarvaze profile image
Mandar Vaze

I think when sending data TO the python process, you'll need to write to stdout (on the node side) rather than stdin
Generally you write to stdout and read from stdin

Collapse
newtfrank profile image
Newton

Very interesting. Had no idea this can happen

Collapse
tonmoytalukder profile image
Tonmoy Talukder

I was also thinking about this from the last few weeks. Nice to have it here. 🥰

Collapse
0shuvo0 profile image
Shuvo Author

Glad you found it.