DEV Community

Cover image for Uploading multiple files at the same time using multithreading in NodeJS

Uploading multiple files at the same time using multithreading in NodeJS

Wesley Miranda on March 19, 2023

NodeJS is well known for being single-threading, but it is not true, because only the event-loop is handled by a single thread. NodeJS gives to us ...
Collapse
 
miketalbot profile image
Mike Talbot ⭐

You don't need multiple threads to upload multiple files in parallel in Node - the single threaded part is only on the instruction pipeline, you could easilly issue 100 uploads and have them run in parallel using a Promise.all() - the Async library has lots of useful calls to batch up things too if you don't just want to start all at once.

Multiple threads and processes are very handy if you are actually doing processing in your Javascript code, where other operations would be blocked.

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Makes sense! thanks for your comment!

Collapse
 
thomasbnt profile image
Thomas Bnt

Hello ! Don't hesitate to put colors on your codeblock like this example for have to have a better understanding of your code 😎

console.log('Hello world!');
Enter fullscreen mode Exit fullscreen mode

Example of how to add colors and syntax in codeblocks

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Thanks a lot! I didnt know that

Collapse
 
vigneshpa profile image
vignesh

You don't need multithreading to upload files concurrently in NodeJS.

NodeJS is not C++ where I/O operations are synchronous by default.

All I/O operations are always async in NodeJS. There are some exceptions though, like synchronous I/O APIs but they are very bad for performance so don't use them.

NodeJS uses libuv to issue non-blocking I/O syscalls and to get I/O event notifications.

Create Workers in NodeJS, only if you are doing CPU bound operations like computing hash of a large ArrayBuffer, or a very simple example would be finding a prime number. These operations will put heavy load on the CPU thread and prevent other tasks from executing.

It is always better to do I/O based operations in an async event loop as OS threads/processes are very expensive. When doing I/O, most of the time, our application spends waiting for a Disk or a Network Device.

If you know Rust see Tokio and Rayon.

See here to learn how Nginx delivers high performance with Non Blocking I/O.

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Makes sense! thanks for your comment!

This approach I used to show how to create worker threads, perhaps is not the best!

Collapse
 
joelbonetr profile image
JoelBonetR 🥇

Awesome insights!

Thanks a lot for sharing such concepts on a perfectly reasonable use-case, 10 out of 10 😁

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Thanks a lot for your comment!

I am planning to write more contents to show the things under the hood.

Collapse
 
joelbonetr profile image
JoelBonetR 🥇

That would be amazing! I'm following you to read more about these topics as soon as you publish them 😄

Collapse
 
ricardo_borges profile image
Ricardo Borges

Nice article, about that conceptual part in the introduction, wouldn't child_processbe an approach for multiprocessing instead of multithreading?

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Good point! Parallel processing should be the best word for that!

thanks for your comment!

Collapse
 
bias profile image
Tobias Nickel

i like the 'piscina' library for using multiple thread in node.js, however shared memory would still be awesome.

Collapse
 
wesleymreng7 profile image
Wesley Miranda

Nice! I know it. It was created by a brazilian guy I guess.

Some comments have been hidden by the post's author - find out more