Released a Library for Synchronous Execution of Asynchronous Processes in JS/TS

I have released a library called sync-actions that allows asynchronous processes to be executed synchronously in JavaScript/TypeScript. Especially in TypeScript, you can call defined functions in a type-safe manner. It is intended for use in cases where you want to execute asynchronous processes within functions that you do not want (or cannot) mark as async.

Features

• Utilizes Node.js worker_threads
  • Asynchronous processes are executed in a sub-thread, and the main thread waits synchronously for their completion.
• Type-safe function calls
  • In TypeScript, you can utilize the type information of defined functions.
• Published as Native ESM
  • It is made simple by not supporting CommonJS.

Repository

https://github.com/koyopro/sync-actions

Usage

Installation

It is published as an npm package, so please install it using npm install or similar.

npm install sync-actions

Basic Usage

By passing an asynchronous function that returns a Promise object to defineSyncWorker(), you can define the interface and start the worker thread with launch(). It is assumed that the file defining the worker is created separately from other processing files.

// worker.js
import { defineSyncWorker } from "sync-actions";

export const { actions, worker } = defineSyncWorker(import.meta.filename, {
  ping: async () => {
    // Execute asynchronous process,
    await new Promise((resolve) => setTimeout(resolve, 1000));
    // Return the result as a return value
    return "pong";
  }
}).launch();

// main.js
import { actions, worker } from "./worker.js";

// You can execute asynchronous functions synchronously
console.log(actions.ping()); // => "pong" is output after 1 second

worker.terminate();

Type-safe Function Calls

In TypeScript, you can call functions defined with defineSyncWorker in a type-safe manner.

// worker.ts
import { defineSyncWorker } from "sync-actions";

export const { actions, worker } = defineSyncWorker(import.meta.filename, {
  // By specifying the types of arguments and return values, type-safe calls are possible
  add: async (a: number, b: number): Promise<number> => {
    return a + b;
  }
}).launch();

// main.ts
import { actions, worker } from "./worker.js";

// Type-safe call
actions.add(1, 2); // => 3 (number)

// @ts-expect-error
actions.add("1", 2);
// => Argument of type 'string' is not assignable to parameter of type 'number'

worker.terminate();

Background

The content so far is the same as the README, so I will describe the background of its creation.

I am developing an ORM called Accel Record.¹ Unlike general ORMs, Accel Record is designed to perform DB access with a synchronous interface.² The part that executes DB access synchronously was realized by executing asynchronous processes in a subprocess started with the child_process module.³ I thought that by using worker_threads instead of child_process, I could reduce the overhead at runtime.

Accel Record is also designed to be similar to Ruby on Rails' Active Record in terms of usability, and one of the things I want to achieve in the future is to create a library like CarrierWave. CarrierWave allows you to save images to external storage services (such as AWS S3) when saving records, and to achieve this with Accel Record, it is necessary to execute asynchronous processes such as image uploads synchronously. I expect that this process can be executed faster by using worker_threads instead of subprocesses.

So I once looked for a library that synchronously executes asynchronous processes using worker_threads. I found several libraries such as synckit and deasync, but none of them worked as expected on my end, so I decided to create my own. Since I was at it, I thought I would make an interface that can be used in a type-safe manner with TypeScript.

More Internal Details

• Until the asynchronous process in the sub-thread started with worker_threads is completed, the main thread is blocked using Atomic.wait().
• MessageChannel is used for communication between threads. The source code of synckit was very helpful for this part of the implementation.
• When starting a Worker with worker_threads, it is necessary to transpile the .ts file to .js. For that part, I am using esbuild.
• When starting a Worker, I wanted to pass the transpiled source code as a string to the Worker for execution, but it did not work properly in my environment. This is the part where I struggled the most. In the end, I wrote the file under node_modules and passed its path to the Worker. This method turned out to be the most stable.

1. Introduction to "Accel Record": A TypeScript ORM Using the Active Record Pattern ↩

2. Why We Adopted a Synchronous API for the New TypeScript ORM ↩

3. Techniques for Synchronous DB Access in TypeScript ↩