[TGrid] TypeScript Grid Computing Framework (open source failure stories)

Preface

In previous 2022, I had introduced my open source library typia here at dev.to, and thanks to your explosive responses, I was selected as "2022 top author" and received a gift. Also, I could get received a large number of stars from you.

In today, I'd like to introduce my other open source projects, that has been failed over 10 years. From those failure stories, I want to share my experiences and lessons learned from them.

Present from dev.to for being selected as "2022 top author"

TGrid

https://github.com/samchon/tgrid

TypeScript Grid Computing Framework(?)

TGrid is a type of RFC (TypeScript Remote Function Call) framework supporting websocket and worker protocols. With TGrid and RFC, it is possible to call remote system's function as if it were local function. Also, accessing to remote object is possible, too.

Below is an example code calling remote functions from client to server. The server is providing CompositeCalculator class instance and client is calling its functions through Driverinstance, with async and await statements. This is the RFC (Remote Function Call).

composite-calculator/server.ts

import { WebServer } from "tgrid/protocols/web";
import { CompositeCalculator } from "../../providers/Calculator";

async function main(): Promise<void> {
    const server: WebServer<object, CompositeCalculator> = new WebServer();
    await server.open(10102, async (acceptor) => {
        await acceptor.accept(new CompositeCalculator());
    });
}
main();

composite-calculator/client.ts

import { WebConnector } from "tgrid/protocols/web/WebConnector";
import { Driver } from "tgrid/components/Driver";

import { ICalculator } from "../../controllers/ICalculator";

async function main(): Promise<void> {
    //----
    // CONNECTION
    //----
    const connector: WebConnector<null, null> = new WebConnector(null, null);
    await connector.connect("ws://127.0.0.1:10102");

    //----
    // CALL REMOTE FUNCTIONS
    //----
    // GET DRIVER
    const calc: Driver<ICalculator> = connector.getDriver<ICalculator>();

    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    } catch (err) {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main();

1 + 6 = 7
7 * 2 = 14
3 ^ 4 = 81
log (2, 32) = 5
SQRT (-4) -> Error: Negative value on sqaure.
Mean (1, 2, 3, 4) = 2.5
Stdev. (1, 2, 3, 4) = 1.118033988749895

Documentation

https://tgrid.com/en/

For TGrid, great effort was put into documentation, such as creating manuals for various functional units, explaining concepts, and teaching how to use them through example codes.

At the same time, various tutorial projects were also provided. Starting with a simple chat service using websocket, grid markets that can buy and sell computing power, and mutex-servers that can control critical areas at the level of the entire network system have been developed.

Chat Application

Simple chat application using websocket protocol.

• Guide Documents: https://tgrid.com/en/tutorial/projects/chat.html
• Demo site: http://samchon.org/chat
• Code repository: https://github.com/samchon/tgrid.projects.chat

Grid Market

A marketplace which can trade computing power.

• Guide Documents: https://tgrid.com/en/tutorial/projects/market.html
• Demo site: http://samchon.org/market
• Code repository: https://github.com/samchon/tgrid.projects.market

Mutex Server

https://github.com/samchon/mutex

Critical sections in the network level.

• RemoteConditionVariable
• RemoteMutex (both read and write locks)
• RemoteSemaphore
• RemoteBarrier
• RemoteLatch

import msv from "mutex-server";
import std from "tstl";

const PASSWORD = "qweqwe123!";
const PORT = 37119;

async function client(index: number, character: string): Promise<void> {
    // CONNECT TO THE SERVER
    const connector: msv.MutexConnector<string, null> 
        = new msv.MutexConnector(PASSWORD, null);
    await connector.connect(`ws://127.0.0.1:${PORT}`);

    // GET LOCK
    const mutex: msv.RemoteMutex = await connector.getMutex("printer");
    await mutex.lock();

    // PRINTS A LINE VERY SLOWLY MONOPOLYING THE MUTEX
    process.stdout.write(`Connector #${index} is monopolying a mutex: `);
    for (let i: number = 0; i < 20; ++i) {
        process.stdout.write(character);
        await std.sleep_for(50);
    }
    process.stdout.write("\n");

    // ALTHOUGH THE CLIENT DOES NOT RELEASE THE LOCK
    if (Math.random() < 0.5)
        await mutex.unlock();
    else // SERVER WILL UNLOCK IT AUTOMATICALLY AFTER THE DISCONNECTION
        await connector.close();
}

async function main(): Promise<void> {
    // OPEN SERVER
    const server: msv.MutexServer<string, null> = new msv.MutexServer();
    await server.open(PORT, async acceptor => {
        if (acceptor.header === PASSWORD)
            await acceptor.accept(null);
        else
            await acceptor.reject();
    });

    // CREATE 10 CLIENTS LOCKING MUTEX
    const promises: Promise<void>[] = new Arrray(4).fill("").map(
        (_, i: number) => {
            const character: string = std.randint(0, 9).toString();
            return client(i + 1, character);
        }
    );

    // WAIT THE CLIENTS TO BE DISCONNCTED AND CLOSE SERVER
    await Promise.all(promises);
    await server.close();
}
main();

Problems

For TSTL

As I'd mentioned in previous article, my 1st library TSTL has been failed to famouse.

However, I wish I had given up right away and started another project, but since it took so much study and effort to make TSTL, something was a pity. Therefore, by inventing the new concept RFC (Remote Function Call) and creating TGrid, he tried to make TGrid famous and revive TSTL at the same time.

And the result was, the second failed open source project has been newly borned. Liquidating the stocks of the company that went bankrupt as quickly as possible, and having to find new stocks, but I ended up buying additional stocks of the bankruptcy.

• TSTL issue: network & thread extesion
• Open source failure stories - TSTL

Wrong Subject

TGrid is a RFC framework using websocket and worker protocols.

However, I'd concentrated on the Grid Computing keyword, instead of the RFC keyword. It was to jump on the bandwagon of the popularity of blockchain at the time of 2017, but looking back, I don't think it was the decisive cause of TGrid's failure. Should have only focused on the key words.

If I'd only concentraed on the RFC keyword, emphasized websocket and worker protocols, could TGrid be famous?

Too Large Project

TGrid was an overly large project for an open source novice developer to make, as TSTL was.

Developing TGrid never be ended up with main program implementation.
The open source developments ends up after writing lots of guide documents and developing dozens of example projects, of documentation and tutorial project developments.

Developing TGrid, I could understand what "tip of the iceberg" was. After experiencing TSTL and TGrid, I decided to start from much small project again.

The latest reason may be I had not known famous developer communities like this dev.to. I hand't known how to advertise and introduce my open source projects to massive developers.