Building a Shared Code-Editor using Node.js, WebSocket and CRDT

#webdev #websocket #node #mongodb

How do collaborative editing platforms like Google Docs, codeshare.io or excalidraw work?

These platforms provide real-time collaboration, synchronization with other users and conflict-resolution mechanisms.

Real-time communication and synchronization of document can be achieved by using WebSocket, since it enables instant updates to be sent and received as the users make an update. For example, when a change is made by User A, this information is sent to the server via WebSocket, the server then broadcasts this change to other connected users.

The difficult part is conflict-resolution. You may be aware how difficult it is to resolve merge conflicts in a git repository. Imagine automating this process 😬.

Fortunately, there are many big brain people working on distributed conflict-resolution algorithms.

Mainly, there are 2 techniques to do this:

Operational Transformation (OT)
Conflict-Free Replicated Data Types (CRDTs)

I won’t dive deep into the algorithmic details of each technique, but I’ll quote few lines from this blog.

The difference is that with OT (Operational Transformation) any time you edit the document your edits have to be sent via a single server. And Google provides that server in the case of Google Docs. So all your communication, all your collaboration has to go via this one server.

And CRDTs are different because they are decentralized. They don’t require a single server to work. But instead, you can sync your devices via any kind of network that happens to be available._

I chose CRDTs for this project as there are many open-source implementations available with well-written documentation.

Bird’s-eye view

All of the platforms have a similar user flow which looks something like this:

User A creates a new room/document
The application provides a unique invite URL / invite code
User A shares this invite code to User B
User B enters the invite code on the app and joins the room/document
User A or User B selects a programming language and starts coding

I’ll use the following schema to store relevant information about a room:

interface Room {
    roomId: string,
    owner: string,
    dateCreated: Date,
    participants: string[],
    programmingLanguage: string
}

And of course, the content of the document can also be stored in the database.

Now, as we know the schema of the data that needs to be stored, we can design our API endpoints for room operations.

Front-end React application

I generated a React + TypeScript project using Vite and used Material UI v5 component library.

A basic home page with a Create Room button and Join Room input field would be enough, as shown below:

For the code room page, we’ll be using yjs, which is a CRDT implementation and has editor bindings for editors like Prose Mirror, Quill, Monaco Editor etc.

We’ll be using Monaco Editor for this application. So, we need the Monaco Editor component for react and Monaco Editor binding from yjs. We’ll also need the WebSocket module of yjs which will propagate changes in the editor to the server. Below are the commands to install all of these modules:

npm install monaco-editor yjs y-monaco y-websocket

Code Room page

Below is the code for the Code Room page:

Create a Monaco Editor component in your Room page
When the editor component mounts, we create a Y.Doc(), which is a shared data structure
Then we connect to the back-end server via a WebSocketProvider from y-websocket and set the shared data type with it.
At last, we bind the Monaco Editor to this shared data type

import { useTheme } from "@mui/material";
import { useState, useRef, useEffect } from "react";
import { Editor } from "@monaco-editor/react";
import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';
import { MonacoBinding } from 'y-monaco';
import { editor } from "monaco-editor";

const serverWsUrl = import.meta.env.VITE_SERVER_WS_URL;

export default function CodeRoom() {
    const theme = useTheme();

    const editorRef = useRef<editor.IStandaloneCodeEditor>();

    function handleEditorDidMount(editor: editor.IStandaloneCodeEditor) {
        editorRef.current = editor;

        // Initialize yjs
        const doc = new Y.Doc(); // collection of shared objects

        // Connect to peers with WebSocket
        const provider: WebsocketProvider = new WebsocketProvider(serverWsUrl, "roomId", doc);
        const type = doc.getText("monaco");

        // Bind yjs doc to Manaco editor
        const binding = new MonacoBinding(type, editorRef.current!.getModel()!, new Set([editorRef.current!]));

    }

    return (
        <>
        <Editor 
            height="100vh"
            language={"cpp"}
            defaultValue={"// your code here"}
            theme={theme.palette.mode === "dark" ? "vs-dark" : "vs-light"}
            onMount={handleEditorDidMount}
        />
        </>
    );
}

Back-end Express, Node.js, WebSocketServer

The back-end consists of a simple express server and a WebSocketServer that listens to the ‘connection’ event and sets up a connection using a utility file provided by y-websocket module. You can explore the setupWSConnection file for a deeper understanding.

import express, { Request, Response } from 'express';
import { createServer } from 'http';
import cors from 'cors';
import { logger } from './logger';
import { WebSocketServer } from 'ws';
const setupWSConnection = require('y-websocket/bin/utils').setupWSConnection;

/**
 * CORSConfiguration
 */
export const allowedOrigins = ['http://localhost:5173'];

/**
 * Server INITIALIZATION and CONFIGURATION
 * CORS configuration
 * Request body parsing
 */
const app = express();
app.use(cors(
  {
    origin: allowedOrigins,
    methods: "GET,HEAD,PUT,PATCH,POST,DELETE",
    allowedHeaders: "Content-Type",
    credentials: true
  }
));
app.use(express.json());


/**
 * Create an http server
 */
export const httpServer = createServer(app);

/**
 * Create a wss (Web Socket Secure) server
 */
export const wss = new WebSocketServer({server: httpServer})

function onError(error: any) {
  logger.info(error);
}

function onListening() {
  logger.info("Listening")
}

httpServer.on('error', onError);
httpServer.on('listening', onListening);

/**
* On connection, use the utility file provided by y-websocket
*/
wss.on('connection', (ws, req) => {
  logger.info("wss:connection");
  setupWSConnection(ws, req);
})

And Voila! You now have a shared code-editor. There are more features that I added here:

Added a drop-down list of programming languages on the front-end, listening to the language change event by a user on the back-end and then broadcasting it to other peers. Created a separate WebSocket connection for these using Socket.IO server and client.
Similarly, whenever a new participant enters a room, the participants list is displayed on the front-end and in the same way, this information is propagated through the back-end to other peers.
Connected the back-end server with MongoDB cloud instance, for storing room information.