DEV Community

Cover image for Screen Sharing with WebRTC: Harnessing JavaScript for Seamless Streaming
Filipe Melo
Filipe Melo

Posted on

Screen Sharing with WebRTC: Harnessing JavaScript for Seamless Streaming

Introduction

WebRTC is a technology that allows web applications to exchange data directly with other browsers, "without" an intermediary. It uses a variety of protocols that work together to achieve this.
Unfortunately, webRTC cannot initiate a connection on its own, so we need a server that will transmit the connection Offer between the clients: we call it Signal Channel or Signaling Server; once the Offer is accepted, the browsers can then share information between themselves.
In this article, we are going to create a screen-sharing application, using webRTC and a Websocket server as our Signaling Server (which we are also going to build ourselves).

WebRTC + Signaling server connection workflow

Creating the initial project

To keep this project simple, we are going to start a React + Typescript project using Vite:

npm create vite scrshr-app
Enter fullscreen mode Exit fullscreen mode

React Project setup with Vite

For now, we are going to modify the App.tsx by adding a form with an input and two buttons. We are going to use this form to allow users to create/join a stream "room".

You can also delete some of the files it generated, such as App.css and index.css.

// App.tsx

import { useAppState } from "./app.state"

function App() {

  const {
    inputRef,
    onCreateRoom,
    onJoinRoom
  } = useAppState();
  return (
    <>
      <form>
        <input type='text' ref={inputRef}/>
        <button type="button" onClick={onJoinRoom}>Join</button>
        <button type="button" onClick={onCreateRoom}>Create</button>
      </form>
      <video width="320" height="240"/>
    </>
  )
}

export default App
Enter fullscreen mode Exit fullscreen mode
// app.state.ts

import { useRef } from "react"

export function useAppState() {

  const inputRef = useRef<HTMLInputElement>(null);

  const onCreateRoom = () => {
    console.log("Create room")
  }

  const onJoinRoom = () => {
    console.log("Join room")
  }

  return {
    onCreateRoom,
    onJoinRoom,
    inputRef
  }
}

Enter fullscreen mode Exit fullscreen mode

Vite App running locally

Now that we have the UI components on the screen, we can start building the logic for transmitting the webRTC offers with WebSocket. So before we continue with the code, we need to install the Socket.io:

npm i socket.io-client
Enter fullscreen mode Exit fullscreen mode

Handling the Websocket Events (Client)
Let's go ahead now and create a new file called websocket.service.ts. In this file, we are going to implement our business rule related to the websocket connection. There, we will add listeners for the events, and also, trigger our events to the server. Below, is a list of all the events we will be triggering/listening to, and their purpose:

  • createRoom: this event will be responsible for creating a "room" in the backend, so we can listen to it when a new user joins the room;
  • joinRoom: this event will be triggered when the user clicks on the "join room" button;
  • onNewUserJoined: this event will tell the room's owner that a user has joined, therefore, it will be able to send an offer;
  • sendOffer: this event will be triggered once a new user joins; we will send the webRTC Offer object to the new user;
  • receiveOffer: this event will bring the offer to the user who's just joined, that way they can trigger the sendAnswer method explained below;
  • sendAnswer: this event will be triggered by the user who's received an Offer, allowing them to respond with their webRFC Answer object;
  • receiveAnswer: this event will be triggered once the user who joined has responded, allowing them to finally communicate.

It is important to note that there are some more events we will need to implement, but we will get into more details further.

import { Socket, io } from "socket.io-client";

export class WebsocketService {
  websocket: Socket;
  constructor() {
    this.websocket = io("http://localhost:3000");
  }

  createRoom(roomName: string) {
    this.websocket.emit("room/create", {
      roomName,
    });
  }

  joinRoom(roomName: string) {
    this.websocket.emit("room/join", {
      roomName,
    });
  }

  onNewUserJoined(callback: () => void) {
    this.websocket.on("user/joined", callback);
  }

  sendOffer(roomName: string, offer: RTCSessionDescriptionInit) {
    this.websocket.emit("offer/send", {
      offer,
      roomName,
    });
  }

  receiveOffer(callback: (offer: RTCSessionDescriptionInit) => void) {
    this.websocket.on("offer/receive", ({ offer }) => callback(offer));
  }

  sendAnswer(roomName: string, answer: RTCSessionDescriptionInit) {
    this.websocket.emit("answer/send", {
      answer,
      roomName,
    });
  }

  receiveAnswer(callback: (answer: RTCSessionDescriptionInit) => void) {
    this.websocket.on("answer/receive", ({ answer }) => callback(answer));
  }
}
Enter fullscreen mode Exit fullscreen mode

Creating the Signaling Server

We can get started creating the Signaling Server. Let's create a new directory called server and run the following commands:

npm init -y
Enter fullscreen mode Exit fullscreen mode

This will generate for us the package.json.
Since we are using typescript on the Front-End, I will add some extra steps here if you want to follow along, but this is entirely optional.

npm install typescript ts-node-dev -D
Enter fullscreen mode Exit fullscreen mode

Now we can initiate the typescript by running the following command:

npx tsc --init
Enter fullscreen mode Exit fullscreen mode

I made a few adjustments to the tsconfig.json and package.json files, here is what they look like:

{
  "compilerOptions": {
    "target": "es2020",
    "module": "commonjs",
    "rootDir": "./src",
    "outDir": "./build",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  }
}
Enter fullscreen mode Exit fullscreen mode
{
  "name": "server",
  "version": "1.0.0",
  "description": "",
  "main": "./src/server.ts",
  "scripts": {
    "start": "node ./build/server.js",
    "dev": "ts-node-dev .",
    "build": "tsc "
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "ts-node-dev": "^2.0.0",
    "typescript": "^5.4.2"
  }
}
Enter fullscreen mode Exit fullscreen mode

Now we can install both Express and Socket.io libraries:

npm install express socket.io
npm install @types/express -D
Enter fullscreen mode Exit fullscreen mode

To make this quick, we can follow the steps to setup socket.io along with express described here:

//server.ts
import express from "express";
import { createServer } from "node:http";
import { Server } from "socket.io";

const app = express();
const httpServer = createServer(app);

const io = new Server(httpServer, {
  cors: {
    origin: "http://localhost:5173",
  },
});

io.on("connection", () => {
  console.log("connected");
});

httpServer.listen(3000);
Enter fullscreen mode Exit fullscreen mode

Finally, we can create a new file and place where the event listeners to handle the WebSocket requests coming from the clients, as we saw earlier.

//room.service.ts

import { Server } from "socket.io";

interface Room {
  owner: string;
  guest: string | null;
};

export class RoomService {
  rooms = new Map<string, Room>();

  constructor(private server: Server) {
  }

  initialize() {
    this.server.on("connection", (socket) => {
      console.log("user connected", socket.id)
    })
  }

}
Enter fullscreen mode Exit fullscreen mode

And make sure to instantiate this service on the server.ts file as shown below:

//server.ts
import express from "express";
import { createServer } from "node:http";
import { Server } from "socket.io";
import { RoomService } from "./room.service";

const app = express();
const httpServer = createServer(app);

const io = new Server(httpServer, {
  cors: {
    origin: "http://localhost:5173",
  },
});

const roomService = new RoomService(io);
roomService.initialize();

httpServer.listen(3000);
Enter fullscreen mode Exit fullscreen mode

To give this all a try, let's make a few adjustments on the Front-End client.

// app.state.ts
...
const socketService = new WebsocketService();

export function useAppState() {
  ...
  const onCreateRoom = () => {
    if (!inputRef.current?.value) {
      return;
    }
    socketService.createRoom(inputRef.current.value)
  }
  ...
}
Enter fullscreen mode Exit fullscreen mode

picture showing the logs of both the Client and Server running

As you can see above, we have the server printing that our client has connected to the websocket successfully. We can now implement each handler to handle each event, as well as emit events when necessary.

Handling the Websocket Events (Server)

Let's go ahead now and implement all the logic we need on the server so it can exchange information.

// room.service.ts

import { Server, Socket } from "socket.io";

interface Room {
  owner: string;
  guest: string | null;
}

export class RoomService {
  rooms = new Map<string, Room>();

  constructor(private server: Server) {}

  initialize() {
    this.server.on("connection", (socket) => {
      this.handleOnCreateRoom(socket);
      this.handleOnJoinRoom(socket);
      this.handleOnSendOffer(socket);
      this.handleOnSendAnswer(socket);
    });
  }

  handleOnCreateRoom(socket: Socket) {
    socket.on("room/create", (payload) => {
      const { roomName } = payload;
      this.rooms.set(roomName, {
        owner: socket.id,
        guest: null,
      });
    });
  }

  handleOnJoinRoom(socket: Socket) {
    socket.on("room/join", (payload) => {
      const {roomName} = payload;

      const room = this.rooms.get(roomName);

      if (!room) {
        return;
      }
      room.guest = socket.id;
      socket.to(room.owner).emit("user/joined");
    })
  }

  handleOnSendOffer(socket: Socket) {
    socket.on("offer/send", (payload) => {
      const { roomName, offer } = payload;

      const room = this.rooms.get(roomName);

      if (!room || !room.guest) {
        return;
      }
      socket.to(room.guest).emit("offer/receive", {
        offer
      })
    })
  }

  handleOnSendAnswer(socket: Socket) {
    socket.on("answer/send", (payload) => {
      const { answer, roomName } = payload;

      const room = this.rooms.get(roomName);

      if (!room) {
        return;
      }

      socket.to(room.owner).emit("answer/receive", {
        answer
      })
    })
  }
}
Enter fullscreen mode Exit fullscreen mode

Implementing the WebRTC service
Now, let's get started setting up our webRTC service. Firstly, we start by setting up the RTC Peer Connection Object:

// webrtc.service.ts
export class WebRTCService {
  private peerConnection: RTCPeerConnection;
  constructor() {
    const configuration: RTCConfiguration = {
      iceServers: [{ urls: "stun:stun.l.google.com:19302" }],
    };
    this.peerConnection = new RTCPeerConnection(configuration);
  }
  ...
}
Enter fullscreen mode Exit fullscreen mode

Then, we are going to implement the following methods for now:

  • makeOffer: this method will be responsible for creating the "offer", which will contain all the information about the current connection, including its media streams;
  • makeAnswer: this method, similar to make offer, creates an answer for an offer;
  • setRemoteOffer: once we receive either an answer or offer from a remote client, we pass it down to the setRemoteDescription;
  • setLocalOffer: once we make an offer or an answer, we need to specify the properties of the local end of the connection;
  • getMediaStream: using the MediaDevices API, we can capture the contents of a display (or specific app), and stream it over the webRTC connection;
  • onStream: using the ontrack event, we can receive the stream coming over the webRTC stream.
// webrtc.service.ts
export class WebRTCService {
  private peerConnection: RTCPeerConnection;

  constructor() {
    const configuration: RTCConfiguration = {
      iceServers: [{ urls: "stun:stun.l.google.com:19302" }],
    };
    this.peerConnection = new RTCPeerConnection(configuration);
  }

  async makeOffer() {
    const offer = await this.peerConnection.createOffer();
    return offer;
  }

  async makeAnswer() {
    const answer = await this.peerConnection.createAnswer();
    return answer;
  }

  async setRemoteOffer(offer: RTCSessionDescriptionInit) {
    await this.peerConnection.setRemoteDescription(new RTCSessionDescription(offer));
  }

  async setLocalOffer(offer: RTCSessionDescriptionInit) {
    await this.peerConnection.setLocalDescription(offer);
  }

  async getMediaStream() {
    const mediaStream = await navigator.mediaDevices.getDisplayMedia({
      video: true,
    });
    mediaStream.getTracks().forEach((track) => {
      this.peerConnection.addTrack(track, mediaStream);
    });
    return mediaStream;
  }

  onStream(cb: (media: MediaStream) => void ) {
    this.peerConnection.ontrack = function ({ streams: [stream] }) {

      cb(stream);
    };
  }
}
Enter fullscreen mode Exit fullscreen mode

Let's move back to the app.state.ts file and implement the logic there. We are going to start from the perspective of a user creating a room.

// app.state.ts
...
const webRTCService = new WebRTCService();

export function useAppState() {
  ...
  const onCreateRoom = () => {
    ...
    const roomName = inputRef.current.value
    socketService.createRoom(roomName)
    socketService.onNewUserJoined(async () => {
      const offer = await webRTCService.makeOffer();
      webRTCService.setLocalOffer(offer);
      socketService.sendOffer(roomName, offer);
    })

    socketService.receiveAnswer(async (answer) => {
      await webRTCService.setRemoteOffer(answer);
    })
  }
 ...
}
Enter fullscreen mode Exit fullscreen mode

As you can see above, once the user creates a room, we start listening to the websocket events. As soon as another user joins the session, we will be making the offer to send it over the websocket.
Finally, let's implement the business rule for when the user joins a session:

// app.state.ts
...
export function useAppState() {
...  
  const onJoinRoom = () => {
    if (!inputRef.current?.value) {
      return;
    }

    const roomName = inputRef.current.value

    socketService.joinRoom(roomName);
    socketService.receiveOffer(async (offer) => {
      await webRTCService.setRemoteOffer(offer);
      const answer = await webRTCService.makeAnswer();
      await webRTCService.setLocalOffer(answer);
      socketService.sendAnswer(roomName, answer);
    })
  }

...
}
Enter fullscreen mode Exit fullscreen mode

Since now the connection is being established, it is time we finally add the stream track to the connection. We are going to do this when the user creates the room, by calling the getMediaStream method. We are also, going to use its return so we can show our stream locally as well, using the video tag we added earlier.

// app.state.ts

...
export function useAppState() {
  ...
  const videoRef = useRef<HTMLVideoElement>(null);

  const onCreateRoom = async () => {
    if (!inputRef.current?.value) {
      return;
    }

    const roomName = inputRef.current.value;
    await startLocalStream();
    socketService.createRoom(roomName)
    ...
  }

  ...

  async function startLocalStream () {
    const mediaStream = await webRTCService.getMediaStream();
    startStream(mediaStream, true);
  }

  function startStream(mediaStream: MediaStream, isLocal = false ) {
    if (!videoRef.current) {
      return;
    }
    videoRef.current.srcObject = mediaStream;
    videoRef.current.muted = isLocal;
    videoRef.current.play();
  }

  return {
    ...
    videoRef
  }
}
Enter fullscreen mode Exit fullscreen mode
// App.tsx
...

function App() {

  const {
    ...
    videoRef
  } = useAppState();
  return (
    <>
      ...
      <video width="500" ref={videoRef}/>
    </>
  )
}

export default App
Enter fullscreen mode Exit fullscreen mode

webRTC local streaming

Lastly, we need to listen to a stream coming from the webRTC callback.

// app.state.ts

  const onJoinRoom = () => {
    ...
    webRTCService.onStream(startStream)
  }
Enter fullscreen mode Exit fullscreen mode

Fixing the latency problem

When starting a WebRTC peer connection, typically a number of candidates are proposed by each end of the connection, until they mutually agree upon one which describes the connection they decide will be best. WebRTC then uses that candidate's details to initiate the connection.

As you may have noticed, we don't get any errors when connecting, but we still cannot see the stream on the other client. This is happening because we haven't implemented the logic that will share the RTCIceCandidate. We are going to create a few more socket events so both clients can finally exchange information and connect much more quickly.
Firstly, let's implement the new methods in our webRTC events so we can listen to them when they change:

// webrtc.service.ts

export class WebRTCService {
 ...
  onICECandidateChange(cb: (agr: RTCIceCandidate ) => void ) {
    this.peerConnection.onicecandidate = (event: RTCPeerConnectionIceEvent) => {
      if (event.candidate) {
        cb(event.candidate);
      }
    }
  }

  async setICECandidate(candidate: RTCIceCandidate) {
    await this.peerConnection.addIceCandidate(candidate);
  }
}
Enter fullscreen mode Exit fullscreen mode

Now we are going to implement two new socket events: ice/send and ice/receive:

// websocket.service.ts

export class WebsocketService {
  ...
  sendIceCandidate(roomName: string, iceCandidate: RTCIceCandidate) {
    this.websocket.emit("ice/send", {
      roomName,
      iceCandidate
    })
  }

  receiveIceCandidate(cb: (arg: RTCIceCandidate) => void ) {
    this.websocket.on("ice/receive", ({iceCandidate}) => {
      cb(iceCandidate);
    })
  }
}
Enter fullscreen mode Exit fullscreen mode

On the backend, since either the owner or guest could send their ICECandidates; we can add a very simple verification to send the candidate to the appropriate client.

// room.service.ts
...
export class RoomService {
  ...
  initialize() {
    this.server.on("connection", (socket) => {
       ...
      this.handleOnSendIceCandidate(socket);
    });
  }

  ...

  handleOnSendIceCandidate(socket: Socket) {
    socket.on("ice/send", (payload) => {
      const { roomName, iceCandidate } = payload;

      const room = this.rooms.get(roomName);

      if (!room || !room.guest) {
        return;
      }

      const isOwner = room.owner === socket.id;
      const to = isOwner ? room.guest : room.owner;

      socket.to(to).emit("ice/receive", {
        iceCandidate,
      });
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

Finally, we can listen to the events when the user creates or joins:

// app.state.ts

...

  const onCreateRoom = async () => {
    ...
    setupIceCandidate(roomName)
  }

  const onJoinRoom = () => {
    ...
    setupIceCandidate(roomName)
  }

  function setupIceCandidate (roomName: string) {
    webRTCService.onICECandidateChange((candidate) => {
      socketService.sendIceCandidate(roomName, candidate)
    })
    socketService.receiveIceCandidate(webRTCService.setICECandidate)
  }
...
Enter fullscreen mode Exit fullscreen mode

webRTC streaming (final app)

Top comments (0)