loading...

My third Side Project was my BCompSc Thesis: Remote Operating Theater

javierx2010 profile image Javier Escobar ・7 min read

Side Projects (3 Part Series)

1) My first Side Project: Windows Desktop Assistant 2) My second Side Project: Monopoly Game 3) My third Side Project was my BCompSc Thesis: Remote Operating Theater

Or, in spanish, "Sistema Quirófano Remoto".

Yes, my BCompSC thesis was my side project because I started a full-time remote job by the end of college and I only could be able to work on my thesis at night and on weekends. I had a tough yet very creative tutor and this was his idea:

"So, we have these analog video recording cards that we can attach to PCI ports and record the video signal of cameras and devices like laparoscopes during a surgical practice. I wonder if we can transmit it somewhere else with high quality so another surgeon could give advice or even teach remotely".

Yes, PCI, not even PCI Express. It was 2008 and there was no money for advanced hardware like that. By that year, commercial internet was also really slow in this corner of the world. 1 Mbps was considered extremely fast and there was no such thing as simultaneous HQ video with that speed.

"But, do you know what Internet 2 is? We have that type of Internet here and so other buildings in the city including the Urology Institute. With Internet 2 you can connect Valencia (Venezuela) and Valencia (Spain) with 56 Mbps and imperceptible delay".

Woohoo! I imagined a new world with that speed! But soon a realized that it isn't commercial so I wouldn't find YouTube on it. 😞

Alt Text

"So your duty, if you accept it, is to build some kind of software where you could transmit and let doctors assist, and review, remote surgeries letting them capture video signal from at least 4 cameras or equipment at the same time, hold video conferencing, and accessible through a web browser"

Alt text

It took me exactly 12 months to complete what was proposed. Mainly because the lack of time, and also testing phase took longer than what was expected because surgeries can't be scheduled based on my free time so I had to quit my job by the end of the project if I was willing to get my BCompSc degree by 2010.

I also wrote a paper, with my tutor, about this awesome project and you can get it here.

Deciding The Stack

Backend

  • PostgreSQL: I wanted so bad to use MySQL, but public university encourage the use of PostgreSQL for reasons that nobody knew and nobody knows. NoSQL was also an option but I had 0 expertise on that field so I played it safe. A database was needed to store user and surgery information so can be acceded later to review a past procedure for teaching or reviewing purposes.
  • PHP: Also, playing safe. My job was about building Drupal apps. So, PHP was the smart choice.

Middleware

  • Media Server: Well, you just can't stream video without something in the middle that could let you watch live or resume a video stream. You need a media server and there were a few options.

The following benchmark exposes an assessment we made in order to select which media server will be used. The were some open source solutions like Red5 and Wowza but Wowza documentation was so poor we couldn't test it without failing on every attempt.

Wowza

Red5

FMS

Multiplatform Yes Yes Yes
Open Source No Yes No
Performance Limited Limited High
Codebase Java Java C++
Documentation Not available Poor Good
Has RTMP Partial Yes Yes
Supported Video Codecs FLV, H.264 FLV FLV, H.264
Supported Audio Codecs FLV, MP3, AAC FLV, MP3 FLV, MP3, AAC
AMF0 and AMF3 Support Partial Partial Yes
Clustering Yes Yes Yes
Server Side Programming JS and limited AS3 JS AS3
Streaming Yes Yes Yes
Shared Objects Partial Partial Yes

Red5 worked pretty fine with low res videos but it couldn't handle 480p using OnVP6. On the other hand, we had Flash Media Server. Written in C++, it was fast and reliable even with resolutions like 720p (remember, 2008 and PIV computers). The free license of Flash Media Server only was able to handle up to 10 streams at the time and it was enough!

Also, streaming isn't the only thing it was able to handle, also Shared Objects so apps like realtime chat and real time whiteboards were possible to implement with such middleware.

  • AMFPHP: this middleware library was responsible for serializing AS3 objects so they could be transferred from and to the backend. It still exists on GitHub. But why am I mentioning Action Script here? Well, you'll find why in the frontend section.

Frontend

Analogous to the media server, we benchmarked 4 scenarios or technologies HTML & JS, Laszlo, Silverlight, and Flex. I won't show that table here, it is bigger than the other one and I don't want to annoy you (more...) 🙈

Before Angular, React or VueJS, there was something called Rich Internet Applications in a world where HTML5 & ECMAScript were not able to deliver desktop like solutions in your browser. Java Applets were already obsolete and Adobe and Microsoft decided to implement runtimes to fill this gap.

Adobe was saying that they Open Sourced an Object-Oriented framework (Adobe Flex) that would let you write amazing apps, using a language very similar to Java (Yes, Action Script 3) and style them using its API or CSS. Also, it would integrate flawlessly with Flash Media Server so we got a winner!

The following code shows how we use AMFPHP inside one of the classes of the project written in AS3. Follow the comments to understand it better.

package assets {
  import flash.events.*;
  import flash.net.NetConnection;
  import flash.net.Responder;

  public class PGDBConexion {
    //The file gateway.php has a class with a set of public methods prepared to receive an object
    private var gateway:String = "http://127.0.0.1/amfphp/gateway.php";
    private var conexion:NetConnection = new NetConnection();

    public function PGDBConexion() {         
      this.conexion.addEventListener(NetStatusEvent.NET_STATUS, netStatusHandler);
      this.conexion.connect(this.gateway);
    }

    private function netStatusHandler(event:NetStatusEvent):void {
      ...
    }

     public function call(funcion:String, responder:Responder):void {
       //The variable 'funcion' is the name of the method inside the php class in gateway.php
       //and responder is the callback function that will receive the response
       this.conexion.call(funcion, responder);
     }
  }
}

And when we need to call a particular endpoint or method we can do something like this

package assets {
  ...
  import flash.net.Responder;

  public class SelectorStream extends Panel {       
    ...     
    private var conexion:PGDBConexion = new PGDBConexion();
    private var responder:Responder;
    ...

    public function SelectorStream() {
      super();  
      ...       
      this.responder = new Responder(onResult, onFault);
      //This is were we invoke the getStreams method from gateway.php
      this.conexion.call("pgDB.getStreams", responder);
    }

    private function onResult(data:Object):void {
      //The object 'data' will contain the response already serialized to be used in AS3
      ...   
    }       

    private function onFault(fault:Object):void {
      ...
    }
  }
}

Formal Software Engineering

Let's remember this was a formal software engineering challenge so we had to documentate professionally everything so I'm sharing a few diagrams that describe the overall architecture of the solution. I used UML, of course. 🤓

Use Case Diagram

This is the Use Case diagram of one of the three possible roles that a user could have inside the system.
Use case diagram

Entity-Relationship Diagram

Entity-Relationship diagram

Class Diagram

This is a small subset showing only two types of relationships and methods
Class diagram

Deployment Diagram

Deployment Diagram

Final words

This Side Project was software engineering as its finest for me!

I did love this project, I researched, I tested, I delivered small increments using the MVP approach without knowing it was called that way.
I did something nobody has done before and that system is still running on a Sun Server at the Urology Institute here, in Valencia, Venezuela.

This was my last side project until 2020, where I started two new side projects. Now I have 10 additional years of experience and technology has changed. I'm willing to share more side projects with you guys!

Unlike the other ones, I do have the source code and binaries of this gem. 💎

Thanks for reading! And if you like it, then like it, clap it or share it! 😜

Side Projects (3 Part Series)

1) My first Side Project: Windows Desktop Assistant 2) My second Side Project: Monopoly Game 3) My third Side Project was my BCompSc Thesis: Remote Operating Theater

Discussion

markdown guide
 

Hi Javier,
I enjoyed the post 😊, tho it was amazingly coding full for me. But I got the message. I have questions tho, I'm currently building a webinar application with mobile app, like you said we have picked red5 as our middleman. Like others webinar we want to be able to streaming to over 10k to 100k viewers. So what do you think and my team agreed to use React and Flutter for the mobile version.

 

I haven’t used red5 in a while but I think the biggest problem you could challenge is concurrency and delay if the media server isn’t hosted on a powerful machine or nor have load balancing. Hopefully, VPS are faster these days. That was the problem we had with red5 10 years ago, too slow, but hardware has evolved. On the other side, I find interesting you’ll be using react! I’m kinda a VueJS person but react has plenty UI tools like video players that could help you. And one question, why don’t you start a MVP first using YouTube live? Is that a big team? Because you could focus first on the UI and later on the middleware.

I definitively would like to test your product one day. I think it will be amazing!

 

Thanks for your response.
So we are looking at broadcasting to over a million viewers. At speaking with red5, the price where off the roof😂. What streaming service can help me streaming over 500k users. One to many.

With no joke, YouTube through WebRTC :/