DEV Community

Cover image for Video+Audio conferencing in your terminal !
darker
darker

Posted on

Video+Audio conferencing in your terminal !

Yeah i know, what you guys are thinking "Why ?", "Why do you do that ?", well "just for fun ?" and a way to mastering "real-time parallelism", this is also my first POST ever made!

Let's jump into it !
First of all, a videoconferencing system is a real-time application which consists in broadcasting to each person connected the audio and video stream of all the other participants with the least possible latency !

Video-Conf

As I had to work on the terminal, I only had characters and "bytes" for data entry!
Bytes

First question : how to transform an image into a perceptible string of characters on a terminal ?

My algorithm is very simple, I first use OpenCv to retrieve the image that the camera /dev/0 device gives me, after that, i convert to black on white using the cvtColor method from OpenCV; then i cut this image into a sort of grid which gives me the color contrast by squares ranging from 0 to 255, I have a sort array equivalence of each pixel! By setting another array of characters ranging from "" to "B", I have a match against the occupancy rate in the pixel !
To complicated ?
Complicated

Let me explain again, more a pixel is dark, the less dense the character will be, so we will have either a "." or a "," with the color value 10 or 12 unlike "B" or "K" which is closer to 250 or 255 From there, I manage to have my transcription of a color image in characters on a terminal !
ASCII

Second question: how to retrieve the audio stream and transmit it ?

To do this I use PyAudio which connects to the microphone to retrieve the audio stream, then transmit it in the form of bytes per sequences, at the reception, i do a remodeling according to bit rate !

Third question: how to send and retrieve the stream(video and audio) for each session ?

Here I am using a byte formatting system in json by adding elements allowing me to identify the one sending and in which session it is sending it, when a client retrieves the data, analyzes it, then retrieves the contents of the package
Sessions

Fourth question: how does security work in such a system ?

I have implemented two types of data encryption, asymmetric encryption and symmetric encryption, for both systems there is key sharing that needs to be done first, but in the version being tested I have disabled all encryption, as this has an impact on the final rendering quality!

AES (Symmetric encryption) RSA (Asymmetric encryption)
Alt Text Alt Text

To use s2c, just install it via pip !

pip install s2c
Enter fullscreen mode Exit fullscreen mode

Start the server :

s2c_server -p 1122
Enter fullscreen mode Exit fullscreen mode

Then start the client :

s2c -s session1 -c darker -i 127.0.0.1 -p 1122
Enter fullscreen mode Exit fullscreen mode

Or you can use help to see the manual :
Manual

Of course, all this article would be useless without a DEMO: WATCH ME

s2c is Open Source and free !

Source Code : https://github.com/sanix-darker/s2c
Pypi-Package : https://pypi.org/project/s2c/

PS : Any comments or constructive advice are welcome !

Top comments (7)

Collapse
 
ninofiliu profile image
Nino Filiu • Edited

Amazing!

I think it's visually interesting how people appear as ascii, that could defo be used in a bad hacker movie from the 90s...

However I think it could be a nice feature to have this:

  1. Sender pose and expression is captured (no need for AI, a simple Viola-Jones-based library could would work)
  2. Pose and expression are transmitted over the network (a lot less bandwith is required! 🎉)
  3. Pose and expression are rendered as little ascii humans like
 _____
/ o < \
|  U  |
\_____/
   |
+--+----+
Enter fullscreen mode Exit fullscreen mode
Collapse
 
cyril_ogoh profile image
ogoh cyril

Just wow
Imagine telling someone "let's video chat through our terminal"

Looking likes something from a SCI-Fi movies

Enjoyed reading

Collapse
 
sanixdarker profile image
darker

Thank you !!!

Collapse
 
krrishdhaneja profile image
Krrish Dhaneja

Awesome 👍 it is!!

Collapse
 
sanixdarker profile image
darker

Thank you !!!

Collapse
 
samueldsr99 profile image
Samuel David Suarez Rodriguez

Mind blowing idea!!! Congrats

Collapse
 
sanixdarker profile image
darker

Thank you !!!