Research is clear that video calls make for a more efficient workforce, particularly those that work remotely. According to Gartner, by 2024, only 25% of meetings will take place in person, down from 60% today
However, using video calls today can be frustrating, with black frames, choppy audio and frequent network disconnects. More often than not, we end up spending precious time trying to switch from video to audio, switch from mobile to WiFi, or moving closer to the router.
To find out which meeting solution works best under poor network conditions, we performed benchmark tests, pitting the leaders of WebRTC API browser-based video calls against the Zoom desktop app, Gartner’s 2020 Magic Quadrant leader for Meeting Solutions.
Also read: The Most Comprehensive Guide On WebRTC
Three Factors Critical To Video Quality : WebRTC Vs Zoom
1. Packet loss
To understand the factors affecting the quality of a video call, we need to know that speed test is not the same as packet loss. In layman’s terms, packet loss represents gaps in data transmitted over the internet.
WebRTC video stream uses the User Datagram Protocol network stream. When packets go missing on the Internet, the most important video frame, the I-frame cannot be reconstructed by the receiver. Missing video frames result in frozen or jerky video call experience.
Unlike streaming a video from Netflix, video meetings are broadcast in real-time. When watching Netflix, missing packets are retransmitted until one receives a full buffer of error free video frames before streaming starts. This latency will make real-time video meetings impossible. In sum, video solutions will face the issue of packet loss and the ability to resume back to full video resolution quickly is critical.
2. Bandwidth
The second factor is the bandwidth. Upload bandwidth is required to publish your video stream and download bandwidth is needed to subscribe to another participant’s video stream. The more participants you have in a video conference, the greater the download bandwidth you need.
When upload bandwidth is insufficient, most video conferencing apps will mute the publishing video of the affected participants, leaving only audio capabilities, until upload bandwidth is fully restored.
3. CPU availability
WebRTC requires your computer’s CPU to transcode video. The higher the video’s resolution, the greater the CPU power required. The more participants you have in a video call, the more CPU power is needed to decode the videos for all participants.
Benchmark Network Congestion Test: WebRTC Vs Zoom
We conducted the basic 1-1 video call with leading WebRTC video chat API providers and then also with Zoom desktop App
The publishing bandwidth for one of the WebRTC participants is throttled at different bitrates ranging from 100kbps, 200kbps, 250kbps and 300kbps.
We then measured the time (T1) taken for the video to resume streaming normally, at more than 1 frame per second (>1fps). The bitrate and resolution may still be ramping up after this time until full adaptation. We also measured the time (T2) taken for full adaption or full resolution after bandwidth congestion is removed.
To measure the duration for which the video is muted or frozen, we used a camera video source with a real-time stop-watch counter accurate to one second. This methodology will clearly identify precisely when video stream froze (0 fps) and when it resumed to partial adaptation (greater than 1fps).
To measure when the video recovered to (full adaption) full resolution and full framerate we used chrome://WebRTC-internals for the browser base 1-1 video call and the Zoom app video statistics.
The raw results are tabularised for two different WebRTC CPaaS providers and the Zoom app below.
During the period when network bandwidth is throttled, a 0fps means that video freezing was evident during the test duration. A range 10fps-25fps means the framerate is inconsistent.
Observations : WebRTC Vs Zoom
- The Zoom App froze the publishing video completely when the network bandwidth dropped below 300kbps while WebRTC A continued to publish at partial video adaptation at 200kbps and 250kbps bandwidths.
- Both Zoom app and WebRTC froze the video when throttled below 100kbps. However, the initial recovery time by Zoom is shorter, taking less than 10 seconds compared, to WebRTC needing over 40 seconds. The recovery to full adaptation for Zoom is longer (needing 80 seconds), compared to the 41 seconds that WebRTC A needed.
- WebRTC delivers acceptable smooth video quality at 300kbps bandwidth and even below that, whereas the Zoom app struggles to deliver a video frame every few seconds at 300kbps and the video froze occasionally during the throttled period.
- The Zoom app excelled at providing acceptable video quality (>1fps) almost immediately after the bandwidth is unthrottled.
- All the tests conducted show a high-quality freeze frame when partial video adaption is not available. We did not observe any black frames.
In summary the WebRTC video experience above 100kbps is considerably better than the Zoom App. Zoom shows a better video recovery time for low bandwidth condition (<100kbps).
Beyond video quality, the strength of WebRTC is the lightweight SDK toolkits provided by numerous CPaaS providers to easily integrate existing application for video and voice calls.
Disclaimer : WebRTC Vs Zoom
This is not meant to be an exhaustive comparison of either Zoom’s media stack nor WebRTC. It is a simple test to evaluate their performance under poor network conditions. Poor network conditions for WebRTC may be due to UDP packet loss and low CPU availability.
We did not perform an empirical evaluation of the audio quality but audio was present for all providers during the period when the bandwidth congested.
To limit the upload bandwidth for 30 seconds, we used traffic control (tc) utility that is available from the operating system.
The tests are all conducted with the same two virtual machines running ubuntu 18.04 with chrome browsers version 78 and above for Zoom & WebRTC app version 3.5.352596.0119.
We have not taken into consideration the location of the WebRTC media servers in the above tests. For completeness and comparison between WebRTC providers, we measured the RTT of the media server to the local test machines as approximately 70ms and 270ms for WebRTC provider A and B respectively. This may account for the slightly poorer performance of WebRTC B compared to WebRTC A.
Source : (1) 2019 Magic Quadrant for Meeting Solutions by Gartner.
To try it out, simply sign up here for free
Top comments (0)