Kishan Nakrani for Video SDK

Posted on May 1 • Originally published at videosdk.live on Apr 30

How to Integrate Active Speaker Indication in iOS Video Call App

#ios #swift

📌 Introduction

Integrating Active Speaker Indication in your iOS Video Call App enhances user experience by highlighting the current speaker, making conversations more engaging and natural. With this feature, users can easily identify who is speaking, leading to smoother communication and better interaction.

Benefits of Integrate Active Speaker Indication in iOS Video Call App:

Enhanced Communication: Active Speaker Indication allows participants to easily identify who is speaking during video calls, facilitating smoother communication and reducing interruptions.
Improved Engagement: By highlighting the current speaker, this feature keeps participants more engaged in the conversation, leading to better interaction and collaboration.
Reduced Confusion: Active Speaker Indication reduces confusion by providing visual cues, eliminating the need for participants to guess who is talking and preventing talking over each other.

Use Case of Integrate Active Speaker Indication in iOS Video Call App:

Enhanced Collaboration: During brainstorming sessions, when one team member starts speaking, their video feed becomes highlighted, allowing others to know who is presenting their ideas.
Reduced Miscommunication: In discussions involving multiple participants, Active Speaker Indication helps avoid confusion by clearly indicating the current speaker, preventing overlapping conversations.
Professional Meetings: Whether it's client meetings or internal discussions, Active Speaker Indication adds a level of professionalism to the video calls, making the interactions more effective and impactful.

By following the guide provided by VideoSDK, you'll seamlessly implement Active Speaker Indication into your iOS app, ensuring that users can see who is actively speaking during video calls. This not only improves the overall usability of your app but also adds a professional touch to your video calling experience. With clear visual cues, participants can focus more on the conversation rather than guessing who is talking.

🚀 Getting Started with VideoSDK

VideoSDK enables the opportunity to integrate video & audio calling into Web, Android, and iOS applications with so many different frameworks. It is the best infrastructure solution that provides programmable SDKs and REST APIs to build scalable video conferencing applications. This guide will get you running with the VideoSDK video & audio calling in minutes.

Create a VideoSDK Account

Go to your VideoSDK dashboard and sign up if you don't have an account. This account gives you access to the required Video SDK token, which acts as an authentication key that allows your application to interact with VideoSDK functionality.

Generate your Auth Token

Visit your VideoSDK dashboard and navigate to the "API Key" section to generate your auth token. This token is crucial in authorizing your application to use VideoSDK features. For a more visual understanding of the account creation and token generation process, consider referring to the provided tutorial.

Prerequisites and Setup

iOS 11.0+
Xcode 12.0+
Swift 5.0+

This App will contain two screens:

Join Screen : This screen allows the user to either create a meeting or join the predefined meeting.

Meeting Screen : This screen basically contains local and remote participant views and some meeting controls such as Enable/Disable the mic & Camera and Leave meeting.

🛠️ Integrate VideoSDK

To install VideoSDK, you must initialize the pod on the project by running the following command:

pod init

It will create the podfile in your project folder, Open that file and add the dependency for the VideoSDK, like below:

pod 'VideoSDKRTC', :git => 'https://github.com/videosdk-live/videosdk-rtc-ios-sdk.git'

then run the below code to install the pod:

pod install

then declare the permissions in Info.plist :

<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>

Project Structure

iOSQuickStartDemo
   ├── Models
        ├── RoomStruct.swift
        └── MeetingData.swift
   ├── ViewControllers
        ├── StartMeetingViewController.swift
        └── MeetingViewController.swift
   ├── AppDelegate.swift // Default
   ├── SceneDelegate.swift // Default
   └── APIService
           └── APIService.swift
   ├── Main.storyboard // Default
   ├── LaunchScreen.storyboard // Default
   └── Info.plist // Default
 Pods
     └── Podfile

Create models

Create a swift file for MeetingData and RoomStruct class model for setting data in an object pattern.

import Foundation
struct MeetingData {
    let token: String
    let name: String
    let meetingId: String
    let micEnabled: Bool
    let cameraEnabled: Bool
}

MeetingData.swift

import Foundation
struct RoomsStruct: Codable {
    let createdAt, updatedAt, roomID: String?
    let links: Links?
    let id: String?
    enum CodingKeys: String, CodingKey {
        case createdAt, updatedAt
        case roomID = "roomId"
        case links, id
    }
}

// MARK: - Links
struct Links: Codable {
    let getRoom, getSession: String?
    enum CodingKeys: String, CodingKey {
        case getRoom = "get_room"
        case getSession = "get_session"
    }
}

RoomStruct.swift

🎥 Essential Steps for Building the Video Calling

This guide is designed to walk you through the process of integrating Active Speaker Indication with VideoSDK. We'll cover everything from setting up the SDK to incorporating the visual cues into your app's interface, ensuring a smooth and efficient implementation process.

Step 1: Get started with APIClient

Before jumping to anything else, we have to write an API to generate a unique meetingId. You will require an authentication token; you can generate it either using videosdk-server-api-example or from the VideoSDK Dashboard for developers.

import Foundation

let TOKEN_STRING: String = "<AUTH_TOKEN>"

class APIService {

  class func createMeeting(token: String, completion: @escaping (Result<String, Error>) -> Void) {

    let url = URL(string: "https://api.videosdk.live/v2/rooms")!

    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    request.addValue(TOKEN_STRING, forHTTPHeaderField: "authorization")

    URLSession.shared.dataTask(
      with: request,
      completionHandler: { (data: Data?, response: URLResponse?, error: Error?) in

        DispatchQueue.main.async {

          if let data = data, let utf8Text = String(data: data, encoding: .utf8) {
            do {
              let dataArray = try JSONDecoder().decode(RoomsStruct.self, from: data)

              completion(.success(dataArray.roomID ?? ""))
            } catch {
              print("Error while creating a meeting: \(error)")
              completion(.failure(error))
            }
          }
        }
      }
    ).resume()
  }
}

APIService.swift

Step 2: Implement Join Screen

The Join Screen will work as a medium to either schedule a new meeting or join an existing meeting.

import Foundation
import UIKit

class StartMeetingViewController: UIViewController, UITextFieldDelegate {

  private var serverToken = ""

  /// MARK: outlet for create meeting button
  @IBOutlet weak var btnCreateMeeting: UIButton!

  /// MARK: outlet for join meeting button
  @IBOutlet weak var btnJoinMeeting: UIButton!

  /// MARK: outlet for meetingId textfield
  @IBOutlet weak var txtMeetingId: UITextField!

  /// MARK: Initialize the private variable with TOKEN_STRING &
  /// setting the meeting id in the textfield
  override func viewDidLoad() {
    txtMeetingId.delegate = self
    serverToken = TOKEN_STRING
    txtMeetingId.text = "PROVIDE-STATIC-MEETING-ID"
  }

  /// MARK: method for joining meeting through seague named as "StartMeeting"
  /// after validating the serverToken in not empty
  func joinMeeting() {

    txtMeetingId.resignFirstResponder()

    if !serverToken.isEmpty {
      DispatchQueue.main.async {
        self.dismiss(animated: true) {
          self.performSegue(withIdentifier: "StartMeeting", sender: nil)
        }
      }
    } else {
      print("Please provide auth token to start the meeting.")
    }
  }

  /// MARK: outlet for create meeting button tap event
  @IBAction func btnCreateMeetingTapped(_ sender: Any) {
    print("show loader while meeting gets connected with server")
    joinRoom()
  }

  /// MARK: outlet for join meeting button tap event
  @IBAction func btnJoinMeetingTapped(_ sender: Any) {
    if (txtMeetingId.text ?? "").isEmpty {

      print("Please provide meeting id to start the meeting.")
      txtMeetingId.resignFirstResponder()
    } else {
      joinMeeting()
    }
  }

  // MARK: - method for creating room api call and getting meetingId for joining meeting

  func joinRoom() {

    APIService.createMeeting(token: self.serverToken) { result in
      if case .success(let meetingId) = result {
        DispatchQueue.main.async {
          self.txtMeetingId.text = meetingId
          self.joinMeeting()
        }
      }
    }
  }

  /// MARK: preparing to animate to meetingViewController screen
  override func prepare(for segue: UIStoryboardSegue, sender: Any?) {

    guard let navigation = segue.destination as? UINavigationController,

      let meetingViewController = navigation.topViewController as? MeetingViewController
    else {
      return
    }

    meetingViewController.meetingData = MeetingData(
      token: serverToken,
      name: txtMeetingId.text ?? "Guest",
      meetingId: txtMeetingId.text ?? "",
      micEnabled: true,
      cameraEnabled: true
    )
  }
}

StartMeetingViewController.swift

Step 3: Initialize and Join Meeting

Using the provided token and meetingId, we will configure and initialize the meeting in viewDidLoad().

Then, we'll add @IBOutlet for localParticipantVideoView and remoteParticipantVideoView, which can render local and remote participant media respectively.

class MeetingViewController: UIViewController {

import UIKit
import VideoSDKRTC
import WebRTC
import AVFoundation

class MeetingViewController: UIViewController {

// MARK: - Properties
// outlet for local participant container view
   @IBOutlet weak var localParticipantViewContainer: UIView!

// outlet for label for meeting Id
   @IBOutlet weak var lblMeetingId: UILabel!

// outlet for local participant video view
   @IBOutlet weak var localParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant video view
   @IBOutlet weak var remoteParticipantVideoView: RTCMTLVideoView!

// outlet for remote participant no media label
   @IBOutlet weak var lblRemoteParticipantNoMedia: UILabel!

// outlet for remote participant container view
   @IBOutlet weak var remoteParticipantViewContainer: UIView!

// outlet for local participant no media label
   @IBOutlet weak var lblLocalParticipantNoMedia: UILabel!

// Meeting data - required to start
   var meetingData: MeetingData!

// current meeting reference
   private var meeting: Meeting?

    // MARK: - video participants including self to show in UI
    private var participants: [Participant] = []

        // MARK: - Lifecycle Events

        override func viewDidLoad() {
        super.viewDidLoad()
        // configure the VideoSDK with token
        VideoSDK.config(token: meetingData.token)

        // init meeting
        initializeMeeting()

        // set meeting id in button text
        lblMeetingId.text = "Meeting Id: \(meetingData.meetingId)"
      }

      override func viewWillAppear(_ animated: Bool) {
          super.viewWillAppear(animated)
          navigationController?.navigationBar.isHidden = true
      }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        navigationController?.navigationBar.isHidden = false
        NotificationCenter.default.removeObserver(self)
    }

        // MARK: - Meeting

        private func initializeMeeting() {

            // Initialize the VideoSDK
            meeting = VideoSDK.initMeeting(
                meetingId: meetingData.meetingId,
                participantName: meetingData.name,
                micEnabled: meetingData.micEnabled,
                webcamEnabled: meetingData.cameraEnabled
            )

            // Adding the listener to meeting
            meeting?.addEventListener(self)

            // joining the meeting
            meeting?.join()
        }
}

MeetingViewController.swift

Step 4: Implement Controls

After initializing the meeting in the previous step, we will now add @IBOutlet for btnLeave, btnToggleVideo and btnToggleMic which can control the media in the meeting.

class MeetingViewController: UIViewController {

...

    // outlet for leave button
    @IBOutlet weak var btnLeave: UIButton!

    // outlet for toggle video button
    @IBOutlet weak var btnToggleVideo: UIButton!

    // outlet for toggle audio button
    @IBOutlet weak var btnToggleMic: UIButton!

    // bool for mic
    var micEnabled = true
    // bool for video
    var videoEnabled = true

    // outlet for leave button click event
    @IBAction func btnLeaveTapped(_ sender: Any) {
            DispatchQueue.main.async {
                self.meeting?.leave()
                self.dismiss(animated: true)
            }
        }

    // outlet for toggle mic button click event
    @IBAction func btnToggleMicTapped(_ sender: Any) {
        if micEnabled {
            micEnabled = !micEnabled // false
            self.meeting?.muteMic()
        } else {
            micEnabled = !micEnabled // true
            self.meeting?.unmuteMic()
        }
    }

    // outlet for toggle video button click event
    @IBAction func btnToggleVideoTapped(_ sender: Any) {
        if videoEnabled {
            videoEnabled = !videoEnabled // false
            self.meeting?.disableWebcam()
        } else {
            videoEnabled = !videoEnabled // true
            self.meeting?.enableWebcam()
        }
    }

...

}

MeetingViewController.swift

Step 5: Implementing `MeetingEventListener`

In this step, we'll create an extension for the MeetingViewController that implements the MeetingEventListener, which implements the onMeetingJoined, onMeetingLeft, onParticipantJoined, onParticipantLeft, onParticipantChanged, onSpeakerChanged, etc. methods.


class MeetingViewController: UIViewController {

...

extension MeetingViewController: MeetingEventListener {

        /// Meeting started
        func onMeetingJoined() {

            // handle local participant on start
            guard let localParticipant = self.meeting?.localParticipant else { return }
            // add to list
            participants.append(localParticipant)

            // add event listener
            localParticipant.addEventListener(self)

            localParticipant.setQuality(.high)

            if(localParticipant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// Meeting ended
        func onMeetingLeft() {
            // remove listeners
            meeting?.localParticipant.removeEventListener(self)
            meeting?.removeEventListener(self)
        }

        /// A new participant joined
        func onParticipantJoined(_ participant: Participant) {
            participants.append(participant)

            // add listener
            participant.addEventListener(self)

            participant.setQuality(.high)

            if(participant.isLocal){
                self.localParticipantViewContainer.isHidden = false
            } else {
                self.remoteParticipantViewContainer.isHidden = false
            }
        }

        /// A participant left from the meeting
        /// - Parameter participant: participant object
        func onParticipantLeft(_ participant: Participant) {
            participant.removeEventListener(self)
            guard let index = self.participants.firstIndex(where: { $0.id == participant.id }) else {
                return
            }
            // remove participant from list
            participants.remove(at: index)
            // hide from ui
            UIView.animate(withDuration: 0.5){
                if(!participant.isLocal){
                    self.remoteParticipantViewContainer.isHidden = true
                }
            }
        }

        /// Called when speaker is changed
        /// - Parameter participantId: participant id of the speaker, nil when no one is speaking.
        func onSpeakerChanged(participantId: String?) {

            // show indication for active speaker
            if let participant = participants.first(where: { $0.id == participantId }) {
                self.showActiveSpeakerIndicator(participant.isLocal ? localParticipantViewContainer : remoteParticipantViewContainer, true)
            }

            // hide indication for others participants
            let otherParticipants = participants.filter { $0.id != participantId }
            for participant in otherParticipants {
                if participants.count > 1 && participant.isLocal {
                    showActiveSpeakerIndicator(localParticipantViewContainer, false)
                } else {
                    showActiveSpeakerIndicator(remoteParticipantViewContainer, false)
                }
            }
        }

        func showActiveSpeakerIndicator(_ view: UIView, _ show: Bool) {
            view.layer.borderWidth = 4.0
            view.layer.borderColor = show ? UIColor.blue.cgColor : UIColor.clear.cgColor
        }

}

...

MeetingViewController.swift

Step 6: Implementing `ParticipantEventListener`

In this stage, we'll add an extension for the MeetingViewController that implements the ParticipantEventListener, which implements the onStreamEnabled and onStreamDisabled methods for the audio and video of MediaStreams enabled or disabled.

The function updateUI is frequently used to control or modify the user interface (enable/disable camera & mic) by the MediaStream state.

class MeetingViewController: UIViewController {

...

extension MeetingViewController: ParticipantEventListener {

/// Participant has enabled mic, video or screenshare
/// - Parameters:
/// - stream: enabled stream object
/// - participant: participant object
func onStreamEnabled(_ stream: MediaStream, forParticipant participant: Participant) {
    updateUI(participant: participant, forStream: stream, enabled: true)
 }

/// Participant has disabled mic, video or screenshare
/// - Parameters:
/// - stream: disabled stream object
/// - participant: participant object

func onStreamDisabled(_ stream: MediaStream, 
            forParticipant participant: Participant) {

  updateUI(participant: participant, forStream: stream, enabled: false)
 }

}

private extension MeetingViewController {

 func updateUI(participant: Participant, forStream stream: MediaStream, enabled: Bool) { // true
        switch stream.kind {
        case .state(value: .video):
            if let videotrack = stream.track as? RTCVideoTrack {
                if enabled {
                    DispatchQueue.main.async {
                        UIView.animate(withDuration: 0.5){

                            if(participant.isLocal) {

        self.localParticipantViewContainer.isHidden = false
    self.localParticipantVideoView.isHidden = false       
    self.localParticipantVideoView.videoContentMode = .scaleAspectFill self.localParticipantViewContainer.bringSubviewToFront(self.localParticipantVideoView)                                                           
    videotrack.add(self.localParticipantVideoView)
    self.lblLocalParticipantNoMedia.isHidden = true

} else {
        self.remoteParticipantViewContainer.isHidden = false
            self.remoteParticipantVideoView.isHidden = false
                                self.remoteParticipantVideoView.videoContentMode = .scaleAspectFill
                                self.remoteParticipantViewContainer.bringSubviewToFront(self.remoteParticipantVideoView)
                                                videotrack.add(self.remoteParticipantVideoView)
 self.lblRemoteParticipantNoMedia.isHidden = true
        }
     }
  }
} else {
         UIView.animate(withDuration: 0.5){
                if(participant.isLocal){

                    self.localParticipantViewContainer.isHidden = false
                    self.localParticipantVideoView.isHidden = true
                    self.lblLocalParticipantNoMedia.isHidden = false
                            videotrack.remove(self.localParticipantVideoView)
} else {
                   self.remoteParticipantViewContainer.isHidden = false
                   self.remoteParticipantVideoView.isHidden = true
                   self.lblRemoteParticipantNoMedia.isHidden = false
                            videotrack.remove(self.remoteParticipantVideoView)
      }
    }
  }
}

     case .state(value: .audio):
            if participant.isLocal {

               localParticipantViewContainer.layer.borderWidth = 4.0
               localParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            } else {
                remoteParticipantViewContainer.layer.borderWidth = 4.0
                remoteParticipantViewContainer.layer.borderColor = enabled ? UIColor.clear.cgColor : UIColor.red.cgColor
            }
        default:
            break
        }
    }
}

...

Known Issue

Please add the following line to the MeetingViewController.swift file's viewDidLoad method If you get your video out of the container view like the below image.

override func viewDidLoad() {

  localParticipantVideoView.frame = CGRect(x: 10, y: 0, 
            width: localParticipantViewContainer.frame.width, 
        height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: localParticipantViewContainer.frame.width, 
            height: localParticipantViewContainer.frame.height)

  localParticipantVideoView.clipsToBounds = true

  remoteParticipantVideoView.frame = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

  remoteParticipantVideoView.bounds = CGRect(x: 10, y: 0, 
        width: remoteParticipantViewContainer.frame.width, 
            height: remoteParticipantViewContainer.frame.height)

    remoteParticipantVideoView.clipsToBounds = true
}

MeetingViewController.swift

TIP:

Stuck anywhere? Check out this example code on GitHub.

Once you've successfully installed the videoSDK into your iOS project, you'll unlock a range of functionality to enhance your video call application. This feature uses VideoSDK's advanced audio processing capabilities to identify the participant with the strongest audio signal in real time. This translates to directing the active speaker during a call, allowing you to provide visual feedback to users.

📸 Integrate Active Speaker Indication

This feature can be especially useful in large meetings or webinars, where there may be many participants and it may be difficult to tell who is speaking. Integrating Active Speaker Indication can significantly improve the user experience in various ways. It facilitates smoother communication by clearly indicating the current speaker, thus reducing confusion, particularly in larger group calls, and creating a more engaging environment for all participants involved.

Whenever any participant speaks in the meeting, the onSpeakerChanged event will be triggered with the participant ID of the active speaker.

For example, Alice and Bob are in a meeting. Whenever one of them speaks, the onSpeakerChanged event will be triggered and the speaker will return the participantId.

extension MeetingViewController: MeetingEventListener {
/// Called when speaker is changed
/// - Parameter participantId: participant id of the speaker, nil when no one is speaking.
func onSpeakerChanged(participantId: String?) {

    // show indicator for active speaker
    if let participant = participants.first(where: { $0.id == participantId }),
        // show indication for active speaker
        // ex. show border color
        // cell.contentView.layer.borderColor = UIColor.blue.cgColor : UIColor.clear.cgColor
    }

    // hide indicator for others participants
    let otherParticipants = participants.filter { $0.id != participantId }
    for participant in otherParticipants {
        // ex. remove border color
        //cell.contentView.layer.borderColor = UIColor.clear.cgColor
    }
}

🔚 Conclusion

Active Speaker Indication is a valuable feature for enhancing the functionality and user experience of your iOS video call app. By providing visual cues to identify the current speaker, this feature improves communication, engagement, and overall efficiency during virtual meetings.

With Active Speaker Indication, users benefit from clearer understanding, reduced confusion, and a more professional video calling experience. Whether it's team collaborations, client meetings, or remote learning sessions, this feature ensures smoother interactions and more productive discussions.

Enhance your iOS video call app today with VideoSDK and offer an unparalleled video calling experience to your users. Remember, VideoSDK provides 10,000 free minutes to empower your video call app with advanced features without breaking any initial investment. Sign up with VideoSDK today and take the video app to the next level.

DEV Community

How to Integrate Active Speaker Indication in iOS Video Call App

📌 Introduction

🚀 Getting Started with VideoSDK

Create a VideoSDK Account

Generate your Auth Token

Prerequisites and Setup

🛠️ Integrate VideoSDK

Project Structure

Create models

🎥 Essential Steps for Building the Video Calling

Step 1: Get started with APIClient

Step 2: Implement Join Screen

Step 3: Initialize and Join Meeting

Step 4: Implement Controls

Step 5: Implementing `MeetingEventListener`

Step 6: Implementing `ParticipantEventListener`

Known Issue

📸 Integrate Active Speaker Indication

🔚 Conclusion

Top comments (0)

Read next

Integrating Live Activity and Dynamic Island in iOS: A Complete Guide

LeetCode in Swift - 9. Palindrome Number

How to use Swift package access modifier with Cocoapods

How to fix the error: DT_TOOLCHAIN_DIR cannot be used to evaluate LIBRARY_SEARCH_PATHS, use TOOLCHAIN_DIR instead

📌 Introduction

🚀 Getting Started with VideoSDK

Create a VideoSDK Account

Generate your Auth Token

Prerequisites and Setup

🛠️ Integrate VideoSDK​

Project Structure

Create models​

🎥 Essential Steps for Building the Video Calling

Step 1: Get started with APIClient​

Step 2: Implement Join Screen​

Step 3: Initialize and Join Meeting​

Step 4: Implement Controls​

Step 5: Implementing MeetingEventListener​

Step 6: Implementing ParticipantEventListener

Known Issue​

📸 Integrate Active Speaker Indication

🔚 Conclusion

Read next

Integrating Live Activity and Dynamic Island in iOS: A Complete Guide

LeetCode in Swift - 9. Palindrome Number

How to use Swift package access modifier with Cocoapods

How to fix the error: DT_TOOLCHAIN_DIR cannot be used to evaluate LIBRARY_SEARCH_PATHS, use TOOLCHAIN_DIR instead

🛠️ Integrate VideoSDK

Create models

Step 1: Get started with APIClient

Step 2: Implement Join Screen

Step 3: Initialize and Join Meeting

Step 4: Implement Controls

Step 5: Implementing `MeetingEventListener`

Step 6: Implementing `ParticipantEventListener`

Known Issue