How to implement sound waves in Android by using ZEGOCLOUD SDK

Introduction

In the karaoke scene, it is often necessary to pull multiple streams and display the user who is speaking. We need to identify whether the user is speaking, the volume of the speech (sound wave), and display it on the UI.

The SDK provides the following two methods to achieve this function：

Sound level refers to the volume of a stream. The ZEGO Express SDK provides the ability to capture the sound level of a stream in real time and delivers the captured sound level data to the app client through related callbacks. A typical use case of this feature is that you can visualize the sound level data on your app UI to indicate the current active speaker and their volume. The following picture shows an example of such use cases.

Audio spectrum refers to a series of values, each of which represents an audio signal's energy level at a point in a frequency range. The ZEGO Express SDK provides the ability to capture the audio spectrum data of a stream in real time and delivers the captured audio spectrum data to the app client through related callbacks. A typical use case of this feature is that you can use it to create and display an audio spectrum visualizer on your app UI, especially for those online karaoke apps. The following picture shows an example of audio spectrum visualizers.

Prerequisites

Before you begin to use sound level and audio spectrum in your project, make sure you complete the following steps:

• Create a project in ZEGOCLOUD Admin Console, and get the AppID of your project.
• The ZEGO Express SDK has been integrated into the project. For details, see Integration and Implementation.

Enable the sound level callbacks

To enable sound level callbacks, call the startSoundLevelMonitor method.

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startSoundLevelMonitor();

After the above step is completed:

• When you start previewing the local video by calling the startPreview method or publishing a local stream by calling the startPublishingStream method, the SDK will trigger the onCapturedSoundLevelUpdate callback at an interval of 100ms to deliver the sound level data of the locally captured audio.

/**
    * The callback to deliver the sound level data of locally captured audio.
    *
    * The callback sends out a notification at an interval of 100 ms.
    * @param soundLevel: The sound level value of the locally captured audio, which is in the range [0.0, 100.0].
    */
public void onCapturedSoundLevelUpdate(double soundLevel){

}

• When you start playing remote streams by calling the startPlayingStream method, the SDK will trigger the onRemoteSoundLevelUpdate callback at an interval of 100ms to deliver the sound level data of the remote streams.

/**
    * The callback to deliver the sound level data of remote streams
    *
    * The callback sends out a notification at an interval of 100 ms.
    * @param soundLevels: The sound level data (key-value pairs) of the remote streams, of which the key is a stream ID, and the value is the sound level value of the corresponding stream. Each sound level value is in the range [0.0, 100.0].
    */
public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels){

}

Enable the audio spectrum callbacks

To enable audio spectrum callbacks, call the startAudioSpectrumMonitor method.

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startAudioSpectrumMonitor();

After the above step is completed:

• When you start previewing the local video by calling the startPreview method or publishing a local stream by calling the startPublishingStream method, the SDK will trigger theonCapturedAudioSpectrumUpdate` callback at an interval of 100ms to deliver the audio spectrum data of locally captured audio.

`

/**
    * The callback to deliver the audio spectrum data of locally captured audio
    *
    * The callback sends out a notification at an interval of 100 ms.
    * @param audioSpectrum: An array of audio spectrum values of the locally captured audio. Each audio spectrum value is in the range [0, 2^30].
    */
public void onCapturedAudioSpectrumUpdate(float[] audioSpectrum){

}

`

• When you start playing remote streams by calling the startPlayingStream method, the SDK will trigger the onRemoteAudioSpectrumUpdate callback at an interval of 100ms to deliver audio spectrum data of the remote streams.

`

/**
    * The callback to deliver the audio spectrum data of remote streams
    *
    * The callback sends out a notification at an interval of 100 ms.
    * @param audioSpectrums: The audio spectrum data (key-value pairs) of remote streams, of which the key is a stream ID, and the value is an array of audio spectrum values of the corresponding stream. Each audio spectrum value is in the range [0, 2^30].
    */
public void onRemoteAudioSpectrumUpdate(HashMap<String, float[]> audioSpectrums){

}

`

Obtain data from the callbacks

The SDK delivers the sound level data and audio spectrum data of the remote streams as key-value pairs in a HashMap. In each key-value pair, the key is the stream ID of a remote stream published by another user in the same room, and the value is the sound level value or an array of audio spectrum values that stream.

You can first obtain and save the list of remote streams published by the other users in the current room through the onRoomStreamUpdate callback, and then use each stream ID in the list as an index to get the sound level value or the audio spectrum data of that stream from the HashMap.

The following examples show how to obtain the sound level and audio spectrum data from the callbacks. For how to render the data onto the UI of the app, refer to the sample code mentioned in Section 2 above.

`

class MyEventHandler extends IZegoEventHandler
{
    @override
    public void onCapturedSoundLevelUpdate(double soundLevel) {
        // Obtain the sound level data of locally captured audio and render it to specific UI controls.
    }
    @override
    public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels) {
        // Obtain the sound level data of remote streams and render it to specific UI controls.
    }
    @override
    public void onCapturedAudioSpectrumUpdate(double[] audioSpectrum) {
        // Obtain the audio spectrum data of locally captured audio and render it to specific UI controls.
    }
    @override
    public void onRemoteAudioSpectrumUpdate(HashMap<String, double[]> audioSpectrums) {
        // Obtain the audio spectrum data (key-value pairs) of the remote streams and render it to specific UI controls.
    }
}

`

Disable the sound level callbacks

To disable the sound level callbacks, call the stopSoundLevelMonitor method.

`

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopSoundLevelMonitor();

`

After the above step is completed, the SDK will stop triggering the callbacks onCapturedSoundLevelUpdate and onRemoteSoundLevelUpdate.

Disable the audio spectrum callbacks

To disable audio spectrum callbacks, call the stopAudioSpectrumMonitor method.

`

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopAudioSpectrumMonitor();

`

After the above step is completed, the SDK will stop triggering the callbacks onCapturedAudioSpectrumUpdate and onRemoteAudioSpectrumUpdate.