Convert MP4 File To Black And White Using GStreamer

Introduction

Hello! 😎
In this tutorial I will show you how to use GStreamer and C++ to play an MP4 video file in Black and White.

Requirements

• GStreamer libraries installed
• CMake installed
• Basic C++ and GStreamer knowledge will help

Creating The Build File

First we will get the build file out of the way. Create a new file called "CMakeLists.txt" and populate it with the following:

cmake_minimum_required(VERSION 3.10)
project(BlackWhiteConverter)

find_package(PkgConfig)
pkg_check_modules(GSTREAMER REQUIRED gstreamer-1.0)

include_directories(${GSTREAMER_INCLUDE_DIRS})

add_executable(BlackWhiteConverter main.cpp)

target_link_libraries(BlackWhiteConverter ${GSTREAMER_LIBRARIES})

target_compile_options(BlackWhiteConverter PUBLIC ${GSTREAMER_CFLAGS_OTHER})

Nothing too complicated. The above checks to see if the GStreamer libraries are installed and then links them to compile the executable.

Now that the easy part is out of the way we can finally start coding in C++.😀

Creating The Application

Next we can start coding the C++ part. Create a new file called "main.cpp", first we need to import the headers like so:

#include <gst/gst.h>
#include <glib.h>

Here we only need two headers. One for GStreamer which is the main framework, the other is glib which provides auxillary support functions.

Next we need to define a callback function that will be used later on in the code:

static void on_pad_added(GstElement *element, GstPad *pad, gpointer data) {
    GstPad *sinkpad;
    GstElement *decoder = (GstElement *) data;

    sinkpad = gst_element_get_static_pad(decoder, "sink");
    gst_pad_link(pad, sinkpad);
    gst_object_unref(sinkpad);
}

This function is called "on_pad_added". Its a callback function that we will use later in the program to dynamically link certain elements of the GStreamer pipeline, which will be shown shortly.

Next we need to define the main function like so:

int main(int argc, char*argv[])
{

}

The above is pretty much the standerd for C++. Next we will accept a command line argument for the MP4 file to play. To do this we check if the user has provided the argument for the video file to play. If not we provide a warning and exit the program:

if (argc != 2) {
    g_printerr("Usage: %s <MP4 File>\n", argv[0]);
    return -1;
}

Now that we know the user has provided some form of argument, we can now define the GStreamer Elements that will be used:

GstElement *pipeline, *source, *demuxer, *decoder, *conv, *filter, *sink;
GstBus *bus;
GstMessage *msg;
GMainLoop *loop;

After that we can initialize GStreamer with the following one line:

gst_init(&argc, &argv);

Once GStreamer is initialized we can now start creating the elements that will be used in the application:

pipeline = gst_pipeline_new("video-black-white");
source = gst_element_factory_make("filesrc", "source");
demuxer = gst_element_factory_make("qtdemux", "demuxer");
decoder = gst_element_factory_make("avdec_h264", "decoder");
conv = gst_element_factory_make("videoconvert", "converter");
filter = gst_element_factory_make("videobalance", "filter");
sink = gst_element_factory_make("autovideosink", "sink");

The above creates the GStreamer elements that will be used in the pipeline to show a video file in black and white.

Its also good practice to check if the elements were created correctly, this can be done via the following check:

if (!pipeline || !source || !demuxer || !decoder || !conv || !filter || !sink) {
    g_printerr("Not all elements could be created.\n");
    return -1;
}

If any of the elements were not created correctly the program will fail here.

Next we need to set the properties of the elements to specify the video file and apply a black and white filter to the video. This is done like so:

const char *video_file_path = argv[1];
g_object_set(G_OBJECT(source), "location", video_file_path, NULL);
g_object_set(G_OBJECT(filter), "saturation", 0.0, NULL);

Next we will build the pipeline with the following code:

gst_bin_add_many(GST_BIN(pipeline), source, demuxer, decoder, conv, filter, sink, NULL);
gst_element_link(source, demuxer);
g_signal_connect(demuxer, "pad-added", G_CALLBACK(on_pad_added), decoder);
gst_element_link_many(decoder, conv, filter, sink, NULL);

The above adds all the elements to the pipeline and links them. The demuxer requires special handling; we use the "on_pad_added" callback fro dynamic linking since its output pads are created dynamically based on the input stream.

Next we can finally set the state of the pipeline to the playing state:

gst_element_set_state(pipeline, GST_STATE_PLAYING);

Once the video starts playing we then need to set up a message bus to wait for an "End Of Stream" event or an error message:

bus = gst_element_get_bus(pipeline);
msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GstMessageType(GST_MESSAGE_ERROR | GST_MESSAGE_EOS));

After the stream is finished due to the user closing the stream or the stream ending etc. We need to start the cleanup process:

if (msg != NULL) {
    gst_message_unref(msg);
}
gst_object_unref(bus);
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);

The above cleans up, unreferences any messages, the bus and sets the pipeline state to NULL, freeing up any used resources.

Great! Now that we have the code down we can now compile. 😆

Compiling The Program

To compile the program we will use cmake. Create a build folder in the current working directory via the following command:

mkdir build

Next run the following commands to compile the program:

cd build
cmake ..
make

The program should compile without any issues, giving you a new executable file in the build directory.

To run the program you just do the following command:

./BlackWhiteConverter [MP4 File]

The MP4 file should be displayed but in black and white. 😯

Conclusion

Here I have shown you how to create a C++ program using GStreamer for video processing. Hopefully this has helped you understand GStreamer a bit more, I certainly enjoyed making this tutorial. 🤓

As always you can find the sample code used for this tutorial on my Github:
https://github.com/ethand91/black-white-filter

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the following course