SObjectizer Tales – 3. Acquisition loop revisited

In the previous episode, we learned that callback-based APIs are easy to integrate into agents since these can handle other messages in the meantime. We also wondered how to communicate with image_producer which, instead, is busy in a tight loop. We came up with an approach that consists in spawning a child broker agent capable of receiving messages on its behalf, and turning them into function calls. This solution comes with sharing state that is a root cause of issues this kind of applications, thus we are interested in exploring other options.

As said, sometimes sharing state is unavoidable, in particular when actor-style needs to be glued with other paradigms. Or rather, in some scenarios, sharing state is easier and possibly more efficient.

But still we might have some room to maneuver and this post is just a discussion on how SObjectizer can help revisit this code below:

void so_evt_start() override
{
    cv::VideoCapture cap(0);
    if (!cap.isOpened())
    {
        throw std::runtime_error("Can't connect to the webcam");
    }
    cv::Mat image;
    while (!m_stop.stop_requested())
    {
        cap >> image;
        so_5::send<cv::Mat>(m_channel, std::move(image));       
    }
}

Disclaimer: this post does not encourage ditching the code above, instead it just discusses a variation. Also, it’s perfectly fine to mix SObjectizer with something else (e.g. plain threads) in the same codebase. You know, everything comes with trade-offs that should be acknowledged and, in case, accepted. We, programmers, express ourselves by making decisions that most of the time are not perfect because rarely one size fits all. However, another side of our expressiveness is the reaction to change, thus being aware of and studying patterns and trade-offs contribute to react in a more effective way.

Reworking a loop into message-passing

Maybe you have already realized what we are getting at. Conceptually, a loop is just a series of iterations. The problem with the code above is the lack of opportunities to do something else between two iterations. Serializing iterations into messages solves the problem.

In other words, image_producer sends a “starting” message to itself (remember that agents provide their own message box) to say “grab an image”. When this message is handle, the agent grabs a frame. Then, it sends the very same message to itself! And so on, until the stop token is triggered. To distinguish this new form of producing images, let’s call the agent image_producer_recursive:

class image_producer_recursive final : public so_5::agent_t
{
public:
    image_producer_recursive(so_5::agent_context_t ctx, so_5::mbox_t channel, std::stop_token st)
        : agent_t(std::move(ctx)), m_channel(std::move(channel)), m_stop(std::move(st)), m_capture(0, cv::CAP_DSHOW)
    {
    }

    void so_define_agent() override
    {
        so_subscribe_self().event([this](bool) {
            cv::Mat image;
            m_capture >> image;
            so_5::send<cv::Mat>(m_channel, std::move(image));
            if (!m_stop.stop_requested())
            {
                so_5::send<bool>(*this, true); // keep on grabbing
            }
        });
    }

    void so_evt_start() override
    {
        if (!m_capture.isOpened())
        {
            throw std::runtime_error("Can't connect to the webcam");
        }

        so_5::send<bool>(*this, true); // grab the first one!
    }

private:
    so_5::mbox_t m_channel;
    std::stop_token m_stop;
    cv::VideoCapture m_capture;
};

Some details:

• so_subscribe_self is a shorthand for so_subscribe(so_direct_mbox()), or the way an agent subscribes to its own message box. Agent’s mbox are MPSC (multiple producer/single consumer) and can’t be subscribed to by others than the owner;
• send(*this, ...) is just a shorthand for send(so_direct_mbox(), ...), as you can expect.

The idea of this solution was proposed by Yauheni Akhotnikau when I was taking the first steps with SObjectizer. As you can read, Yauheni has a point here: “agents work well when you have to deal with different incoming messages in a state”. But in trivial scenarios like an infinite loop, “agents only create additional boilerplate without any benefits”. Indeed, if the agent only grabs images until stopped, the new producer is a bit forced.

However, back to our original discussion, we want to handle other requests in between. Maybe accessing a function of the device (e.g. GetBatteryLevel), or the state of the agent (e.g. a frame counter). So in this more complex case, this image_producer_recursive can fit.

Well, you are thinking about the price of introducing message handling here. You are right, it comes with some overhead that might be worth measuring, depending on the case. However, consider that often calls like m_capture >> image (e.g. retrieving and decoding the next image) are those causing most of the waiting time between two images. It’s where the frame rate of the camera counts. For example, 100 frames per second (FPS) means the time for retrieving two subsequent frames can’t be faster than 10 milliseconds (m_capture >> image might take up to 10ms).

Whether we use this solution or not, we can take the opportunity to learn something new on SObjectizer.

First observation: we don’t need the stop token anymore! Actually, when the agent is deregistered, it will terminate automatically after dequeuing all the remaining messages:

class image_producer_recursive final : public so_5::agent_t
{
public:
    image_producer_recursive(so_5::agent_context_t ctx, so_5::mbox_t channel)
        : agent_t(std::move(ctx)), m_channel(std::move(channel)), m_capture(0, cv::CAP_DSHOW)
    {
    }

    void so_define_agent() override
    {
        so_subscribe_self().event([this](bool) {
            cv::Mat image;
            m_capture >> image;
            so_5::send<cv::Mat>(m_channel, std::move(image));
            so_5::send<bool>(*this, true);
        });
    }

    void so_evt_start() override
    {
        if (!m_capture.isOpened())
        {
            throw std::runtime_error("Can't connect to the webcam");
        }

        so_5::send<bool>(*this, true);
    }

private:
    so_5::mbox_t m_channel;
    cv::VideoCapture m_capture;
};

Second observation: what’s the meaning of that bool message? Does it make sense? No, it’s just the first type that crossed my mind. Alternatives?

After all, it’s a message without state. It’s just a signal. Indeed, SObjectizer provides a specific way to define signals: classes inheriting from signal_t. Signals can’t be created nor assigned:

• we send a signal by calling send<signal_type>(destination);
• we receive a signal only into mhood_t.

We define a signal type inside the class (you can put it anywhere you like):

struct grab_image final : so_5::signal_t{};

Then we apply what we have just learned:

class image_producer_recursive final : public so_5::agent_t
{
    struct grab_image final : so_5::signal_t{};
public:
    image_producer_recursive(so_5::agent_context_t ctx, so_5::mbox_t channel)
        : agent_t(std::move(ctx)), m_channel(std::move(channel)), m_capture(0, cv::CAP_DSHOW)
    {
    }

    void so_define_agent() override
    {
        so_subscribe_self().event([this](so_5::mhood_t<grab_image>) {
            cv::Mat image;
            m_capture >> image;
            so_5::send<cv::Mat>(m_channel, std::move(image));
            so_5::send<grab_image>(*this);
        });
    }

    void so_evt_start() override
    {
        if (!m_capture.isOpened())
        {
            throw std::runtime_error("Can't connect to the webcam");
        }

        so_5::send<grab_image>(*this);
    }

private:
    so_5::mbox_t m_channel;
    cv::VideoCapture m_capture;
};

The advantage of this approach is evident: image_producer_recursive can handle other messages by subscribing to self or other message boxes. We get some interesting opportunities that will be discussed in the next post. On the other hand, it’s clearly something more convoluted than a simple loop and can introduce some overhead – when in doubt, just measure.

Bear in mind that another intermediate approach is possible. It consists in periodically extracting a message from a message queue or whatever. If you are familiar with GUI message loops (e.g. MFC), you should get an idea. This was proposed by Nicolai Grodzitski but it’s not discussed here since it uses a couple of things that will be introduced in the future.

Takeaway

In this episode we have learned:

• a loop can be reworked into messages by turning each iteration into a signal that the agent send to itself;
• SObjectizer provides a simple way to model a signal: a class inheriting from signal_t;
• signal_t can’t be created nor assigned, nevertheless it can be sent from send and received into mhood_t;
• every agent provides its own message box that can be accessed by so_direct_mbox();
• to subscribe to its own message box, an agent can simply call so_subscribe_self();
• to send to its own message box, an agent can simply call send<Message>(*this,...);
• message-passing style and explicit infinite loop have both pros and cons that need consideration.

As usual, calico is updated and tagged.

What’s next?

Our colleague Lucas is interested in our software, however he needs a new feature: he wants to start and stop the acquisition on demand and possibly multiple times. He provides an example of usage:

• I run the program and it does nothing
• at some point, I decide to start the webcam when I press a button or shout “Alexa, start the webcam”. I expect frames start flowing at that moment;
• I make a funny face for some time, then I decide to stop somehow and the program should get back idle;
• maybe I want to do it again, and again, ad again…

This new requirement is quite clear. Lucas wants to send commands to control the acquisition.

There are many irons in the fire for the next post!

Thanks to Yauheni Akhotnikau for having reviewed this post.