DEV Community

Maxime Guilbert
Maxime Guilbert

Posted on • Updated on

Kafka Architecture : Synchronous to Asynchronous [2]

Today we will see the second of 3 cases to make this communication between the synchronous application more efficient depending your case.

This solution is more data secured and you will understand why.


2nd solution - Everybody reads solution

On paper, this solution is quite easy.

Everybody reads solution - schema

Like in the previous example, you have 2 instances of the API X which send requests on TOPIC REQUEST and is waiting a response on TOPIC RESPONSE.

But this time each instance of the API have it's own group, so each instance will read each message.

So if the instance 1 of the API receive an http request, it will be able to answer!


How to setup this?

In your ACL, define "PREFIXED" for the consumer group of your api.

Then, to have to be sure that each instance of your API have a unique consumer group prefixed by what your declare in your ACLs.

Post about random values

As explain in this post, in Java it's quite easy.

consumer_group: consumerGroupApiX${random.uuid}
Enter fullscreen mode Exit fullscreen mode

Like this, it doesn't matter that you have 2 or 100 instances of your service, each and every pod will have an unique consumer group.


Pros & Cons

Pros Cons
More data-secured than the first solution We still have an issue when a service is restarted
Each API instance consume every message

It's more secured for the datas, but if the process is long or if you have a very high volume of requests, you will have the same issue than the first solution.

The advantage for this solution than the first one is that you won't have any issue when you scale up. You can have some issues when you are scaling down, but there are some solution to help to limit this.


In the last part we will see the best solution to avoid this kind of issues!

I hope it will help you! 🍺


You want to support me?

Buy Me A Coffee

Oldest comments (0)