DEV Community

Cover image for 10-Day .Net Aspire Challenge: Day 5 — Apache Kafka
Sukhpinder Singh
Sukhpinder Singh

Posted on • Originally published at singhsukhpinder.Medium

10-Day .Net Aspire Challenge: Day 5 — Apache Kafka

Step-by-step guide on how to use the .Net Aspire Kafka component in Visual Studio.

Introduction

.Net Aspire framework is used to develop cloud and production-ready distributed applications. It consists of components to handle cloud-native concerns such as Redis, Postgres etc.

Prerequisites

Objectives

Learn how to create a starter project using .Net Aspire with the Apache Kafka component.

Github Sample: The solution structure is divided into the following projects

  • DotnetAspireChallenge.ApiService

  • DotnetAspireChallenge.AppHost

  • DotnetAspireChallenge.ServiceDefaults

  • DotnetAspireChallenge.Web

Getting Started

Step 1: Install the following NuGet package

Install the following Nuget package into the subsequent project “DotnetAspireChallenge.AppHost

dotnet add package Aspire.Hosting.Kafka
Enter fullscreen mode Exit fullscreen mode

In the above project, register Kafka UI as shown below



    var messaging = builder.AddKafka("messaging")


```                           .WithKafkaUI();

Then finally add a reference to both the Producer and Consumer where the producer is DotnetAspireChallenge.ApiService and the consumer is DotnetAspireChallenge.Web project respectively.
```csharp


    var apiService = builder.AddProject<Projects.DotnetAspireChallenge_ApiService>("apiservice")
        .WithReference(messaging);



    builder.AddProject<Projects.DotnetAspireChallenge_Web>("webfrontend")
        .WithExternalHttpEndpoints()
        .WithReference(cache)
        .WithReference(apiService)
        .WithReference(messaging);


Enter fullscreen mode Exit fullscreen mode

Step 2: Add dependency of Kafka Producer

Add the dependency in the Program.cs file of the project “DotnetAspireChallenge.ApiService”



    builder.AddKafkaProducer<string, string>("messaging");


Enter fullscreen mode Exit fullscreen mode

and add a relevant minimal API endpoint using the following code.



    public static class AspireKafkaExtension
    {
        public static void MapAspireKafkaEndpoint(this WebApplication app)
        {
            app.MapGet("/send", async (IProducer<string, string> services, string key, string value) =>
            {
                try
                {
                    var message = new Message<string, string> { Key = key, Value = value };
                    DeliveryResult<string, string>? result = await services.ProduceAsync("messaging", message);
                    return result;
                }
                catch (Exception ex)
                {

                    throw;
                }

            });
        }
    }


Enter fullscreen mode Exit fullscreen mode

The endpoint takes two parameters namely key and value as route values, and produces the message on the docker-hosted Kafka server.

https://localhost:7313/send?key=key&value=1
Enter fullscreen mode Exit fullscreen mode

Step 3: Add dependency of Kafka Consumer

Now move “DotnetAspireChallenge.Web” project wherein register as a Kafka producer



    builder.AddKafkaConsumer<string, string>("messaging", options =>
    {
        options.Config.GroupId = "my-consumer-group";
        options.Config.AutoOffsetReset = AutoOffsetReset.Earliest;
        options.Config.EnableAutoCommit = false;
    });


Enter fullscreen mode Exit fullscreen mode

*Note: * It's mandatory to provide a default group ID.

Step 4: Create a Razor Page

Create a basic razor page named “KafkaConsumer.razor” to show the consumed message from the Kafka server.


<span class="n">@page</span> <span class="s">"/kafka"</span>
<span class="n">@attribute</span> <span class="p">[</span><span class="nf">StreamRendering</span><span class="p">(</span><span class="k">true</span><span class="p">)]</span>
<span class="n">@attribute</span> <span class="p">[</span><span class="nf">OutputCache</span><span class="p">(</span><span class="n">Duration</span> <span class="p">=</span> <span class="m">5</span><span class="p">)]</span>
<span class="n">@using</span> <span class="n">Confluent</span><span class="p">.</span><span class="n">Kafka</span>
<span class="p">&lt;</span><span class="n">h3</span><span class="p">&gt;</span><span class="n">KafkaConsumer</span><span class="p">&lt;/</span><span class="n">h3</span><span class="p">&gt;</span>

<span class="n">@inject</span> <span class="n">KafkaConsumeMessageClient</span> <span class="n">kafaConsumeMessageClient</span>
<span class="p">&lt;</span><span class="n">PageTitle</span><span class="p">&gt;</span><span class="n">Kafka</span> <span class="n">Consumed</span> <span class="n">Message</span><span class="p">&lt;/</span><span class="n">PageTitle</span><span class="p">&gt;</span>

<span class="p">&lt;</span><span class="n">h1</span><span class="p">&gt;</span><span class="n">Kafka</span><span class="p">&lt;/</span><span class="n">h1</span><span class="p">&gt;</span>

<span class="p">&lt;</span><span class="n">p</span><span class="p">&gt;</span><span class="n">This</span> <span class="n">component</span> <span class="n">demonstrates</span> <span class="n">showing</span> <span class="n">data</span> <span class="n">loaded</span> <span class="k">from</span> <span class="n">a</span> <span class="n">backend</span> <span class="n">API</span> <span class="n">service</span><span class="p">.&lt;/</span><span class="n">p</span><span class="p">&gt;</span>

<span class="nf">@if</span> <span class="p">(</span><span class="n">consumedMessage</span> <span class="p">==</span> <span class="k">null</span><span class="p">)</span>
<span class="p">{</span>
    <span class="p">&lt;</span><span class="n">p</span><span class="p">&gt;&lt;</span><span class="n">em</span><span class="p">&gt;</span><span class="n">Loading</span><span class="p">...&lt;/</span><span class="n">em</span><span class="p">&gt;&lt;/</span><span class="n">p</span><span class="p">&gt;</span>
<span class="p">}</span>
<span class="k">else</span>
<span class="p">{</span>
    <span class="p">&lt;</span><span class="n">table</span> <span class="k">class</span><span class="err">="</span><span class="nc">table</span><span class="s">"&gt;
Enter fullscreen mode Exit fullscreen mode

<thead>
<tr>
<th>Topic</th>
<th>Value</th>
</tr>
</thead>
<tbody>
<tr>
<td>@consumedMessage.Topic</td>
<td>@consumedMessage.Value</td>
</tr>

        <span class="p">&lt;/</span><span class="n">tbody</span><span class="p">&gt;</span>
    <span class="p">&lt;/</span><span class="n">table</span><span class="p">&gt;</span>
<span class="p">}</span>
<span class="n">@code</span> <span class="p">{</span>
    <span class="k">private</span> <span class="n">ConsumeResult</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">&gt;?</span> <span class="n">consumedMessage</span><span class="p">;</span>

    <span class="k">protected</span> <span class="k">override</span> <span class="k">async</span> <span class="n">Task</span> <span class="nf">OnInitializedAsync</span><span class="p">()</span> <span class="p">=&gt;</span> <span class="n">consumedMessage</span> <span class="p">=</span> <span class="n">kafaConsumeMessageClient</span><span class="p">.</span><span class="nf">GetKafkaMessage</span><span class="p">();</span>
<span class="p">}</span>
Enter fullscreen mode Exit fullscreen mode
Enter fullscreen mode Exit fullscreen mode




Step 5: Configure HttpCall to the ApiService



<span class="k">public</span> <span class="k">class</span> <span class="nc">KafkaConsumeMessageClient</span><span class="p">(</span><span class="n">HttpClient</span> <span class="n">httpClient</span><span class="p">,</span> <span class="n">IConsumer</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">&gt;</span> <span class="n">_consumer</span><span class="p">)</span>
<span class="p">{</span>

    <span class="k">public</span> <span class="n">ConsumeResult</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">&gt;?</span> <span class="nf">GetKafkaMessage</span><span class="p">(</span><span class="n">CancellationToken</span> <span class="n">cancellationToken</span> <span class="p">=</span> <span class="k">default</span><span class="p">)</span>
    <span class="p">{</span>
        <span class="n">ConsumeResult</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">,</span> <span class="kt">string</span><span class="p">&gt;?</span> <span class="n">deliveryResult</span> <span class="p">=</span> <span class="k">null</span><span class="p">;</span>
        <span class="n">_consumer</span><span class="p">.</span><span class="nf">Subscribe</span><span class="p">(</span><span class="s">"messaging"</span><span class="p">);</span>
        <span class="n">deliveryResult</span> <span class="p">=</span> <span class="n">_consumer</span><span class="p">.</span><span class="nf">Consume</span><span class="p">(</span><span class="n">TimeSpan</span><span class="p">.</span><span class="nf">FromSeconds</span><span class="p">(</span><span class="m">10</span><span class="p">));</span>

        <span class="k">return</span> <span class="n">deliveryResult</span><span class="p">;</span>
    <span class="p">}</span>
<span class="p">}</span>
Enter fullscreen mode Exit fullscreen mode
Enter fullscreen mode Exit fullscreen mode




Kafka Produce Demo

Kafka UI Demo

Kafka Consume Demo

Github Project

GitHub - ssukhpinder/DotnetAspireChallenge: 10 Day .Net Aspire Challenge

More Cheatsheets

Cheat Sheets — .Net

C# Programming🚀

Thank you for being a part of the C# community! Before you leave:

Follow us: Youtube | X | LinkedIn | Dev.to
Visit our other platforms: GitHub
More content at C# Programming

Top comments (0)