Dotnet AsyncApi specification for Avro Kafka channel

#dotnet #tutorial #kafka #eventdriven

Problem

How do we document Event-Driven Architecture (EDA) and provide discovery for events and transport?

Solution

Generate AsyncAPI specifications document - it is defacto standard for EDA as OpenAPI for HTTP API. This document may be used by a lot of tools to visualize and share specifications:

Implementation

Our ASP .Net Core application publishes events to Kafka using avro serialization protocol. There is a library that can help to build AsyncApi specification. A stable version v5.2.4(when the article was published) does not support Avro schema, but the new beta version (6.0.0-beta.97) - has implementation.

1. We will start with the high-level document

var document = new AsyncApiDocument()
{
    Info = new AsyncApiInfo
    {
        Title = "My AsyncAPI",
        Version = "0.0.1"
    }
};

2. Next, we will create components(Kafka messages with Avro schema). Also, there is an option to reference the existing Avro file schema.

document.Components = new AsyncApiComponents
{
    Messages = new Dictionary<string, AsyncApiMessage>
    {
        ["MyAvroMessage"] = new AsyncApiMessage
        {
            SchemaFormat = "application/vnd.apache.avro;version=1.9.0",
            Payload = new AsyncApiAvroSchemaPayload(new AvroRecord
            {
                Doc = "Doc for event",
                Name = "MyAvroMessage",
                Namespace = "Test.MyAvroMessage",
                Fields = [
                    new AvroField {
                        Doc = "Doc for field",
                        Name = "Id",
                        Type = AvroPrimitiveType.String,
                    }
                ]
            })
        }
    }
};

3. Reference messages to channels(Kafka topic)

document.Channels["my.kafka.topic"] = new AsyncApiChannel
{
    Subscribe = new AsyncApiOperation
    {
        Message = [
           new AsyncApiMessage
           {
               Reference = new AsyncApiReference
               {
                   Id = "MyAvroMessage",
                   Type = ReferenceType.Message,
               }
           }
        ]
    }
};

4. Serialize document to AsyncApi specification

var serializedDoc = document.Serialize(
    AsyncApiVersion.AsyncApi2_0,
    AsyncApiFormat.Yaml
);

5. In the end, we will have the next specification

asyncapi: 2.6.0
info:
  title: My AsyncAPI
  version: 0.0.1
channels:
  my.kafka.topic:
    subscribe:
      message:
        $ref: '#/components/messages/MyAvroMessage'
components:
  messages:
    MyAvroMessage:
      payload:
        type: record
        name: MyAvroMessage
        namespace: Test.MyAvroMessage
        doc: Doc for event
        fields:
          - name: Id
            type: string
            doc: Doc for field
      schemaFormat: application/vnd.apache.avro;version=1.9.0

6. Finally, visualization

Conclusion

With AsyncApi spec document we can share it via endpoint as OpenAPI document and use a tool to visualize
We can build an Avro scheme from scratch or reference schema file
We can provide information for Kafka server and bindings, but it is not our case
We can even generate clients for some programming languages, but that is also not our case
Currently AsyncAPI.NET don't support 3.0.0 specification