Idempotence in microservices using .NET, C#, and Devprime.

The development of microservices brings a new perspective to software architecture. Facing challenges is common, and this includes familiar aspects of traditional applications, such as idempotence. This terminology has its roots in mathematics and concerns the concept that identical operations executed repeatedly should not alter the outcome.

In a practical scenario, consider repeatedly pressing the "buy" button on an e-commerce shopping cart, resulting in multiple purchase orders. This issue is quite common, whether in traditional applications or in distributed systems like microservices. Besides being concerned about redundant processing in the API, it is crucial to implement the same strategy for received events.

The Devprime platform has revolutionized modern application development by offering a comprehensive software architecture project that utilizes hexagonal architecture and event-driven architecture for microservices. This enables the development of the first microservice in just 30 minutes, saving around 70% of backend software implementation costs. Among the various strategies offered, a crucial feature is the automatic idempotence approach. This strategy can be applied both to received posts in the API and events received through streams like Kafka, RabbitMQ, and others. To quickly understand idempotence behavior, let's observe the following log from a microservice implemented with Devprime.

This log shows the moment when idempotence is automatically initiated and later finalized. It represents the first POST in the microservice API.

The next step involves replicating the same POST in the API to observe the idempotence feature's internal operation. This will allow identification of duplicate processing, as illustrated in the image, and the system will automatically reject this duplication without going through the business rules executed in the previous example. It's important to note that this implementation is performed automatically and is provided by the Devprime platform. This approach contributes to reducing errors and processing costs.

In the first article of this series, titled "Accelerating Microservices Development using .NET/C# and Devprime," we demonstrated the creation of the first microservice. In this example, we will use the same scenario, making modifications to represent the idempotence context, automatically safeguarding an API against duplicate processing.

Installation

• .NET 8.0 or higher
• Visual Studio Code
• Docker
• Devprime CLI.

Preparing the environment
To set up the initial environment using Docker, we'll use MongoDB as the database and RabbitMQ as a queue, both in a local Docker environment. Additionally, Redis will be employed as a secondary database for idempotence functionality. Ensure that you have Docker installed and running, with MongoDB, RabbitMQ, and Redis containers active. For detailed guidance on creating the "devprime" exchange and the "orderevents" and "paymentevents" queues in RabbitMQ and performing the Bind with the exchange, I suggest referring to the Devprime documentation. The applications will use the default credentials provided in the Devprime documentation.

Obtaining and configuring the example application
In this article, we will modify an example microservice based on the Devprime platform to have idempotence functionality.

1) Clone the repository from GitHub:

   git clone [https://github.com/devprime/devprime-microservices-order-payment](https://github.com/devprime/devprime-microservices-order-payment)

2) Navigate into the cloned folder and update the Stack using the following Devprime CLI command:

   dp Stack

3) Configure the MongoDB, RabbitMQ, and Redis credentials:
a) Navigate to the ms-order folder and open the appsettings.json file in Visual Studio Code:

  ```
  code src/app/appsettings.json
  ```

b) Locate and update the RabbitMQ configuration in Stream.

   "DevPrime_Stream": [
      {
        "Alias": "Stream1",
        "Enable": "true",
        "Default": "true",
        "StreamType": "RabbitMQ",
        "HostName": "Localhost",
        "User": "guest",
        "Password": "guest",
        "Port": "5672",
        "Exchange": "devprime",
        "ExchangeType": "direct",
        "Retry": "3",
        "Fallback": "State1",
        "Threads": "30",
        "Buffer": "5",
        "Subscribe": []
      }
   ]

c) Locate and update the MongoDB (State1) and Redis (State2) configurations under State.

   "DevPrime_State": [
      {
        "enable": "true",
        "alias": "State1",
        "type": "db",
        "dbtype": "mongodb",
        "connection": "mongodb://mongoadmin:LltF8Nx*yo@localhost:27017",
        "timeout": "5",
        "retry": "2",
        "dbname": "ms-order",
        "isssl": "true",
        "numberofattempts": "4",
        "durationofbreak": "45"
      },
      {
        "enable": "true",
        "alias": "State2",
        "dbtype": "redis",
        "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
        "timeout": "5",
        "retry": "2",
        "durationofbreak": "45"
      }
   ]

Once you have reviewed the basic settings, execute the "Order" microservice and perform some API posts to observe if it's functioning normally and recording duplicate posts in the database.

To execute the microservice:

.\run.ps1 or ./run.sh (for Linux, macOS)

After conducting the initial tests mentioned above, stop the application to proceed to the next steps.

Enabling idempotence in the microservice API
The configuration of idempotence functionality is done in the "appsettings" file, under the "DevPrime_App" section, within the "Idempotency" item. To enable idempotence, you should modify the "Enabled" option to "true". The next parameter, "Alias," defines the storage location for persistence, set as "State2". The "Duration" parameter controls the duration period of idempotence. The "Flow" parameter determines the strategy to be adopted, for example, "backend". The "Scope" parameter establishes the scope of operation, which can be "all", "web", or "stream". Lastly, the "Action" parameter configures the mode of operation, which can be automatic or manual.

To edit, open the appsettings file in Visual Studio Code:

code src/app/appsettings.json

"Idempotency": {
   "Enable": "true",
   "Alias": "State2",
   "Duration": "86400",
   "Flow": "backend",
   "key": "idempotency-key",
   "

Scope": "all",
   "Action": "auto"
}

Now it's time to perform the first test by running the microservice again:
a) Execute the microservice again and open a browser at https://localhost:5001.
b) Fill in the post data using Swagger:

{
  "customerName": "Ramon Durães",
  "customerTaxID": "ID887612091",
  "items": [
    {
      "description": "Iphone",
      "amount": 1,
      "sku": "IP15",
      "price": 1200
    }
  ],
  "total": 1200
}

Upon confirmation, you will have successfully made a post as demonstrated in the log. During the "Initialize" moment, the Devprime Stack checks for the previous existence of this idempotence object. Not finding it, it processes normally and then creates the object to identify a second duplicate processing.

The same result can be confirmed in Swagger.

When trying to perform a new POST repeating the previous payload, you'll receive an error, indicating that the processing was not possible, as shown by the 500 error code in the API.

A quick review of the microservice log demonstrates that the automatic idempotence functionality offered by the Devprime platform discarded this duplicate request without any additional action required. It's essential to note that the shorter log indicates that the business rule processing did not occur.

When performing a new POST but this time changing some values in the payload, it will be accepted successfully because the strategy acts only on duplicate events.

Enabling a key as an idempotence parameter in the microservice
Now that you've learned how idempotence works in practice, let's customize the flow to "frontend" to have a personalized key, which in this context, we'll define as "TransactionID" in the "Key" field, as shown in the configuration below.

"Idempotency": {
   "Enable": "true",
   "Alias": "State2",
   "Duration": "86400",
   "Flow": "Frontend",
   "key": "TransactionID",
   "Scope": "all",
   "Action": "auto"
}

To edit, open the appsettings file in Visual Studio Code:

code src/app/appsettings.json

After editing, run the microservice again and observe the configuration information present in the log, specifying that we'll use a custom key "TransactionID" that must be provided in the Header when consuming the API.

a) Run the microservice again and open a browser at https://localhost:5001
b) This time, perform a post using Postman.
c) Create a new post for the URL https://localhost:5001/v1/order, set as raw and JSON, and add the content below.

{
  "customerName": "Ramon",
  "customerTaxID": "999871903773",
  "items": [
    {
      "description": "string",
      "amount": 0,
      "sku": "string",
      "price": 0
    }
  ],
  "total": 0
}

In the image, you can see the content in the post on Postman.

When performing a new POST via Postman, you'll immediately receive an error indicating that including a "TransactionID" in the Header is now mandatory.

Now, return to Postman in the Headers section and add the "transactionid" key with a value to successfully perform a POST.

When attempting to perform a second identical POST in the API, you'll encounter an error as presented below in the log, indicating that idempotence rejected this duplicate processing. To perform it, you'll need to include a new value in the "TransactionID" field in the Header.

In this article, we've demonstrated how to utilize the idempotence feature in conjunction with the Devprime platform's Stack, which incorporates this functionality automatically, simplifying the developer's experience. In the presented example, we used the automatic approach with parameters received in the API and later with a custom key transmitted via the Header.

To learn more and create a free account:
Devprime

[],

Ramon Durães
CEO, Devprime

Image: Freepik