Practical usages of Idempotency

Idempotency is a crucial concept in distributed systems and web applications, ensuring that performing the same operation multiple times produces the same result.
This is particularly important in scenarios where operations might be retried due to failures or where duplicate requests might occur. Here are some practical use cases for idempotency

Idempotency on Webhooks

You can't be sure that you will only receive a webhook once, most applications provide an at least once policy. This means they will deliver the webhook at least once but it does not mean it will be only once.
In this case, you can find something that uniquely identifies that webhook payload or create a hash from the webhook payload.
Using a state machine to avoid processing the same webhook twice and also using a unique database index to handle concurrency duplicated webhook requests.

Idempotency on HTTP Requests

Most HTTP Requests that create a new resource should provide an idempotency ID to avoid duplicating the resource to be created.
When doing an HTTP Request you can't be sure if a request was a success or a failure in some network failure scenarios. To make sure you create the resource you need, you need to call the HTTP request again.
If the API does not have an idempotency ID, you can wait some time to try to GET the resource before trying to call the POST request again. This is a workaround, but the best way to avoid this is for the API to provide the idempotency ID.

Idempotency in your codebase

For anything that you don't want to duplicate, you can have a pattern to always check if something exists before trying to create it. A simple implementation below:

const getOrCreateCharge = async (payload) => {
   const existingCharge = await Charge.find({identifier: payload.identifier);


   if (existingCharge) { return existingCharge }

   const charge = await new Charge(payload).save();

   return charge;
}

Idempotency in Queues

It is common to use Queues in distributed systems to load processing to asynchronous and also to communicate in a decoupled way among services.
When processing a queue event, something can be wrong and you would need to reprocess it. The only safe way to reprocess is if the event processor is idempotent.

Idempotency in Data Migrations

As your system evolves you need to write data migrations to migrate the old data patterns to new data patterns in your database.
Your migrations need to be idempotency, as it can break in the middle of the data migration. You want to be able to reprocess the migration without causing unexpected side effects.

Idempotency in Seed

A seed script creates some basic database data to make it easy for developers to get up and running in your system.
Making your seed idempotent can avoid some problems if you run the seed script twice.

How to implement Idempotency?

Idempotency Keys: Unique identifiers (e.g., UUIDs) associated with each request or operation.
Databases: Storing the state of operations (e.g., processed requests, completed transactions) to check against duplicates.
Hashing: Creating hashes of request payloads to detect and handle duplicates.

In Conclusion

At Woovi we use idempotency a lot. Every time we are consuming a new service we ask ourselves, is this idempotency? Is this code idempotency? How can we make this code more idempotent?

This concept is used a lot in fintech and payments, but I think it can be applied when building any reliable software.

Woovi is an innovative startup revolutionizing the payment landscape. With Woovi, shoppers can enjoy the freedom to pay however they prefer. Our cutting-edge platform provides instant payment solutions, empowering merchants to accept orders and enhance their customer experience seamlessly.

If you're interested in joining our team, we're hiring! Check out our job openings at Woovi Careers.