Azure Durable Functions

#azuredurablefunction #serverless #azurefunctions #statefulworkflow

Azure durable functions help to define stateful workflow in code for stateless functions.

Durable Functions in a nutshell:

An extension to Azure Functions
Stateful functions in a serverless environment
Define workflows in code

Key Benefits of Durable Functions

Some of the key benefits of Durable Functions are

Parallel execution of functions (Fan-out > Fan-in)
Central Error handling
Easy to understand the dependencies among functions via “Orchestrator Function”

Overview

Starter Function: Simple Azure Function that starts the Orchestration by calling the Orchestrator function. It uses an OrchestrationClient binding.

OrchestrationClient is responsible for starting/stopping orchestrators and monitoring their status.

Orchestrator Function: Defines a stateful workflow in code and invokes the activity functions. Sleeps during activity invocation and replays when wakes up. Which also provides separation of concerns for defining the workflow and performing actual activity.

The code in an orchestrator function MUST be deterministic because during the flow the code will be executed again and again till all activity functions finish. You declare a function as an orchestrator by using a OrchestrationTrigger

Orchestration Function limitations:

Be Deterministic: No NewGuid(), Random, DateTime.Now, Http calls etc..
Be non-blocking: No I/O operations (e-g: table storage logic etc..) , no Thread.sleep etc..
Never initiate any async operations without using its context

Activity Functions: Simple Azure Functions that performs a single step in a workflow and can receive or return data. An Activity function uses an ActivityTrigger so that can be invoked by the orchestrator

Sub Orchestration:

Durable Functions also supports Sub Orchestration. This feature enables us to create sub orchestrators which orchestrate several activities. An orchestrator function can call another orchestrator function using the CallSubOrchestratorAsync or the CallSubOrchestratorWithRetryAsync methods.

It could be handy for:

Complex orchestration
In cases where retry mechanism is needed.
In cases where delay is needed in the activity function. Delay in activity function will be considered as part of function execution time, thus its billed where as timmer delay in orchestrator will not be billed as the orchestrator will go to sleep till timer is triggered.

Patterns

Durable Functions internal mechanism

Durable functions make use of storage queues. **Message in the queue triggers the next function. State of orchestrations is saved in the **storage table, it make use of event sourcing to play the current events and to trigger the next event.

Event source events:

During work flow execution, events are stored for following activities

Orchestrator Started
Activity 1 Scheduled
Orchestrator Goes to Sleep (No explicit event is registered for it)
Orchestrator Started (Wakes up) > Activity 1 Completed
Activity 2 Scheduled ……
Orchestrator Completed

Storage tables:

DurableFunctionsHubInstances: Contains records for each orchestrator instance with its input , final output and the RuntimeStatus
**DurableFunctionsHubHistory: **Contains event source events for each orchestrator instance.

**Note: The tables & queues names mentioned above are default names for durable function; if storage is shared among different Function apps then it is advisable to specify custom hub name in host.json.

Function Version 2.0 host.json:

"extensions": {

     "durableTask": {

           "hubName": "PeppolSupportedDocumentUpdaterHub"

      }
}

Monitoring progress

Starter / OrchestrationClient with HttpTrigger can return a response with “CreateCheckStatusResponse” which contains the “statusQueryGetUri” that can be used to monitor the progress of the work flow.

statusQueryGetUri:

http://{BaseUri}/runtime/webhooks/durabletask/instances/{InstanceId}?taskHub=DurableFunctionsHub&connection=Storage&code={FunctionKey}

If there are sub tasks involved then the progress / output of sub tasks can be monitored by adding following additional query parameters to “statusQueryGetUri”:

  &showHistoryOutput=true&showHistory=true

Delay & Retry Mechanism

context.CreateTimer, can be used to add delay between individual task execution in the chain.

var activityTask = context.CallActivityAsync<ActivityLog>(nameof(FunctionName), inputValueToFunction);

//add 3-sec delay between execution of tasks
var timeToStart = await context.CurrentUtcDateTime.AddSeconds(3);

var delayedActivityTask =  context.CreateTimer(timeToStart, CancellationToken.None).ContinueWith(t => activityTask);

tasks.Add(delayedActivityTask);

await Task.WhenAll(tasks);

var result = tasks.Select(task => task.Result).ToList();

return result;

Activity function can be retried in Orchestrator , and can also specify the delay between each retry. Optionally can also specify the exception for which to retry the activity.