Day 21 of the #25DaysOfServerless Challenge

This article is part of #25DaysOfServerless. New challenges will be published every day from Microsoft Cloud Advocates throughout the month of December. Find out more about how Microsoft Azure enables your Serverless functions.

Have an idea or a solution? Share your thoughts on Twitter!

Last Minute Gift Registry

Oh no! The 🧝‍♀️🧝‍♂️ elves are upset. It's almost time to deliver gifts, but a glitch in the North Pole 💈 data center destroyed Santa's master registry list (should have backed it up to the cloud!). As a last-minute effort to modernize North Pole operations, the lead developer elf decided to create a set of REST APIs to track gift registries. To create a sense of urgency and ensure the orders can be fulfilled, registries are only available for 5 minutes before they are automatically closed and sent for processing.

For this challenge, create a set of WebHooks for managing registries. The WebHooks should support the following operations:

• Open to create a new registry and return a unique identifier for that registry
• Add to add an entry (gift item) to the registry based on id (any text)
• Finish to close a registry based on id
• Stats to get the following stats: total registries (open and closed) and total items in the registry list (an aggregate across all registries)

If Finish is not called within 5 minutes after the registry is opened, it should automatically close regardless of whether any items have been added.

Bonus: Although not required for this challenge, it would be great to have a Peek WebHook to look at a registry and show if it is open or closed and what the contents are.

A Solution: Durable Functions

A gift registry is really a long-running workflow with some state associated to it. The workflow looks like this:

In addition to the workflow, there are two stateful items to track. The first is the overall statistics to keep track of registries and items, and the second is the individual registries with their associated list of items. A perfect solution is the Azure Functions extension known as "Durable Functions."

Prerequisites

To get started, be sure you have these (free) tools installed. The links provide the necessary instructions to install.

• A code editor (I prefer Visual Studio Code)
• Azure Functions Core Tools
• .NET Core 2.2 or later
• Azure Storage Emulator

These tools enable you to build, run, and test the solution locally. No Azure account needed! An Azure account will come in handy later if you want to publish your solution, you can pick up a free Azure account.

The full solution is available at:

Scaffold the Serverless App

A function app is a scalable host for serverless endpoints. Your "functions" are what do the work, based on some event (a trigger). The app is a host for functions and provides the necessary runtime and environment to manage triggers and execute code. You can have multiple serverless functions hosted by the same app. In this example, we'll host HTTP-triggered functions to implement our WebHooks, orchestration-triggered functions for our workflow and entity-triggered functions to manage state. First, we need to create the app. Here are three possible ways:

• Create a function app from the command line
• Create a function app from Visual Studio Code
• Create a function app from Visual Studio 2019

Whatever you choose, this solution used C# as the language, .NET for the runtime, and HTTP Trigger as the first function to scaffold. By default, a simple "echo" function is created that looks for name in either a POST body or a GET query string and returns a greeting.

Run it Locally

Now would be a good time to test that everything is working properly. Make sure your storage emulator is running. From Visual Studio Code and Visual Studio 2019, you should be able to press F5 and see your app running. You can also publish your project, navigate to the publish directory, and type func host start to launch the functions runtime.

The runtime will automatically find HTTP endpoints and write them to the console. The default port is 7071 and you should be able to see your function in action by accessing http://localhost:7071/api/HttpTrigger1?name=LeadElf. Replace the function name with whatever name you gave when generating the project.

Add State to your Serverless

Azure Functions follows a "have it your way" model. The bare functionality is available out of the box and you can opt-in to new features when they are needed. To manage state and orchestrate workflows, we'll use the Durable Functions extensions.

Now would be a great time to take a minute and get familiar with the feature by reading the Durable Functions documentation

You can either use the NuGet package manager built-in to Visual Studio 2019, use this Visual Studio Code extension, or type this from the command line:

dotnet add package Microsoft.Azure.WebJobs.Extensions.DurableTask --version 2.0.0

With that in place, we can tackle state next.

Create the Functions Entities

Durable Functions provides a feature called entity functions to manage state. State can easily be represented by a plain old C# object, or POCO. There are two aspects to managing state:

1. A set of properties you wish to track that can be scoped to a unique identifier called and Entity ID
2. A set of operations that may mutate the state and are guaranteed to execute sequentially so there is no risk of conflicts due to concurrency at scale

The statistics for registries and items are global, so the key can be any arbitrary value. First, I created an interface to describe the available operations.

public interface IRegistryStats
{
    void NewRegistry();
    void NewItem();
}

Next, I implement the class and properties.

public class RegistryStats : IRegistryStats
{
    public int RegistryCount { get; set; }
    public int ItemsCount { get; set;}

    public void NewItem()
    {
        ItemsCount += 1;
    }

    public void NewRegistry()
    {
        RegistryCount += 1;
    }
}

So far this is basic C#. There is one more step: I have to "glue" the entity to the Durable Functions framework. This is done by adding a single method that is triggered when the entity is referenced and informs the runtime to use the class. First, a few using statements:

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using System.Threading.Tasks;

Next, the method I add to the RegistryStats class.

[FunctionName(nameof(RegistryStats))]
public static Task Run([EntityTrigger] IDurableEntityContext ctx)
    => ctx.DispatchAsync<RegistryStats>();

The EntityTrigger is the entry point to interact with the entity. The function name is the type the entity will be referenced as, which is dispatched. We'll look closer at how this integrates with our app in a bit. Next, let's define the operations for a registry. Unlike the stats, registries can have multiple instances, so they have a unique identifier.

public interface IRegistryList
{
    void New(string id);
    void AddItem(string item);
}

What I love about the runtime is that it has no problem handling lists, so the full implementation looks like this:

public class RegistryList : IRegistryList
{
    public string Id { get; set; }
    public List<string> Items { get; set; }

    public RegistryList()
    {
        Items = new List<string>();
    }

    public void AddItem(string item)
    {
        this.Items.Add(item);
    }

    public void New(string id)
    {
        this.Id = id;
    }

    [FunctionName(nameof(RegistryList))]
    public static Task Run([EntityTrigger] IDurableEntityContext ctx)
        => ctx.DispatchAsync<RegistryList>();
}

Like the stats, this class implements properties (the unique identifier and the items list), operations (New and AddItem) and the "glue" for Durable Functions. Notice that I make sure to initialize the list in the parameterless constructor.

Build the Orchestration

The entities will manage state. Now it's time to tackle the workflow! I'm working backwards by creating the workflow first, then going back to set up the HTTP endpoints that interact with the workflow. There is a simple rule of thumb to use when writing orchestrations. Behind the scenes, they use event sourcing to manage the workflow. This means that when a workflow is suspended, it may "replay" existing events to reach the current state. Therefore, there are code constraints to follow. I simplify them all with this rule:

Don't await any method that is not available on the orchestration context!

Let's look at the orchestration (I'll refer to it as "the workflow"):

[FunctionName(nameof(RegistryOrchestration))]
public static async Task<bool> RegistryOrchestration(
    [OrchestrationTrigger]IDurableOrchestrationContext context,
    ILogger log)
{
    // TODO: create the list 

    bool closedByUser = false;

    using (var timeoutCts = new CancellationTokenSource())
    {
        var dueTime = context.CurrentUtcDateTime
            .Add(TimeSpan.FromMinutes(5));
        var approvalEvent = context.WaitForExternalEvent<bool>(CLOSE_TASK);
        var durableTimeout = context.CreateTimer(dueTime, timeoutCts.Token);
        var winner = await Task.WhenAny(approvalEvent, durableTimeout);

        if (winner == approvalEvent && approvalEvent.Result)
        {
            timeoutCts.Cancel();
            closedByUser = true;
        }
    }

    return closedByUser;
}

You may be surprised at how little code it takes! Like other function endpoints, this function has a name (RegistryOrchestration) and a trigger (OrchestrationTrigger). The trigger passes in the context. I also pass in a log (I omitted the log statements to simplify the example). The first step is to create a timer. The timer is created using a date from the context (this is one of the code constraints) and set for 5 minutes out. Another task is created to wait for an external event. When the user closes the registry, an event is sent that cancels the timeout. The CLOSE_TASK constant is just a simple string literal.

The orchestration waits for either 5 minutes to pass or the close event to be sent, in whatever comes first style. The orchestration status will be used to determine if the registry is "open" or "closed." Once the final status is returned, the orchestration will end and save a flag indicating whether it timed out or was closed by the user.

The magic happens at the await statement. The workflow state is automatically saved, and a message placed on a queue with a timestamp 5 minutes in the future. Either the queue will "wake up" and re-enter the method after the await, or the external event will be sent. The state will be automatically refreshed, and the orchestration will end.

The only thing left to do is add the code to open a new registry! This is a "side effect" that doesn't use one of the context methods, so we wrap it in a special ActivityTask.

Create the Activity Function (Side Effects!)

Orchestration steps with side effects (basically, whenever you must await something not on the context) are wrapped in an ActivityTask, yet another function. This is the implementation:

[FunctionName(nameof(NewList))]
public static async Task NewList(
    [ActivityTrigger]string id,
    [DurableClient]IDurableEntityClient client,
    ILogger log)
{
    await client.SignalEntityAsync<IRegistryStats>(
        StatsId,
        entity => entity.NewRegistry()
    );

    await client.SignalEntityAsync<IRegistryList>(
        id.AsRegistryId(),
        entity => entity.New(id));
}

The activity is triggered with the id of the job. For this solution, I share the auto-generated workflow id with the id of the registry. The activity injects an IDurableEntityClient instance to explicitly manage state. There are two steps. The first step uses the operations interface for the stats to signal that a new registry is created. The second step uses the operations interface for the registry to create a new instance.

In both cases, if the instance doesn't exist yet, it will be created prior to calling the method. The "signal" call takes an identifier for the entity and a callback with that entity so that you can call the corresponding method (operation) on the entity.

The id for the stats is stored in StatsId:

public static readonly EntityId StatsId = new EntityId(
    nameof(RegistryStats), string.Empty);

It contains the entity name (RegistryStats) and the unique identifier (an empty string because there is only one overall stat instance).

The id for the registry is generated from the id using an extension method:

public static EntityId AsRegistryId(this string id)
{
    return new EntityId(nameof(RegistryList), id);
}

Again, it contains the entity name and the unique identifier. Finally, we can replace the TODO line in the workflow with a call to the activity:

await context.CallActivityAsync(nameof(NewList), context.InstanceId);

This triggers the activity and passes in the workflow id. The orchestration is ready, but it must be triggered somehow. For that, we'll implement the Open API.

Kick-off the Orchestration with an HTTP Trigger

The Open endpoint takes no parameters. Its sole responsibility is to kick off a new workflow and return the unique identifier for that workflow so it can be used in subsequent API calls. It is triggered by an HTTP GET request, so the implementation looks like this:

[FunctionName(nameof(Open))]
public static async Task<IActionResult> Open(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req,
    [DurableClient]IDurableClient starter,
    ILogger log)
{
    var id = await starter.StartNewAsync(nameof(RegistryOrchestration),
        (object)$"List opened at {DateTime.Now}");

    return new OkObjectResult(id);
}

The trigger sets up the ability to invoke the function from HTTP. The IDurableClient instance is injected to trigger the workflow. Workflows can take input parameters (the same way this workflow outputs whether the user closed the registry, or it timed out) but there are no parameters, so we just pass some arbitrary information. The workflow is scheduled, and the unique identifier passed back. This is returned to the client. There is no guarantee the workflow has started when StartNewAsync returns, but because it is scheduled there should be minimal delay.

Implement Add

The Add API is an HTTP endpoint that allows the end user to add an item to an open registry. The signature for the function looks like this:

[FunctionName(nameof(Add))]
public static async Task<IActionResult> Add(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "Add/{id}")]
        HttpRequest req,
    string id,
    [DurableClient]IDurableClient client,
    ILogger log)

It is triggered by a GET request. Notice the route maps the identifier as part of the URL. The item to add will be parsed from the query. An IDurableClient is passed to signal the registry entity.

The first step is to retrieve the related workflow:

var instance = await client.GetStatusAsync(id);

If the instance is null it means the id passed is incorrect. Otherwise, we check the RuntimeStatus to ensure it is OrchestrationRuntimeStatus.Running because if the workflow has completed, it means the registry is closed and can't be added to. After all the checks pass, there are two operations.

First, the item is added to the registry:

await client.SignalEntityAsync<IRegistryList>(
    id.AsRegistryId(),
    list => list.AddItem(item));

Next, the item count for stats is updated:

await client.SignalEntityAsync<IRegistryStats>(
    StatsId,
    stats => stats.NewItem());

Finally, 200 - OK is passed back to the client.

Implement the Finish Function

The Finish function is an HTTP API endpoint as well. The signature is like Add:

[FunctionName(nameof(Finish))]
public static async Task<IActionResult> Finish(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "Finish/{id}")] HttpRequest req,
    string id,
    [DurableClient]IDurableClient client,
    ILogger log)

It performs the same steps of grabbing the related workflow and ensuring it is running. If it is found and still active, the function simply raises the CLOSE_TASK event that the orchestration is waiting for:

await client.RaiseEventAsync(id, CLOSE_TASK, true);

That's it!

Bonus: Add the Peek Function

The bonus is to provide a function that returns the status and contents of a registry. The signature should be familiar by now:

[FunctionName(nameof(Peek))]
public static async Task<IActionResult> Peek(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "Peek/{id}")] HttpRequest req,
    string id,
    [DurableClient]IDurableClient client,
    ILogger log)

A status variable is used to determine whether it is open (i.e. running) or closed:

var status = "Open";
if (instance.RuntimeStatus != OrchestrationRuntimeStatus.Running)
{
    status = "Closed";
}

The state of the registry is obtained by calling ReadEntityStateAsync:

var registry = await client.ReadEntityStateAsync<RegistryList>(
    id.AsRegistryId());

This returns an EntityStateResponse<T> where T is RegistryList. The POCO we defined will be available at registry.EntityState. The function returns an object with an anonymous type that contains the status and the registry:

return new OkObjectResult(new
{
    status,
    registry = registry.EntityState
});

That will serialize to JSON that looks like this:

{
    "status":"Open",
    "registry":{
        "id":"57c9e29268dd47298ff77e0d5fa6e0f2",
        "items":[
            "Commodore 64",
            "Zork Trilogy",
            "Dune Movie Poster"]
    }
}

... and that's a wrap!

Next Steps 🏃

The full repository is available here: Durable Registry.

Learn more about Durable Functions with Free Training!

✅ Create a long-running serverless workflow with Durable Functions

Want to submit your solution to this challenge? Build a solution locally and then submit an issue. If your solution doesn't involve code, you can record a short video and submit it as a link in the issue descciption. Make sure to tell us which challenge the solution is for. We're excited to see what you build! Do you have comments or questions? Add them to the comments area below.

Watch for surprises all during December as we celebrate 25 Days of Serverless. Stay tuned here on dev.to as we feature challenges and solutions! Sign up for a free account on Azure to get ready for the challenges!