Netherite Storage Provider is a backend implementation of Durable Functions that delivers high performance.
It's still a preview version, but if you are using Functions Premium (EP1-3), it's worth evaluating it now.
microsoft
/
durabletask-netherite
A new engine for Durable Functions. https://microsoft.github.io/durabletask-netherite
Netherite
Netherite is a distributed workflow execution engine for Durable Functions (DF) and the Durable Task Framework (DTFx).
It is of potential interest to anyone developing applications on those platforms who has an appetite for performance, scalability, and reliability.
As Netherite is intended to be a drop-in backend replacement, it does not modify the application API. Existing DF and DTFx applications can switch to this backend with little effort.
Getting Started
To get started, you can either try out the sample, or take an existing DF app and switch it to the Netherite backend. You can also read our documentation.
The hello sample.
For a comprehensive quick start on using Netherite with Durable Functions, take a look at hello sample walkthrough, and the associated video content We included several scripts that make it easy to build, run, and deploy this application, both locally and in the cloud.…
The use of Netherite requires some additional work, so we will briefly introduce how to use it.
Install Netherite
First, install the Microsoft.Azure.DurableTask.Netherite.AzureFunctions package in your Azure Functions project.
Install-Package Microsoft.Azure.DurableTask.Netherite.AzureFunctions -Version 0.4.0-alpha
dotnet add package Microsoft.Azure.DurableTask.Netherite.AzureFunctions --version 0.4.0-alpha
After installing the package, declare that you want to use Netherite in host.json. There are many configurations in the storageProvider, please refer to the official documentation.
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": false,
"excludedTypes": "Request"
}
},
},
"extensions": {
"durableTask": {
"hubName": "<YourHubName>",
"useGracefulShutdown": true,
"storageProvider": {
"type": "Netherite",
"StorageConnectionName": "AzureWebJobsStorage",
"EventHubsConnectionName": "EventHubsConnection"
}
}
}
}
Set the connection string names for Storage and Event Hubs to the key names in the Azure Functions App Settings or local.settings.json.
Preserve System.Threading.Channels.dll
The current version 0.4.0-alpha does not work with an error when deployed to Azure Functions, so you need to add FunctionsPreservedDependencies to the csproj file to avoid this problem.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<AzureFunctionsVersion>v3</AzureFunctionsVersion>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Azure.DurableTask.Netherite.AzureFunctions" Version="0.4.0-alpha" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.5.0" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.13" />
</ItemGroup>
<ItemGroup>
<FunctionsPreservedDependencies Include="System.Threading.Channels.dll" />
</ItemGroup>
<ItemGroup>
<None Update="host.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Update="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
</Project>
Add a setting to prevent System.Threading.Channels.dll from being removed on deployment. This fix will allow it to work on Azure Functions.
This problem is expected to be fixed in a future version.
Create Function App and Event Hubs
Create Function App and Event Hubs Namespace by using Azure Portal or ARM Template.
- Functions Premium (EP1-3)
- Consumption Plan is not supported.
- Event Hubs Namespace (Standard, 1TU)
- Event Hubs are automatically created by Netherite
Since Netherite creates the Event Hub automatically, it is easier to use RootManageSharedAccessKey for the SAS key.
Use 64-bit worker process
FASTER needs to run in a 64-bit process, so change the Platform setting of Azure Functions to 64-bit.
If you run the application with the 32-bit Platform setting, the following exception will be thrown at runtime and it will not work properly.
Part** !!! Encountered exception while working on store in StoreWorker.Process: FASTER.core.FasterException: Overflow in AddressInfo - consider running the program in x64 mode for larger address space support
Note that when you deploy Azure Functions (Windows) from Azure Portal, 32-bit Platform will be selected by default.
Enable Runtime Scale Monitoring
Durable Functions with Netherite installed running on Functions Premium (EP1-3) need to have Runtime Scale Monitoring enabled in order to scale out.
Since Runtime Scale Monitoring requires warmed-up instances, set preWarmedInstanceCount to 1 when creating resources with ARM Template or Terraform.
After these steps, Azure Functions using Netherite is now working perfectly. Now you can verify the performance and have fun.
Oldest comments (0)