[Configuration in Azure Functions Series - Part 2] A deeper dive into Azure Functions configuration

This article was originally published at blogs.siliconorchid.com on 1-Nov-2019

This is part two of a series exploring .NET Core configuration, with an emphasis on Azure Functions. In this article, we review how configuration in Azure Functions is recommended to be used, how it differs from ASP.NET Core and some of the potential issues and confusions surrounding its use.

In part 1 of this series, we introduce the subject of configuration and review how ASP.NET Core configuration works.

In part 2 of this series, we look at how configuration in Azure Functions (v2) works and talk about some of the issues.

In part 3 of this series, we show how you could include ASP.NET Core configuration into an Azure Function project.

In part 4 of this series, we look at using other configuration services, specifically Azure App Configuration and Azure Key Vault

As a reminder from part 1, this series has been written from the perspective of the developer who may already be accustomed to developing with ASP.NET Core, and is wanting to learn about the slightly different way that Azure Functions v2 goes about handling application configuration. This is why we draw parallels to ASP.NET Core throughout this article on Azure Functions.

Configuration, the Azure Functions v2 way

Let's now focus our attention towards Azure Functions.

The starting point for the Azure Function documentation can be found here: Microsoft Documentation: Azure Functions documentation

Unlike the original Azure Functions v1, with the newer Azure Functions v2, we can develop our solution using .NET Core.

Azure Functions v2 can actually be written in a number of different languages, which again illustrates that ASP.NET Core and Azure Functions have evolved over time as different products.

If you are familiar with ASP.NET Core, it may feel appropriate, given the common language and other similarities, that we should just be able to both define and consume configuration in exactly the same way.

However, with Azure Functions this is not the case - it's a totally different runtime - and it often goes about things differently.

For example, as ASP.NET Core developers, our instinct will most likely be to develop the code locally (using an IDE such as Visual Studio or Rider) and ultimately, build and deploy our work someplace (let's assume an Azure WebApp) as a separate process.

In contrast, with Azure Functions, local development is not the defacto workflow. Microsoft's original intent was to create standalone units of code that could be edited directly within the Azure Portal, using a much wider range of programming languages, with the hosting and scaling aspects completely abstracted away.

It is only in the past year, with the release of Azure Function v2, that we now have the option to develop locally, using Azure Functions Core Tools and deploy compiled code to the cloud. The same runtime is used both locally for development and for deployment in the cloud.

We are reminded of these origins, by the fact that we can still take our C# code and copy it into Azure, in the form of C# script (.csx) files.

What does serverless really mean?

If you've looked into Serverless computing before, you'll most likely be familiar with the long-running joke, made by most presenters of Serverless Technology, that "of course there are servers".

From a marketing perspective, the Serverless mantra suggests that self-contained units of code are untethered from our conventional concept of a host (especially when using the consumption model) and that Functions exist in isolation, with everything they need, in terms of configuration, either hard-coded or made available through a simplified configuration system.

We need to clarify that in reality, multiple Functions do in fact, still just run on individual AppService hosts.

Even without modification, it is the host that facilitates the various triggers. If we peek under the hood of the Functions host code we can see that really, it's just an ASP.NET Core app!

There is, of course, much more to Azure Functions than just AppService hosts, as the runtime of the wider platform is aware of which triggers need to be monitored (e.g. monitoring a Service Bus Queue) and manages the appropriate provisioning and scaling of hosts) - but for the purpose of this article, we're only really interested in thinking of a single host in isolation.

As with an MVC/WebAPI application, running on a more conventionally-provisioned and long-running Azure WebApp host, this means that an individual Function can in fact have access to any longer-running services and resources if needed.

We can readily demonstrate this by introducing an appropriate startup class to the project. We can then dive in and write code that does tasks such as setting up our own common services, which then exist for the lifecycle of that host.

By doing this, Functions start to look much more like an ASP.NET Core application running on comparatively short-lived server instances.

A distinction is that unlike ASP.NET Core, Azure Functions effectively have controllers that can be written in a wide range of different languages.

Why do I need to know this?

The relevance of knowing that we have the option to access the host (and that we can configure it, just like any other .NET Core application) is that this means that in theory, we could define alternative configuration-providers.

This opens the door to using the ASP.NET Core style of configuration. We'll be looking at this subject properly, in part 3 of this series.

However, before we start diving into that particular subject, let's back up and talk about how we are supposed to be using configuration, in Azure Functions, under conventional circumstances...

Different types of Azure Function

Let's start by looking into how Azure Functions are used from a high-level.

You can read the official documentation on this subject here: Microsoft Documentation: Azure Functions triggers and bindings concepts

The type of a Function is determined by its Trigger and Bindings.

• a trigger is what causes a function to run (e.g. an HTTP-trigger).
• the binding is a way to connect the input and/or output of the Function to some other resource (e.g. outputting a result to a queue).

Microsoft provides the platform with support for a wide variety of different Triggers and Bindings.

It's possible to define your own Function types, but that is out of scope for this article. Instead, check out Jason Roberts: Creating Custom Azure Functions Bindings

In code, the type of a particular Function is declared by using a parameterised C# Attribute.

These parameters vary from type to type, but typically they are the place where information related to configuration can be identified.

The following sample of code was produced directly by the template for the CosmosDbTrigger function:-

public static class Function1
{
    [FunctionName("Function1")]
    public static void Run([CosmosDBTrigger(
        databaseName: "databaseName",
        collectionName: "collectionName",
        ConnectionStringSetting = "connectionstringsettingname",
        LeaseCollectionName = "leases")]
           IReadOnlyList input, ILogger log)
    {
        if (input != null && input.Count > 0)
        {
            log.LogInformation("Documents modified " + input.Count);
            log.LogInformation("First document Id " + input[0].Id);
        }
    }
}

At first glance, we could be forgiven for thinking that the above-highlighted code is expecting us to directly hard-code all of the configuration information, including the connection string to the Cosmos Database.

Thankfully, this is not the case, as the values being requested are usually the key-name that correlates to an item in configuration.

It's a mildly ambiguous situation, so it helps to be warned that you need to pay attention to the naming of the parameters. In this example, it doesn't say ConnectionString, it says ConnectionStringSetting.

With that said, it's not uniformly the case that parameters can always be expected to relate to a configuration-key.

In the above example, some of the values (databaseName, collectionName and leases) are expected to be supplied as hard-coded string literal values. We could say that this is not ideal, however, it should be possible to layer-in our own supplemental configuration and replace the string literals demonstrated in line-5, above, with a call to something like this:

databaseName : Environment.GetEnvironmentVariable("<myDatabaseName>")

What are the prescribed configuration sources?

You should refer to Microsoft Documentation: Manage your function app: Application settings.
and Microsoft Documentation: Work with Azure Functions Core Tools : Local settings file

So what can we take away from the docs?

• In Azure Functions, there is no baseline configuration which is common to all environments (i.e. no appsettings.json or equivalent).

  • If you recall our earlier learning, that not all Azure Function implementations are even deployed from a local development version, then we could say that this is understandable - because the focus has originally been about editing standalone solutions in the cloud, not about deploying multiple versions with different configurations.

• Because there is no baseline configuration to be deployed, when hosting in an Azure App Service, we must use App Service Configuration to specify each and every key.

  • If we have a relatively large number of configuration items, entering via the Azure Portal is likely to be laborious and error-prone, so automated environment scripting may be an option to improve this issue.

• The configuration file for local development is local.settings.json. This is intended for all local settings, including any secrets that we need to use.

  • ASP.NET Core normally uses a default builder in the application startup, to enable us to work with appsettings.json. In contrast, when using Azure Functions, the ability to read settings from local.settings.json is an example of behaviour that is baked-into the runtime. This was exemplified in the previous code sample, where, simply by identifying the configuration-item name, this was enough to for the Function to automatically locate and use the appropriate configuration source.

• We can still layer-in configuration from other configuration providers, such as Azure App Configuration and Azure Key Vault.

Opinion: Compared to the exhaustive information found in Microsoft Documentation: ASP.NET Core Configuration, the equivalent documentation for Azure Functions, as available at the time of writing, is not as insightful. This is not to say that the documentation isn't useful - I just found it to be somewhat terse, short on links to other resources and doesn't provide a great deal of guidance for beginners.

What do we use for local configuration?

The official documentation Microsoft Documentation: Work with Azure Functions Core Tools: Local settings file
tells us that when developing locally, we should be using local.settings.json.

We are advised that the purpose of local.settings.json is (quote):

"stores app settings and connection strings that are used when running locally. This file contains secrets and isn't published to your function app in Azure."

Summarising from various documents, Microsoft has communicated that:

• We should be using the Values section of local.settings.json for our local application settings, secret or otherwise. Here is an example in the docs.

  • The runtime supports a handful of expected nodes (e.g. Host and ConnectionStrings are indicated in the docs) and ignore any custom nodes or structures that we include in the file.
  • Settings found in this Values node equate to counterparts located directly in the root of an Azure AppService Configuration collection.
  • When hosted in Azure, there is no need to attempt to mirror the JSON structure found in local.settings.json by specifying a parent node called "Values". I.e. none of those key-names needs to include any hierarchy delimiters, such as Values__MySetting.

Great. So now we are clear about where we should be saving local configuration settings, right?

Not quite! At this point, the documentation becomes a bit disjointed and suggests a different approach.

According to Microsoft Documentation: Quickstart: Create an Azure function with Azure App Configuration and Microsoft Documentation: Azure Functions C# developer reference, the demonstrated way to store configuration during local development makes no mention of local.settings.json and instead uses Environment Variables.

Environment Variables aren't new to this story, because if you recall from part 1 of this series, they were identified as one of several possible Configuration Providers.

The suggestion to use Environment Variables for local development, was likely intended as a way to maintain consistency with how apps are configured when deployed to the cloud.

Environment Variables are used as a primary configuration choice for Azure Functions because they are consistently available across the various languages and environments that Functions support. For example, you'll find them used in places such as containers.

These instructions show you how to set these values using the CLI for different platforms.

On Windows, you can also access these settings by visiting System Properties → Environment Variables and adding/editing values from there.

But wait, there are even more Environment Settings!

Remember back in part 1, we said that it's easy to be overwhelmed with too many options?

In addition to OS Environment Settings, Visual Studio also provides even more "Environment Settings" for use when debugging:

• In Visual Studio Solution Explorer, right-click on the Functions project and select the Properties option.
• Select Debug - we are presented with a dialogue box with several options. One of these options is Environment Variables.

If you fill out this section with configuration information, the settings get saved not to the environment-settings of the operating system, but as part of yet another type of configuration file: properties/launchsettings.json.

The file could look like this:

{
  "profiles": {
    "HTTPFunction": {
      "commandName": "Project",
      "environmentVariables": {
        "OurCustomSetting2": "settingsfromVS"
      }
    }
  }
}

Something to watch out for, is that:-

• Unlike OS Environment Variables, this file will usually be included in version control - so don't use it to save secrets!
• As far as configuration files go, this is probably not an obvious place to expect to find settings, so it's difficult to recommend using this option.

Enough already - what should I use?

Microsoft presents us with several different options for local configuration, so which one should we use?

The answer to that question inevitably stumbles firmly into the usual territory of "it depends, there is no right answer".

I personally would recommend using local.settings.json over Environment Settings, as this is generally more in keeping with documentation and examples. However, this choice is not without other considerations, which we'll talk about next:

What's the issue with local.settings.json?

It's widely accepted that you should not keep secrets in version control.

However, as we discussed earlier, in Azure Functions, Microsoft advises us to use local.settings.json for local secrets. This is a file that is located within a folder that we can normally expect to be version-controlled.

In my opinion, this brings potential complications and compares poorly to alternative options, specifically User Secrets, which locate the file secrets.json in the developers OS user-profile, completely away from the rest of the project code.

If you use a Microsoft template to create a new Azure Function project, it's rather easy to overlook that there is a .gitignore file included in the root of the project. To be clear, this will be in addition to the .gitignore that will quite likely already be present in the root of our solution.

Even more easy to overlook, is the fact that amongst the large list of common-looking exclusions, there is an entry near the top of that file, which is used to explicitly exclude local.settings.json from version control.

In theory, this exclusion addresses the problem of not including the file in version control.

I believe this poses some potential issues:

• What happens when someone on your development team is not aware of this subtle exclusion, and then deletes this customised .gitignore from the project (e.g. "why have we got a duplicate of the .gitignore, when we've got the same one correctly at solution level?")?
  • Your secrets are at risk of being unintentionally introduced into version control.

• If the local.settings.json is not included in version control, and a team member clones the repo to work afresh on the project - how are they easily supposed to know what configuration items are meant to be part of the project's configuration (without having to hunt around)?

  • I've seen the answer to this in several demonstration projects - which is to include the file local.settings.sample.json in the project, as a way to convey the configuration schema.

    This solution is a somewhat awkward workaround, which as far as I can tell, is undocumented as a recommended practice. It also requires developers to purposefully update this "sample" file, as opposed to updating an actual working version (as we would, if working with an ASP.NET Core appsettings.json)

    This problem of "what are the baseline configuration-settings?" is not unique to local.settings.json as it also rears its head, in just the same way, when using Environment Settings.

Opinion: I should stress that the above is just an opinion and that there is nothing wrong with using local.settings.json with its accompanying use of a version-control exclusion rule. It is simply that everyone in our development team needs to be briefed about the nuances and workarounds.

Note: Whilst researching these articles, I discovered that I am pretty late to the game with my learnings, observations and opinions. A concisely written article by Tom Faltesek, covered many of the issues that I wanted to write about, a whole year beforehand (2018).

How do we manually read configuration settings?

We've learned that Function attributes are used as a mechanism to specify certain configuration-settings, which typically are required as binding parameters.

However, there are plenty of other scenarios where we need to request configuration-settings from within the body of our own code. For example, we may want our function to add some information to a queue, so how do we go about retrieving a connection string (to that queue resource) in the body of our Function?

Referring back to those same two documents; Microsoft Documentation: Quickstart: Create an Azure function with Azure App Configuration and Microsoft Documentation: Azure Functions C# developer reference, the demonstrated way to retrieve configuration during local development is to use this piece of code:

var configValue =  System.Environment.GetEnvironmentVariable("<app setting name>", EnvironmentVariableTarget.Process);

• This does imply that we need to use a string literal value for the configuration-setting key, everywhere that it needs to be requested. We could say that this is a somewhat brittle way to write code (we all make typos and this kind of bug can be hard to spot).

  • One way to slightly mitigate against this, is to use a static class with const strings - as exemplified in this earlier article under the heading "GlobalConstants.cs".

• Although the method name GetEnvironmentVariable() sounds like it may only work for retrieving Environment-Variables specifically, this in fact is not the case and this single command can be used to retrieve configuration from all of your sources, whether they be Azure AppService configuration, or the local local.settings.json.

How do we add structure to our configuration settings?

When we talk about configuration sections (aka "Hierarchical values"), we're referring to the ability to group similar pieces of configuration together in groups.

When we are working with ASP.NET Core, with its default use of an appsettings.json file, the structure of such "configuration sections" are clearly visible to both view and work with.

For example, an appsetting.json file could look like this:

{
"EmailSettings": {
     "ApiKey": "secretKeyValue",
     "FromAddress": "do-not-reply@test.com"
     }
}

However, because the use of a JSON file is an abstraction, when we need to work with other configuration-providers, this structure needs to be mapped down into a single string (a basic key/value pair).

This is where things can start to get a little confusing, because the various Microsoft services don't always work in quite the same way.

It's an issue which is further compounded by environment diversity (i.e. Windows, Linux, Containers, etc), so our options for what does and doesn't work will vary.

Usually, hierarchy can be described by using some combination of the delimiting characters : (colon), __ (double-underscore) and/or -- (double-dash).

In most cases, the _ (single-underscore) character can be used to punctuate long strings, to make them more human-readable. This could be used as a way to make related configuration-settings appear to be grouped together (e.g. EmailSettings_ReplyAddress and EmailSettings_ApiKey), but it has no impact other than cosmetic.

Be aware that you may need to experiment, as the official guidance shows there are a mixture of approaches. Let's look at some examples:

• ASP.NET Core. In the guide Configuration in ASP.NET Core : Conventions, the following advice seems to be the most encompassing and most relevant for a majority of scenarios (quote):

  "Within the Configuration API, a colon separator (:) works on all platforms. In environment variables, a colon separator may not work on all platforms. A double underscore (__) is supported by all platforms and is automatically converted into a colon. In Azure Key Vault, hierarchical keys use -- (two dashes) as a separator. You must provide code to replace the dashes with a colon when the secrets are loaded into the app's configuration.
  "

  Later in the same document, in the section Environment Variables Configuration Provider, the advice is refined, by noting that the Bash console may not support colons.

  "When working with hierarchical keys in environment variables, a colon separator (:) may not work on all platforms (for example, Bash). A double underscore (__) is supported by all platforms and is automatically replaced by a colon."

  So, that means that the Azure AppService Application settings that correlates to the JSON example we just looked at, would have keys that look like this (the screenshot demonstrates both the use of a colon and double-underscore):

  

• Azure Functions In the guide Work with Azure Functions Core Tools : Local settings file we are advised that when working with local.settings.json, things don't quite work the same and that we can't describe hierarchy (quote):

  " Setting names can't include a colon (:) or a double underline (__). These characters are reserved by the runtime."

  It's worth underlining (ahem ;-) ), that in the case of Azure Functions, we can still use the _ (single underscore) character to help to improve legibility. You can see an example of this if we provision a new Azure AppService. Out of the box, it comes with a selection of settings already populated, including for example, FUNCTIONS_WORKER_RUNTIME.

• Azure Key Vault. We'll be looking at Azure Key Vault more closely in part 4 of this series, but in the context of this particular topic, it's worth noting that Azure Key Vault introduces its own unique set of restrictions. The advice given in Azure Key Vault Configuration Provider in ASP.NET Core : Secret storage in the Production environment with Azure Key Vault
  , is (quote):-

  "Azure Key Vault secret names are limited to alphanumeric characters and dashes. Hierarchical values (configuration sections) use -- (two dashes) as a separator. Colons, which are normally used to delimit a section from a subkey in ASP.NET Core configuration, aren't allowed in key vault secret names. Therefore, two dashes are used and swapped for a colon when the secrets are loaded into the app's configuration."

Wrapping up part two

If you're adopting Azure Functions having previously worked with ASP.NET Core, it pays to be aware of the subtle differences. Hopefully, this article will have helped you to navigate this adjustment.

To summarise:

• When it comes to configuration, Azure Functions are a little bit opinionated because of baked-in behaviour in its runtime. This applies both to where configuration-settings are expected to be found and also how they should be consumed.

• The Azure Functions configuration system doesn't support hierarchical configuration-settings. The various configuration-providers have differing character restrictions for setting-names, so test that your choices can be used in the environments that you intend to use.

• Azure Functions still have a host. In the startup, we can define services just as we can like any other .NET Core app.

• There are nuances that come with using local.settings.json that your team needs to be aware of.

• This article is based upon Azure Functions v2, so may be outdated in the near future as an announcement has been made that Azure Functions v3 is on the immediate horizon, with support for .NET Core 3 having already been rolled out.

In part 3 of this series, we'll look in detail at the code needed to recreate ASP.NET Core style configuration, by being able to read configuration JSON files and inject strongly-typed configuration models into your Azure Functions.

NEXT: Read Part 3