In most functional programming languages data structures are immutable by default, which is great because immutability eliminates a whole raft of issues from our code, freeing our brains up to worry about the higher level problems we're trying to solve. One of the drawbacks of immutability is how cumbersome it can be to modify nested data structures. In this post we're going to independently discover a better way of "updating" immutable data and in doing so re-invent lenses.

The scenario

In this post we'll imagine that we're working with the following data model.

type Postcode = Postcode of string

type Address =
    { HouseNumber: string
      Postcode: Postcode }

type CreditCard =
    { Number: string
      Expiry: string
      Cvv: string
      Address: Address }

type User = { CreditCard: CreditCard }

So a User has a CreditCard which has an Address. Now imagine that we've been asked to write some code that lets a user update their postcode for the address of their credit card. Pretty easy right?

let setCreditCardPostcode postcode user =
    { user with
          CreditCard =
              { user.CreditCard with
                    Address =
                        { user.CreditCard.Address with
                              Postcode = postcode } } }

Yikes! That's not pretty. Compare that to the imperative version in something like C#.

public static User SetCreditCardPostcode(User user, Postcode postcode)
{
    user.CreditCard.Address.Postcode = postcode;
    return user;
}

Ok, the data model might be mutable and there might be a bit more faff in the method declaration, but it's hard to argue with the fact that the actual set operation is much clearer in the imperative style.

Composing a solution 🎼

Instinctively, what we'd like to do is write functions that take care of setting their respective bits of the model and then compose them when we want to set data that is nested inside a larger structure. For example let's write some setters for Address, CreditCard and User in their respective modules.

module Address =
    let setPostcode postcode address = 
        { address with Postcode = postcode }

module CreditCard =
    let setAddress address card =
        { card with Address = address }

module User =
    let setCreditCard card user =
        { user with CreditCard = card }

I've omitted writing setters for every single property for brevity.

These functions are nice because they're very focused on a singular piece of the data model. Ideally, to write setCreditCardPostcode we'd be able to compose these individual functions to create a new function that can update the postcode inside a user's credit card. Something like this.

let setCreditCardPostcode: (Postcode -> User -> User) =
    Address.setPostcode
    >> CreditCard.setAddress
    >> User.setCreditCard

Aside: >> is the function composition operator, so that f >> g is equivalent to fun x -> x |> f |> g. More concretely if we had let addOne x = x + 1 then we could write let addTwo = addOne >> addOne.

But... it's not going to compile! The problem is that Address.setPostcode has the signature Postcode -> Address -> Address and CreditCard.setAddress has the signature Address -> CreditCard -> CreditCard. So when we write Address.setPostcode >> CreditCard.setAddress then the output of Address.setPostcode (which is Address -> Address) does not match the input to CreditCard.setAddress (which is just Address).

Aligning the types

Our first attempt, whilst not quite right, is pretty close. The types nearly line up. Let's see if we can align the types so that the output of one setter can feed straight in to the input of the next one.

If we look again at the output from Address.setPostcode postcode then we see it's a function whose signature is Address -> Address. That is, when we partially apply a postcode to setPostcode, it creates a function that can transform an address by setting the Postcode property to the value we partially applied. So how about if we change the input of CreditCard.setAddress to take an address transformation function, rather than just a new address value. In fact, there's nothing special about CreditCard.setAddress so let's make this change for all of our setters.

module Address =
    let setPostcode (transformer: Postcode -> Postcode) address =
        { address with
              Postcode = address.Postcode |> transformer }

module CreditCard =
    let setAddress (transformer: Address -> Address) card =
        { card with
              Address = card.Address |> transformer }

module User =
    let setCreditCard (transformer: CreditCard -> CreditCard) user =
        { user with
              CreditCard = user.CreditCard |> transformer }

You might feel like this is a hack in order to make composition work, but what we've actually done is made a much more powerful "setter" function. Each "setter" is now capable of taking any transformation function which it applies to the current value and then returns a new version of the data with this modification. If we think about it, setting a property is just a special case of this more general transformation where we ignore the existing value.

What we've actually created here are more like property modifiers than just setters. Each modifier has the signature ('child -> 'child) -> ('parent -> 'parent), which means given a function that can modify some child property, then I'll return you a function that updates the parent type. So let's rename them to modifyX instead and see if we can now create setCreditCardPostcode in the composition style that we wanted.

let setCreditCardPostcode: (Postcode -> User -> User) =
    Address.modifyPostcode
    >> CreditCard.modifyAddress
    >> User.modifyCreditCard

Hmmm, it's still not quite right. The type of setCreditCardPostcode is actually (Postcode -> Postcode) -> (User -> User), which in hindsight is obvious because all we've done is compose modifiers, not setters. So we've actually just created a new "modifier" here that lets us modify the postcode property of the user's credit card. In order to do a "set" operation we just apply the transformation that does the "set" to the "modifier".

let setCreditCardPostcode (postcode: Postcode): User -> User =
    (Address.modifyPostcode
     >> CreditCard.modifyAddress
     >> User.modifyCreditCard)
        (fun _ -> postcode)

So we compose our modifiers and then partially apply it with a transformer that just ignores the input and sets the value to the supplied postcode.

If you've followed up to this point then you've grokked the core principles, which is that if we have modifier functions that know how to update their one piece of the model, then we can chain them together to build modifiers that operate across many nested layers of a larger data structure. Everything that follows from now will be just tidying this up and extracting the generic parts.

Generic property modifiers

It should be clear from the last implementation of setCreditCardPostcode that in order to set a nested property we do two things.

1. Compose the necessary property modifiers to create one that can operates across many layers of a nested data structure.
2. Apply a transformation function that ignores the current value and just returns the new value that we want to set the property to.

Given that all of our property modifiers are of the form ('child -> 'child) -> ('parent -> 'parent), we should be able to write a set function that works for any modifier. It's really simple and just looks like this.

let set modifier (value: 'child) (parent: 'parent) =
    modifier (fun _ -> value) parent

We can even define the modifyCreditCardPostcode in the User module.

module User =
    let modifyCreditCardPostcode =
        Address.modifyPostcode
        >> CreditCard.modifyAddress
        >> modifyCreditCard

And then use it whenever we want to set a new value, as in user |> set User.modifyCreditCard "A POSTCODE" and we could also use it to transform a Postcode, as in user |> User.modifyCreditCardPostcode (fun (Postcode postcode) -> postcode |> String.toUpper |> PostCode). That's a nice separation of concerns.

Combing getters and setters

We might be tempted to stop here, and for the purposes of our initial problem regarding awkward data updates we've achieved our goal, but it would be nice if we could make this concept of property modifiers even more universal. In particular if we could combine the closely related acts of getting and setting a property in a single function.

If we look at Address.modifyPostcode again we'll see that it contains a "get" operation for the Postcode property.

module Address =
    let modifyPostcode (transformer: Postcode -> Postcode) address =
        { address with
              Postcode = address.Postcode |> transformer }
                       // ^ getting here ^

It's possible to rearrange this slightly and put the "get" first and pipe it in to a function that does the "setting".

let modifyPostcode transformer address =
    address.Postcode
    |> transformer
    |> (fun postcode -> { address with Postcode = postcode })

It's now clear to see that our modifiers perform the following operations.

1. Get the child data.
2. Transform the child data.
3. Update the parent with the transformed child value.

So if we could somehow find a way to skip the final step, then we'd have ourselves a getter. The only thing we can do to affect the behaviour of modifyPostcode though is to provide a different transformer. Unfortunately, try as we might there's no function we can supply here that will stop the final "setter" step from also running.

One trick we can do though is to make the transformer return a functor, see Grokking Functors if you need a recap. If we do this then in order to then call the final "setter" step we need to map it so that we can apply this "setter" to the contents of the functor we returned from the transformer. So, for example, modifyCodeProperty would look like this.

let modifyPostcode (transformer: Postcode -> '``Functor<Postcode>``) address =
    address.Postcode
    |> transformer
    // Everything's the same until the final line where we call map
    |> map (fun postcode -> { address with Postcode = postcode })

You might still be wondering how that lets us avoid calling the final "setter" step? Well, we can now exploit the map function to change the behaviour of modifyPostcode. If we remember how functors work then map is defined on a per functor basis, so by returning different functors from the transformer we can get different mapping behaviours at the end.

What we need then is a functor whose map instance just ignores the function being applied to it. One that just returns its input without transforming it. Fortunately for us such a functor already exists called Const and it's defined like this.

type Const<'Value, 'Ignored> =
    | Const of 'Value
    static member inline Map(Const x, _) = Const x

Map for Const just returns the input x. With that we're in a position to write a generic get function that will extract the child value from any of our modifiers.

let inline get modifier parent =
    let (Const value) = modifier Const parent
    value

What about our set function? Which functor should we return from the transformer in there? Well we need one that just runs the function without modification and that happens to also be a well known functor that goes by the name of Identity. Identity is defined like this.

type Identity<'t> =
    | Identity of 't
    static member inline Map(Identity x, f) = Identity(f x)

It's map instance just calls the function f on the input x and wraps the result back up in another Identity constructor. People often wonder why we'd need such a boring functor, but it comes in handy in these situations. With that set only requires a slight modification from before.

let inline set modifier value parent =
    let (Identity modifiedParent) =
        modifier (fun _ -> Identity value) parent

    modifiedParent

Putting it all together 🧩

We've made quite a few changes to things now, so let's see it all together. We'll start with the signature that a modifier must have, then show the get and set functions that work for any such modifier and finally show how we can use them to solve our original problem.

// Modifier signature - notice how the output is completely generic now which supports both our get and set use cases
('child -> '``Functor<child>``) -> ('parent -> 'a)

let inline get modifier parent =
    let (Const child) = modifier Const parent
    child

let inline set modifier value parent =
    let (Identity modifiedParent) =
        modifier (fun _ -> Identity value) parent

    modifiedParent

module Address =
    let modifyPostcode (transformer: Postcode -> '``Functor<Postcode>``) address =
        address.Postcode
        |> transformer
        |> map (fun postcode -> { address with Postcode = 
    postcode })

module CreditCard =
    let modifyAddress (transformer: Address -> '``Functor<Address>``) card =
        card.Address
        |> transformer
        |> map (fun address -> { card with Address = 
    address })

module User =
    let modifyCreditCard (transformer: CreditCard -> '``Functor<CreditCard>``) card =
        user.CreditCard
        |> transformer
        |> map (fun card -> { user with CreditCard = 
    card })

let setCreditCardPostcode postcode user =
    user
    |> set
        (User.modifyCreditCard
         << CreditCard.modifyAddress
         << Address.modifyPostcode)
        postcode

let getCreditCardPostcode user =
    user
    |> get (
        User.modifyCreditCard
        << CreditCard.modifyAddress
        << Address.modifyPostcode
    )

Our code is now very close to an imperative style setter. In fact, by reversing the composition operator, from >> to << and switching the order of the modifiers, we've even been able to order the property access in the same way that an imperative programmer would be familiar with, from the outermost to the innermost property. Using << is often frowned upon in general because it can be confusing, so use it at your own judgement.

You just discovered Lenses 🔍

These things we've been calling "modifiers", well they're better known as lenses. Lenses are a better name for them because they're not actually doing any modification, they're just composable functions that focus on a specific part of a data structure. We can define functions like get, typically called view, and set, usually called setl (for set lens), that let us read or write the value that any lens points to because the structure of a lens is completely generic.

There are also many more things that we can do with lenses, which is part of a broader topic called optics, which we haven't covered here. For instance we can easily work with data that might be missing, or focus our lens on specific parts of every element in a list.

Lenses are also about more than just composable getters and setters. They also provide an abstraction barrier for our code. If we access data through a lens rather than directly it means that if we later refactor a data structure we only have to modify the lens and the rest of the code will remain unaffected.

Lenses in the wild 🐗

There are a few lens "conventions" that are probably worth pointing out at this stage, as it's how you'll likely see them written in the wild. This is all just syntactic sugar on top of what we've already discovered, such as things like special operators which just make them a bit more pleasant to write. Below is the same example from above, but written using the FSharpPlus lens library.

#r "nuget: FSharpPlus"

open FSharpPlus.Lens // <- bring the lens operators in to scope

module Address =
    // Lenses are usually named with a leading underscore
    let inline _postcode f address =
        f address.Postcode <&> fun postcode -> { address with Postcode = postcode }

module CreditCard =
    // We also usually just name after the property they point to
    let inline _address f card =
        f card.Address
        <&> fun address -> { card with Address = address }

module User =
    // The <&> is just an infix version of map
    let inline _creditCard f user =
        f user.CreditCard
        <&> fun card -> { user with CreditCard = card }

let setCreditCardPostcode postcode user =
    // We can use the .-> as an infix version of setl
    user
    |> (User._creditCard
        << CreditCard._address
        << Address._postcode)
       .-> postcode

let getCreditCardPostcode user =
    // We can use the ^. operator as an infix version of view
    user
    ^. (User._creditCard
        << CreditCard._address
        << Address._postcode)

A few things to point out here:

1. Typically lenses are named like _propertyName.
2. What we used to call transformer we often just denote as f.
3. Instead of writing map it's common to write the lens using the <&> operator. This is just a flipped infix version of map and it lets us create the lens from a getter (to the left of the operator) and a setter (to the right of the operator).
4. We can use the .-> operator as an infix version of setl, which gives us an even more imperative style looking setter.
5. We can also use .^ instead of view to get the value, which is a kind of analogous to the . operator in OOP.

What did we learn? 🧑‍🎓

Lenses allow us to write property accessors which we can compose to focus on different parts of a large data model. We can then pass them to functions like view or setl to actually view the data or set it.

Lenses are also a great abstraction barrier that we can use to decouple our code from the specifics of our data models current structure. They also allow us do other useful transformations which we haven't gone into here. Lenses, and the broader topic of optics, is a large one, but with this intro you should find it much easier to explore what else they have to offer.