Ruminations on ValueObjects, DataTransferObjects, Back-end For Front-ends, and Functional Programming Data Modelling

Interesting domain modelling challenge happened this week. Remember the difference between ValueObjects and DataTransferObjects? DTO's are over the wire and VO's are your domain objects, usually bereft of behavior?

This was a pattern I thought I'd never use again.

Context: I've seen some commonalities to the work I've been doing lately. In GraphQL/ReScript/Elm, we define records which are basically VO's; data that represent a thing in our domain.

The UI will show a "Loan", the API will fetch Loans, GraphQL makes sure the types line up.

However, throughout the system, not all Loans are the same. Borrowing ideas from Domain Driven Design, I've attempted to make a bounded context on my back-end for front-end: a Loan is the same thing for my BFF and UI; there is only 1. I have an entire library devoted to this.

Multiple services return Loans, but not all the loans they return are the same. I make them the same. It's hard and requires work and time, but it's worth it: the UI and API speak the same language and there is no ambiguity to what a Loan is. The compilers enforce it.

Of course the trade off is a bit more data than most people need. GraphQL allows you to only select a sub-set of fields for data you don't need, and we do this even for an API built specifically for a UI.

And this is where the interesting part comes in.

Traditionally, I've rarely used DTO's and VO's in the same code base. I didn't know about DDD at the time, nor a simple "map" function. The idea of "the back-end has their version of a thing, and I have mine" was strong, but that never correlated into clear code boundaries.

Instead, I traditionally would mirror whatever the back-end said in my UI since BFF's weren't used back in the 2,000's. The back-end was the source of truth and the UI would model based on those objects.

As code bases grew, this started to become painful for numerous reasons.

The first was if the back-end didn't have a consistent "thing", this would infect the UI as well. Conversely, if the UI needed to add context to a thing, it'd "infect" the data. Debugging with a back-end dev, they'd get confused "why does your thing have an isSelected boolean?"

Any React/Angular developer knows why; I need it for List Item/Checkbox state. This spiraled into a lot more business stuff when you'd need context from another part of the app, and "stuff" extra data on the VO. Back-end devs would have no idea what "other parts of the app" meant

In their eyes, "Here's the data for this REST call, and here's the data for this REST call that's slightly different", but in a statefull SPA UI... it's all there, there is none of this "page by page" request you see in Jamstack/traditional web pages.

Communicating the data differences got really challenging. You'd separate "your version of the data vs mine", but that idea wasn't really accepted, and DDD wasn't really talked about, and barely anyone talks about UI vs BFF vs API data in blogs.

Sam Newman published his BFF article in 2015 when they started to become commonplace, both from Node.js, FaaS/PaaS empowering front-end folks to build their own API's outside of their UI code, and more server-side devs doing UI work.

Suddenly, UI's got a lot cleaner on the service front; the calls were "just fetch some JSON", and removed TONS of boilerplate to compensate for weird data models, multiple calls just to show 1 screen, or the worse multiple calls to make 1 ValueObject.

Domain Modelling, making records for your business data, is hard. It's harder still when you're working with legacy code that's hard to change, but your business changes. Throw a UI + BFF on top of this, and no one really knows what a Thing really is.

I had a situation this week where I was getting a Loan, and noticed 3 places with slight deviation with critical data. The defacto way for me to handle that now is, in ReScript using Jzon, it parses the DataTransferObject, and make my own ValueObjects for the UI.

It's not that cut and dry, sadly. For example, if you need to know if a version of a thing is v2, but that Object doesn't have it, you add it. That's normally on your ValueObjects in Object Oriented parlance:

type thing = {
  id: string,
  __version: int
}

... but the DTO doesn't have it, it just has id:

type thing = {
  id: string
}

The normal flow I've gotten into is each REST call has a module that simplifies it into a single function. Don't like the types that come out of that module? Map them to your own thing.

This works well if each service is unrelated, but when many start requiring inputs of, and returning the same "thing", suddenly lack of DRY (don't repeat yourself) gets out of control. This is a common problem of microservices.

But for stateful UI's, the code is all co-located, so it's not... right? Well, that's what the BFF is trying to facilitate, sure, but it's not that straightforward.

I defined that previous Loan record earlier with an id & a version so the entire library "knows" what a Loan is.

Problem? Service A returns a Loan with an id, Service B returns a Loan with an id and version. Suddenly Service A doesn't compile; it needs a version. You can't add it to the REST API for "reasons", so what do you do?

My pattern is to use Jzon as a DTO parser as much as possible:

However, if there are missing fields, or fields I need to add, I'll do my best to enforce other modules map to their own types.

In the case of microservices/Lambdas required to respond to GraphQL/AppSync, they have to follow the GraphQL contract and provide all data. So they'll often end up making multiple calls, mapping these DTO's to their own VO's that have all the data they need.

This clear separation seems to make sense on the surface:

• service: encapsulated module that types a REST call
• GraphQL Lambda: encapsulated BFF call that returns correct GraphQL type

... and you just map back-and-forth when they talk.

But it's not that clean, and is way more messy in practice. Here is the first Jzon parse in Service A:

let thing = Jzon.object1(
  ({ id }) => ( id ),
  (( id )) => { id } -> Ok,
  Jzon.field("id", Jzon.string)
)

And here's Service B:

let thing = Jzon.object2(
  ({ id, version }) => ( id, version ),
  (( id, version )) => { id, version } -> Ok,
  Jzon.field("id", Jzon.string),
  Jzon.field("version", Jzon.string)
)

Are they both DTO's? Is Service A the DTO and Service B the VO and I just map Service A's DTO to the VO?

let mapDTOtoVO = (dto, version) =>
    { id: dto.id, version }

Where do I do this; in a parent service that "fixes" the data? In the consumer Lambda?

Jzon has the ability to make properties optional. Saying "version" is optional is wrong, though; in your bounded context for the VO, it's not wrong it all, it's required! Optional would be middle name in a form field; some people just don't fill it out.

Jzon also has the ability to make properties default; meaning if it's undefined/not there, you just provide a default. This is the tactic I've been using to "fix the DTO's that are supposed to be VO's". This is familiar to UI devs decorating JSON objects they get from back-end.

I'll add things like Jzon.field("__version", Jzon.string) -> Jzon.default("???")

This breadcrumb means in the Jzon parser, I know it doesn't have that data, and I have to either fetch it or add it.

switch versionMaybe {
| "???" => { id, version: getVersion() } -> Ok
| _ => { id, version: versionMaybe } -> Ok
}

That seems just like a default parameter in a function right? Well... no. This parsing code is in a module. That module is wrapping a service.

That service won't ever pass in version... so that _ block will never run, it's just there to make the compiler happy. You know you'll never get version, and will always generate it yourself.

Opposite is true in Service B: API always returns version, ??? never happens.

The pro? Your Loan record is the same across the entire code base. If you have a library or BFF abstracting many microservices, in his bounded context, a "Loan" or any "thing" is a single entity, you only have 1, and everyone knows what it is because the compiler helps you.

This meshing of DataTransferObjects and ValueObjects seems to work extremely well in green field projects, but as soon as you have any deviation, things get messy fast. This is because more services may not match up, don't have all the data, or they don't share record types.

Their lack of clear domain modelling can infect you, and now you're writing compensation code to "fix it". I've struggled to find ways to indicate this. Using double-underscores in front of these properties "__var" is a start.

For example, look at this record...

{
  id: "14",
  name: "Cow",
  isSelected: false
}

A UI dev would immediately know what was added by the UI, and a back-end dev probably not.

Contrast that with a mapper:

let mapDTOToVO = (dto:BackEndDTO):UIVO =>
  ({ id: dto.id, name: dto.name, isSelected: false })

Even if you're not a UI dev, that function makes it way more obvious what you're doing.

"Ah, the database data has no idea what a checkbox is. The UI dev needs to store this state somewhere, so decided to store it with the data itself since that's what the UI is drawing, makes sense."

I've done my best to follow the mapping back-and-forth between services/modules, but that becomes a ton of code to maintain. The compiler helps, but even with sleep, the sheer amount becomes tiring.

Creating a single record of what a "thing" is sounds great in theory.

... but then when each service may differ, you get into these weird "ok, maybe treat it like a DTO, and we'll make a VO for just this service". Once you bring in versions of API's changing over time... well, I hope you have ample coffee.

In summary, it's fascinating the tradeoffs and interplay between a UI's ideal typed "thing" and all the work the BFF does to make that a reality. Sometimes it's not as simple as "loading a JSON object from MongoDB" when you start adding legacy + complicated domain into the mix.

I'm still learning the tradeoffs, maintenance cost, and most importantly, trying to indicate clear intent in the code.

In OOP, you can find copious articles about VO's and DTO's. In Functional Programming, you can find a few articles about how you mirror the VO concept since almost all of them (F#, Elm, Haskell, Scala) have records... which when used with a back-end are basically VO's.

... but the DTO concept, and mapping between back-end record and the record you actually intend to use in your code/module/service/whatever is rarely discussed. I guess they (FP people) assume since "map" is a cornerstone of FP, why even acknowledge it?

But in Domain Modelling, mapping between domains/bounded contexts is a huge deal.