Effective communication between services (and people)

As programmers we known full well that code without side-effects is both perfect and perfectly useless, which should be enough to convince you of how important communication and boundaries (or I/O, using software jargon) are.

But they are not just important, they are ubiquitous: there is a boundary between databases and servers, a boundary between servers and clients, a boundary between clients and the final user, but also boundaries between servers and servers, between clients and third-party softwares, and the list goes on..

At the end of the road, any software we can think of is just implementation detail over the mapping it creates between its domain and codomain.

If that wasn't enough, with the rising of micro-services the problem of communication has become even more apparent as there is an ever increasing number of actors and communications to juggle.

Communication is facing the same scaling problems that it was trying to solve in the first place.

But what is a communication? How can we achieve success when communicating?

I argue that you need two things (plus an additional one that I will mention at the end):

• a shared format: in software engineering this is usually represented by a protocol like HTTP or SOAP, in verbal communication it is sound.
• a shared meaning: in verbal communication it is created by validating decoded input against a more or less shared "common sense" while in software engineering it usually corresponds to input parsing.

Shared format

As applications are loosely coupled, there is no guarantee that the same context will apply to both ends of the communication, which is why you usually have to translate your communication using a "shared format" to make sure that what is being sent can be identified as communication by the receiving party.

When a communication takes place four things happen:

• The sender "serializes" a communication intent into the shared format.
• the sender then sends the communication.
• The receiver receives the communication.
• The receiver "deserialize" the communication using the same shared format used by the sender, then process it accordingly.

Summarizing, we have:

• 2 actors talking in their own dialects
• 1 shared communication protocol

And that is all apparently. Except it is not.

Agreeing on a protocol and knowing how to translate to/from such shared protocol is in fact not enough to guarantee a communication is taking place: it only guarantees the syntactical correctness of the content but there are still zero guarantees about its contextual coherence and its "purpose".

As an example, is this a conversation?

Person_A: "hi person_B! How are you? Everything fine?"
Person_B: "I own five dogs"

Sure both persons were able to "decode" each other's sentences, but I have difficulties marking it as a "successful conversation"... Person_A was probably able to "deserialize" the message sent by Person_B but it doesn't make any sense.. It could mean that life is a blessing because he/she is enjoying the company of five (five!) dogs or it could mean that he/she is on the verge of death because Person_B is very allergic to dogs fur.

There is likely a context missing or a a problem with how Person_B communicate "meaning".

Shared meaning

There is indeed another level of abstraction that we have to secure to deem our conversation successful: we have to agree on a set of intelligible communication intents. In other words, we have to make sure that the receiver can interpret the messages forwarded by the sender in a meaningful way and viceversa.

How do we do it? Well, in human interactions we (usually) have a second "shared asset" other than the communication protocol which is common sense: both the sender and the receiver just assume that they both have the "common-sense" resource available and validate communications against it.
Nonetheless, this is not a perfect guarantee that the conversation will be successful: if the versions of "common-sense" held by the two actors differ in substantial ways the conversation intent will be lost in translation.

Software development is a whole other story. We have to deal with extremely limited resources compared to our brains and even if someone ever succeded in encoding a digital "common sense" (something very far from happening, if ever so) we would still not have a guarantee that our softwares are able to communicate effectively.

The poor man solution to this dilemma is to always encode all valid communication intents and validate every deserialized content against it. If we can decipher the communication intent it is all good, otherwise the best thing we can do is to fail fast, break everything (maybe with a gracious error message) and wait for someone to fix it.

In real life the set of valid communication intents is huge, but in software development it is usually much smaller, often just a singleton set, so this encoding effort, albeit sometimes expensive, is at least doable.

Ergonomicity

That would be all if it wasn't for one last thing that we always do (usually going unnoticed in human communication), which is the translation from the sender dialect to the receiver dialect.

When we talk with someone we always unconsciously map the content of its communication intent to the set of our own internal meanings before analyzing it and creating our response. This is particularly evident when we talk with someone in a different language: we receive a communication that we are able to deserialize through our shared format (sound), validate through our shared meaning (common sense) then, before processing it and continue the conversation with a response that we deem reasonable, we parse it to translate to our own language.

This parsing phase is merely for the sake of ergonomics: we already known that the message is well-formatted and make sense but we transform it into something that our brain can handle faster and better in order to "smoothen the gears".

Conclusions

Each of the three steps depicted are of fundamental importance for a conversation to be successful, yet, especially in software using dynamic languages like JavaScript or PHP, one or more of these steps is often overlooked or even skipped altogether.

My passionate suggestion is to never try to skip the un-skippable, there are tons of great libraries that helps you not only validating but parsing your payloads like io-ts or yup (if you are not willing to give functional programming a try :))