DEV Community

Cover image for Data-Oriented programming: Liberate yourself from objects
Yehonathan Sharvit
Yehonathan Sharvit

Posted on

Data-Oriented programming: Liberate yourself from objects

The main lesson that Clojure teaches its developers is to embrace simplicity.

The secret sauce used by Clojure to embrace simplicity is that it treats data as a first-class citizen that is manipulated with generic functions.

In the Clojure community, we call this secret sauce: Data-Oriented programming.

The good news is that this paradigm is in fact language agnostic, meaning that it is applicable in other programming languages, both OOP and FP, both statically and dynamically typed.

In Data-Oriented programming, we follow 3 principles:

  1. We separate our code from our data
  2. We represent data with generic data structures
  3. We never mutate our data

What does it mean in practice for you to adhere to Data-Oriented programming?

  1. You write your code in stateless functions (or static methods)
  2. You represent your data with hash maps and vectors
  3. You use a library that implement efficient immutable data structures

What do you get from that?

In one word: SIMPLICITY!

In a few sentences:

  1. No more complex UML diagrams!
  2. You benefit from dozens of generic data manipulation functions
  3. You can visualize any part of your system, as it is just data
  4. You liberate your mind from concurrency-related issues.

Curious to see how it looks like in a real production system?

Start reading my book Data-Oriented programming.

Here is a 50% off discount code for you: mlsharvit2. Valid until March 15.

The principles of Data-Oriented programming are explored in greater detail here.

Enjoy!

Discussion (20)

Collapse
peerreynders profile image
peerreynders • Edited on

We never mutate our data
...
The good news is that this paradigm is in fact language agnostic.

Then apparently data-oriented programming has nothing to do with data-oriented design.

Data-Oriented Design came about to address the growing performance gap between CPU caches and conventional memory - Data-Oriented Design (Or Why You Might Be Shooting Yourself in The Foot With OOP), Noel Llopis, 2009.

CPU vs Memory Performance Gap
source

Richard Fabian's Data-Oriented Design goes into great detail to show how OO-style C++ completely negates the performance enhancing features of modern CPUs.

The most amazing achievement of the computer software industry is its continuing cancellation of the steady and staggering gains made by the computer hardware industry. — Henry Petroski

So as such DOD is about optimizing code to take advantage of modern hardware performance features such as generously sized CPU caches. So it is highly "place-oriented" (rather than "value-oriented") and embraces in-place mutation. It is incredibly important to collocate data that is processed and used together. So to practice DOD you have to have control over memory layout. High level languages don't let you control memory layout - systems languages like Rust do.

From that perspective something like ECSY is nothing but a curiosity because it is written in JavaScript.

Because of these requirements Unity created HPC# (High-Performance C# - a custom language based on a subset of C# with some adaptations from C/C++) for their "Data-Oriented Technology Stack" (DOTS).

Data Oriented Design Resources


PS: While your approach seems to borrow some of the organizational principles from Entities, Components, Systems (ECS) the games industry has already established some semantics around the "Data-Oriented" space for the past 10+ years (and it has been spreading from there) that don't align with your ideals. So it may make sense to shift to a different moniker like "data-centric" (though there the relation to data-centric architecture needs to be carefully considered).

Collapse
viebel profile image
Yehonathan Sharvit Author

The term data-oriented programming was coined by Eugene Kuznetsov in 2004. Here are the two references I found for it:
adtmag.com/articles/2004/08/05/dat... and
community.rti.com/sites/default/fi...

As the idiom says: naming is hard!

Collapse
peerreynders profile image
peerreynders

The term data-oriented programming was coined by Eugene Kuznetsov in 2004.

That's fine but a quick google will demonstrate that the "other meaning" has been gaining a lot more momentum in the intervening years and with Unity pushing its Data-Oriented Technology Stack (DOTS) (largely guided by Mike Acton's work) it's that much closer to gaining mainstream acceptance.

Now within the Clojure community Data Oriented has been used as a synonym to Rich Hickey's 'value-oriented' — for example the 2014 Joy of Clojure and later in Paul deGrandis's 2017 Speaking Data: Simple, Functional Programming with Clojure.

However this book's stated objective goes beyond the confines of Clojure:

You’ll learn to write DOP code that can be implemented in languages like JavaScript, Ruby, Python, Clojure and also in traditional OO languages like Java or C#.

i.e. more mainstream — and it's in this arena where the other meaning of DO is already that much further ahead.

Most recently (2021 August) this made an appearance:

A Data-Centric Introduction to Computing

Their definition:

We view a data-centric curriculum as
data centric = data science + data structures

In the 2004 paper The Structure and Interpretation of the Computer Science Curriculum both Shriram Krishnamurthi and Matthias Felleisen (among others) advocated for a "Value-Oriented Programming"-first computer science education based on their experience with How to Design Programs. They didn't explicitly call it that because it's the defacto style of programming in Scheme.

Eventually though they had to concede that the Scheme-like student languages didn't work for some students. This lead to the development of Pyret.

Pyret doesn't look anything like Scheme — it deliberately borrows heavily from Python — but Python doesn't have an ingrained practice of "Value-Oriented Programming" (I argue quite the opposite).

So it's at this point that they decided that "Value-Oriented Programming" needed a name and that in the contemporary climate Data Centric made the most sense (leaning a bit into the hype around data science).

Thread Thread
viebel profile image
Yehonathan Sharvit Author

Naming is definitely hard!

Collapse
brunooliveira profile image
Bruno Oliveira

Bought your book, can't wait for it to be completed 😁 the current ~75 pages from the Manning MEAP are amazing, but I need moreeeee😄😄 ideally it'll be the kind of book I'll get back to many many times in the future, just like with Grokking Simplicity

Collapse
viebel profile image
Yehonathan Sharvit Author

Spread the word about the book among your friends👩‍💻.
Tweet about it.

The world really need to get out of this insane 😱 complexity!

Collapse
peppesilletti_4 profile image
Giuseppe Silletti

Thanks for sharing :) When do you think a functional approach should be used instead of an OOP approach?

Collapse
viebel profile image
Yehonathan Sharvit Author

Always🤗 !

Collapse
peppesilletti_4 profile image
Giuseppe Silletti

That's not convincing :) Could you tell me a bit more about why you think that?

Thread Thread
viebel profile image
Yehonathan Sharvit Author

Read the book and you'll be convinced 😁!

Thread Thread
brunooliveira profile image
Bruno Oliveira

Hmm, seeing this type of reply makes me a bit sad :)

I think it's fine to advocate for the book, which is actually quite interesting so far, but, most working programmers will read it with the perspective of gaining new insights and/or better leveraging the functional style tools available in their working language, and not to "abandon" objects in any way, simply because it's not possible. Only a TINY fraction of us can use functional programming on our daily job or switch technologies completely.

I think most readers will approach this book with the idea of gaining new insights to apply to their daily work or simply to gain a fresh perspective.

There is no silver bullet, and if there is, it certainly won't be functional programming.... I mean the foundations have been laid out more than 50 or 60 years ago, so if it didn't take off into the mainstream industry in that time frame, it most likely won't now :)

Thread Thread
viebel profile image
Yehonathan Sharvit Author

Actually, in the book I show how to apply the principles of Data-Oriented programming to both FP or OOP languages.

Collapse
belgoros profile image
Serguei Cambour

In short - switch to Elixir :)))

Collapse
viebel profile image
Yehonathan Sharvit Author

Why Elixir and not Clojure?

Collapse
peerreynders profile image
peerreynders

Clojure works great when you need to integrate with legacy enterprise systems.

But if you need a scalable, fault-tolerant (i.e. resilient) and highly concurrent platform with a functional language that includes primitives to support those objectives on the language-level then Erlang/Elixir is a better fit.

And frankly there is always Jinterface

The Erlang Rationale

Thread Thread
marciol profile image
Marcio Lopes de Faria

Hi @peerreynders , it'd be nice if Idiomatic Elixir makes use of more Maps and Lists and less use of data encapsulation on Structs so that we can exchange maps and lists freely and make the quote "100 functions over one data structures" more present in Elixir than "10 functions over 10 data structures" that it's what happens with the overuse of Structs.

Collapse
bias profile image
Tobias Nickel

data oriented programming. I was looking for a word for that.

"let data be data"

Collapse
viebel profile image
Yehonathan Sharvit Author

I wrote a song about that: blog.klipse.tech/databook/2021/02/...

Collapse
macsikora profile image
Pragmatic Maciej

Interesting trick, as too many books have functional programming in name, you made the name different - data oriented programming, what looks different, where in real ain't ;)

Collapse
viebel profile image
Yehonathan Sharvit Author

If you read the book you'll learn what is the difference between DOP and FP.