The case for statechart and xstate -- why it matters and how we can benefit from it

Article originally posted here

Due to popular requests I'm currently providing remote training and consulting services to help your team pick up statecharts-based development approach and adopting it in the product real quick, guaranteed to make the devs work smarter, faster, safer, happier and bring more value to the company, contact me for details

Bottom line up front

• redux is a global data management tool, not a proper state management tool, hence causing a lot of troubles

• statecharts is an extension to Finite State Machine (FSM) which provides explicit and safe state management capabilities, perfectly fit for front-end development

• statecharts had been used intensivelly in all industries (be it embedded systems, hardware, electronics, aeronautics, automotive, game development and more), it's us front-end developers late to the party

• statecharts will be the next major paradigm shift that changes how people work on front-end development and it's coming in real quick

  • react taught us thinking in component
  • redux taught us unidirectional flow and data transition
  • statecharts will teach us thinking in state

• statecharts make development time at least 50% shorter and the product 200% robust along with a bunch of other benefits

• in short, statecharts saves us time, effort and budget and it will soon become the main stream approach for front-end development

Note: fsm, finite state machine, statecharts and xstate were used interchangeably in this doc, for that they more or less means the same thing but fsm is shorter to write.

What's wrong with our current approach?

• we are doing it the bottom-up way

  • by using a lot of if/else clause and isLoading, isLogggedIn, isEditable flags to piece together the program and trying to maintain the app in correct state which is never possible
  • in essence when building the program we're implicitly writing state machines all the time, just in the form of code, and making it extremely error-prone and hard to debug let alone testing

• this bottom-up approach is unreliable because

  • events are scattered across the codebase and have unpredictable effects, it often grows out of control pretty quick
  • as the amount of state variables grow we will eventually lose count of all the possible combinations of program states and that's when bugs happen

• worse yet, code written this way will be very hard to debug because the business logic is spread across multiple places and they intertwine with each other which make the situation even worser, let alone trying to come back and fix bugs six months from now and we all have been there before

Why react/redux is not enough?

• redux is a data container

  • which provides a global data storage container and formal methods to update the content (via actionCreator and reducer)

• but it can not guarantee the program from going into wrong state

  • because users could dispatch actions in any state at any time, which might brings the program into wrong or unknown state and starts to behave unexpectedly

• hence redux is more like a data management or data flow management tool instead of a proper state management

• in short, people are confusing the data aspect of redux with true state management

  • and that's the root cause for a lot of problems we are facing up with everyday with front-end development
  • which also cost us tremendous both in terms of time and money
    • late to market
    • unepected and hard to fix bugs which makes customers unhappy

What we really need is state management, not data management

• every program has different business logics and certain contexts they were running in

• these contexts and business logics details should not be solely controlled by a global data storage like redux

• in stead, fsm provides a clear way of managing complex states and the transition between them, this is the major difference between redux and fsm

What does a fsm do?

• define all possible states the UI could be in up front

• define what exactly could happen in each of the states

• define transition between states when certain events happen

• define what actions need to be taken with each transition

• define what side effect shall be triggered by those actions

Why fsm is better?

• fsm eliminates weird bugs and prevent wrong application states from happening because it won't let the UI transition to state which we don't know about.

• fsm does not accept input which is not explicitly defined as acceptable for the current state. This eliminates the need of code that protects other code from execution.

• fsm forces the developer to think about overall program states in advanced and declaratively define them up front, which help to focus on the core business logic

• fsm (with the visualization chart) is easy to understand by all stakeholders like Project Manager, designers and developers (think of it as the single source of truth and final definitive doc of the project)

• fsm gives us the power to reason about complex UI flows and at the same time prevents unexpected behavior to happen

Technical benefits

• encode business logic inside FSM

  • that can be tested separately
  • also means business logic can be brought anywhere, instead of being locked inside any given UI framework
  • which means the business logic is no longer limited to one platform or ui framework, it can be freely transferred to any other future ui technologies (AR, VR, terminal, chatbot...)
  • think of it as the insurance policy for the most important logic in the long run

• which also encourages working on both writing code and testing in parallel

  • by implementing new feature and writing tests at the same time to verify the feature is actually working
  • tests are no longer an after thought, instead it become a first class citizen and covers the whole program, at all times

• declarativelt listing of states, events, actions and side effects in fsm

  • most of these are pure functions (without side effects)
  • which makes them extremely easy to write and debug

• making codebase much easier to reason about

  • for that core logic is declaratively listed and encoded within fsm
  • and the structure of fsm is easy to read, understand, trace and modify

• FSM and UI are separately developed

  • focus on modeling the application state with fsm first
    • which forces developers to thinking through how the program should work before writing first line of code
  • then implement events, actions and side effects for each fsm state
  • ans all these can be tested by writing test at the same time
  • hence when the implementation is completed, all tests are completed too

• making UI a dumb layer

  • because business logics are declaratively encoded and implemented in fsm, there's nothing much to do in the ui layer
  • UI has only two purpose then:
    • translate user input into events and send it to the fsm
    • re-render the view whenever fsm state changes
  • in short, we are basically making ui irrelevant, and switching between different UI frameworks will be a snap

• gone are the days of re-writing the app every two years

  • because core business logic is encoded inside fsm
    • which can be brought anywhere and reused in the future
  • only the ui layer needs rewrite (when there are better ui options out there)

• program becomes highly flexible

  • because it's easy to test new ideas by moving things around inside fsm without worrying about breaking a lot of things
  • hence it's faster to meet requirement changes (which is a norm in a startup environment)

• making developers less stressed

  • for that there is much fewer things to worry about with this fsm approach
  • everything can be easily modeled through the fsm (and it's various actions and events)

• fsm can be used anywhere, not just front-end

  • as long as the program has states (which means every single one)
  • front-end, back-end, embedded system...
  • and it can be implemented in any programing language, no need for a framework or library at all
  • hence there's no vendor lock-in to worry about

• fsm is the doc

  • because core business logic and user flow are encoded in it, and it drives the implementation, hence making it always up to date
  • gone are the days that docs are always outdated and no one ever wanted to maintain it and making it a lie waiting to happen

Overall benefits

• saving time and money

  • because the project can be completed faster by having states and ui developed in parallel saves a lot of time)
  • having fewer bugs because fsm always ensures the program runs in correct states
  • this approach encourages developers to write tests and code at the same time which further ensures the correctlness
  • in the long run fsm will save the company a lot of time and money, and being able to release robust products that make customers happy (even by simply reducing the number of bugs will help)

• requires lower technical skills hence easier to hire talents

  • no struggle to hire senior developers with deep knowledge in certain frameworks
    • it's hard or nearly impossible to hire those devs anyway
    • as long as the devs know the basics (javascript, DOM), we can train them in no time
  • newbies can do the job as long as they know how basic ui framework works
  • it's also easier to train the team for that fsm is very easy to pick up
  • and with less technical requirements on the UI framework, switching between new framework or library is easy too
    • for example, no need to learn a bunch of 3rd party libs like redux-saga, rxjs, thunk, observables..., jut the plain basic javascript will do the trick

• fsm can be used as a single source of truth for communication between all stakeholders

  • because it has a built-in visualizer to show the overall program structure, how the program should work, hence there's one single source of truth that all stakeholders could refer to at any time, this saves a lot of communication costs
  • hence making the fsm visualization a great tool be used by Project Manager, designers and developers

• faster development cycle, more robust product

  • dev speed will be much faster for that the code complexity was simplified by the declarative nature of fsm
  • and writing tests and implementation at the same time also saves a lot of manual testing time
  • it's also easier to debug the program when new bug appears due to the problem surface will be limited and could be quickly isolated into certain state and side effects, hence less time spent on tracing and fixing bugs
  • and because the program is precisely controlled by the fsm it's rare for it to go wrong, which makes the product extremely robust (which brings in happy customers)

Tooling

• xstate is the most mature statecharts library written for web development

  • it had been in active development for more than 4 years by brilliant developers
  • major contributor David Khourshid works for Microsoft
  • https://github.com/davidkpiano/xstate
  • https://xstate.js.org/docs/

• xstate comes with a visualizer

  • https://xstate.js.org/viz/
  • which makes reasoning about the program structure much easier
  • better yet, the visualizer is interactive so one can actually poke around and see how the state transitions
  • 

• no need to use library at all

  • one can easily implement basic fsm with switch/case which is available in all programming languages
  • but xstate comes with full support of the statecharts specification, which includes hierarchical, parallel and history states macahines and more, hence it's highly recommended

• in the future fsm could be compiled to WebAssembly (wasm) and runs across many platforms (including browser), this way the same business logic can be truly re-used across platforms

Sample case

• Apple Facetime bug can be avoided with this approach

  • the bug was user can hear other people's voice before they even pick up, it's an obvious fault on state management but very difficult to found before hand, bugs like this could be easily prevented with fsm
  • see details here
    • http://bit.ly/2PYBTl5

• NASA used statecharts to design the Curious Mars mission

  • http://bit.ly/2TXiwcc

Academic research

• first paper on FSM released in 1949

  • The Mathematical Theory of Communication (W. Weaver)
  • that's 70+ years ago

• first paper on statecharts released in 1987

  • A visual formalism for complex systems (David Harel)
  • 30+ years ago

• It had been deeply researched in academy for the past 30 years

  • even has a official W3C specification named SCXML
  • https://en.wikipedia.org/wiki/SCXML

Difference between fsm and statecharts

• fsm was released in 1949

  • main drawback was it can not handle hierarchical and parallel states which caused the state explosion issue (too many states making the fsm chart difficult to understand and work with)

• statecharts was released in 1987

  • it's based on fsm
  • added hierarchical, parallel, history, extended context, action and services
  • which scales pretty well and can be used to model all business requirements

Abundant resources

• statecharts

  • https://statecharts.github.io/

• books

  • Constructing the User Interface with Statecharts
    • Ian Horrocks (1999)
  • Press On: Principles of Interaction Programing
    • Harold Thimbleby (2007)
    • MIT Press

• talks

  • Indefinitely better ui
    • https://m.youtube.com/watch?v=VU1NKX6Qkxc
  • Simplifying Complex UIs with Finite Automata & Statecharts
    • https://m.youtube.com/watch?v=RqTxtOXcv8Y
  • Reactive State Machines and Statecharts
    • https://m.youtube.com/watch?v=DrHccvns-L0
  • Using statecharts for complex ui (with react native)
    • https://www.facebook.com/fraktio/videos/780228525664779/?t=4263
  • Adaptive and intelligent UIs in React (2019)
    • https://www.youtube.com/embed/gDkUpx0dVc0?t=4969

• articles

  • You are managing state? Think twice
    • http://krasimirtsonev.com/blog/article/managing-state-in-javascript-with-state-machines-stent
  • The Rise Of The State Machines
    • https://www.smashingmagazine.com/2018/01/rise-state-machines/
  • Are statecharts the next big UI paradigm?
    • https://www.slideshare.net/lmatteis/are-statecharts-the-next-big-ui-paradigm
  • How I Learned to Stop Worrying and ❤️ the State Machine
    • http://bit.ly/2TXmyRS
  • Formally Specifying UIs
    • https://www.hillelwayne.com/post/formally-specifying-uis/
  • Tackling UI complexity with State Machines
    • http://bit.ly/2TUvsiY
  • Modeling a Screensaver with a Statechart, a real use case
    • http://bit.ly/2Vjw4Az
  • How to model the behavior of Redux apps using statecharts
    • http://bit.ly/2WC8GP9

Downside of FSM

• requires shifting of metal model to pick it up

  • just like shifting from jquery to react, it takes some getting used to and switching the brain around, a.k.a the "give it five minute" moment to grok it

• the way to write code needs to be changed too

  • but in a good way, no need for redux and other libraries
  • just learn basic javascript and react will do the trick
  • focus will be put on modeling the program state in fsm and implement actions and services for it, which is much less complex than juggling between a bunch of redux/thunk/saga/observables libraries

• project seems to be moving slower in the beginning (which is not true)

  • because developers had to spend more time on up front modeling the program state instead of jumping into writing code
  • but in fact the project is just slower at the beginning and it will make up the speed faster laster in the cycle and eventually be completed quicker then the traditional approach (and with fewer bugs, more correctness, up-to-date docs and full test coverage)

• requires time to learn new tools and skills

  • learn modeling program with statecharts
  • learn to work with xstate (it's various APIs)
  • learn to integrate xstate and react
  • in my experience this could be done in a few days, and once past that period everything will be different (in a good way)

• migrating legacy applications to fsm requires time and patience

  • good news is most redux logics (actionCreators, reducer, side-effects) can be re-used with minimal rewrite
  • and the migration could be implemented on a component basis
  • meaning writing local fsm for each component and make sure it works
  • then connect the local fsm to a larger/parent one, repeat the process till all components are converted to fsm

What I have done so far in the area

• gaining thorough understanding of fsm/statecharts and xstate

  • by reading all papers, books, talks and articles
    • see a list of the most important ones listed above

• built real-world examples that demonstrates different use cases of statecharts (include CRUD, multi-threads...)

• established a formal dev approach to work with fsm and ui

  • core idea is to model the state first and encapsulate business logic inside then work on the UI development
  • parallel progression on writing code and tests to save time and ensure program correctness
  • this approach is demonstrated in the examples I provided above
  • it has been verified to efficiently shorten the dev time and produce much less bugs
  • it could be learned by any developer in a couple of days

How I can help your team with

• help the team to pick the skills

  • learn how to model program state with statecharts
  • learn to efficiently work with fsm and ui with the formal approach I developed

• I could provide courses, training materials, code examples and answer questions to the team

About the Author

• Jeremy Lu (he/him, fluent in English and Chinese)

• 18+ years of software development experience with a focus on functional programming (haskell, rust, javascript, react)

• Creator of Stateskit, a free web-based state modeling tool to help people quickly dump ideas in the head to formal statechart and collaborate with all stakeholders (clients, managers, designers, developers) from there.

• Currently I provide professional training and consulting services which helps bringing the team up to speed for learning the statechart-based development approach and adopting it in the product real quick

• I can help your team work smarter, faster, safer and happier to bring more value in the long run with this approach, contact me for details

• fsm@twmug.com | @stateskit