DEV Community

Andrey Frolov
Andrey Frolov

Posted on

How We Migrated from Javascript and Flow to TypeScript at Osome

This fairy tale started a long time ago. Often, you make a bet on the wrong horse in life, but, unfortunately, you cannot do anything about it. You're always working with limited information (especially about the future), when making decisions, so essentially you are betting. This is life: sometimes it happens in your personal life and sometimes in your work life. But the only thing that really matters is how you learn from your bets.

One such decision was betting on Flow. If you aren’t aware of what Flow is, it is Facebook's JavaScript language superset that brings types and type system to JavaScript. The idea is pretty straightforward: write your code with a kind of syntactic sugar Flow that takes your files and builds an extended abstract syntax tree (AST), analyze it, and if everything works, transform it into regular JavaScript.

Although, we started developing our apps in 2017/2018, when TypeScript tooling, support, and documentation were not as strong as it is now, and there was no clear winner between Flow and TypeScript. The developers who started it, were more familiar with Flow. The turning point was around 2019, when TypeScript started developing much faster, and Flow was essentially abandoned and we too decided that we should migrate to TypeScript at some point. Flow users will be faced with the following problems:

  • Undocumented features, that you can only know by diving into Facebook's codebase or Flow source code
  • Slow compilation process, that blows out your computer
  • Substantially weak support from the maintainers, regarding your issues and problems
  • Abandoned userland typing's for npm packages

This list can go on and on but within Y amount of time, we got the idea that we should transition into another superset of JavaScript. Well, if you ask me why, there are a few reasons, which can be divided into two subsections:

Company-specific reasons:

  • We already utilized TypeScript on the backend and have the required expertise and experience.
  • We already had contract-first backend programming. Where SDKs for different clients are automatically generated based on specification.
  • A significant part of our frontend is CRUD-like interfaces, based on backend API. Therefore, we want to use SDK for our frontends with strong typings.

Common sense-based reasons:

  • Great documentation, many tutorials, books, videos, and other stuff.
  • Weak type system. It is debatable because it's not like ML typesystem (Reason, Haskell, Ocaml), so it easily provides a clear understanding to people from any background.
  • Great adoption by the community, most of the packages you can imagine have great TypeScript support.

How has it started?

As far as I know, migration was started by adding tsconfig.json with:



{
  // ...
  "compilerOptions": {
    "allowJs": true
  }
}


Enter fullscreen mode Exit fullscreen mode

This TypeScript flag allowed us to start adding ts and tsx files to our system.

Following this, we began to enable ts rules one after the one (the options below are just for example — choose those which fits to your codebase)

Image description

But we already had Flow, so let's fix it and add to .flowconfig



module.name_mapper.extension='ts' -> 'empty/object'
module.name_mapper.extension='tsx' -> 'empty/object'


Enter fullscreen mode Exit fullscreen mode

Now, you are ready to start the migration. Let's go! Wait, it's that easy?

Where did we end up?

Due to a variety of reasons, we ended up hanging in limbo. We had plenty of files in Flow and in TypeScript. It took us several months to understand that it was not okay.

One of the main problems here is that we didn't have a plan as to which files we should start with, how much time and capacity it should take, and a combination of other organizational and engineering-based issues.

Another day, another try

After accepting the fact that the health of our codebase is not okay, we ultimately settled on trying again.

This is one more time when graphs save the day.  Take a look at this picture:

image

Any codebase is no more than a graph. So, the idea is simple: start your migration from the leaves of the graph and move towards the top of the graph. In our case, it was styled components and redux layer logic.

image

Imagine you have sum function, which does not have any dependencies:



function sum(val1, val2) {
   return val1 + val2;
} 


Enter fullscreen mode Exit fullscreen mode

After converting to TypeScript, you get:



function sum(val1: number, val2: number) {
   return val1 + val2;
} 


Enter fullscreen mode Exit fullscreen mode

For example, if you have a simple component written in JavaScript when you convert that specific component, you immediately receive the benefits from TypeScript.



// imagine we continue the migration and convert the file form jsx to tsx
import React from 'react'

// PROFIT! Here's the error 
const MySimpleAdder = ({val1}) => <div>{sum("hey", "buddy")}</div>

// => Argument of type 'string' is not assignable to parameter of type 'number'.


Enter fullscreen mode Exit fullscreen mode

So, let's create another picture but for your current project.

The first step is to install dependency-cruiser.

Use CLI



depcruise --init


Enter fullscreen mode Exit fullscreen mode

Or add .dependency-cruiser.js config manually. I give you an example that I used for a high level overview of such migration.



module.exports = {
  forbidden: [],
  options: {
    doNotFollow: {
      path: 'node_modules',
      dependencyTypes: ['npm', 'npm-dev', 'npm-optional', 'npm-peer', 'npm-bundled', 'npm-no-pkg'],
    },
    exclude: {
      path: '^(coverage|src/img|src/scss|node_modules)',
      dynamic: true,
    },
    includeOnly: '^(src|bin)',
    tsPreCompilationDeps: true,
    tsConfig: {
      fileName: 'tsconfig.json',
    },
    webpackConfig: {
      fileName: 'webpack.config.js',
    },
    enhancedResolveOptions: {
      exportsFields: ['exports'],
      conditionNames: ['import', 'require', 'node', 'default'],
    },
    reporterOptions: {
      dot: {
        collapsePattern: 'node_modules/[^/]+',
        theme: {
          graph: {
            rankdir: 'TD',
          },
          modules: [
            ...modules,
          ],
        },
      },
      archi: {
        collapsePattern:
          '^(src/library|src/styles|src/env|src/helper|src/api|src/[^/]+/[^/]+)',
      },
    },
  },
};


Enter fullscreen mode Exit fullscreen mode

Play with it — the tool has a lot of prepared presets and great documentation.

Tools

Flow to ts — helps to produce raw migration from Flow to TS files and reduces monkey job.
TS migrate — helps convert JS files to Typescript files. A great introduction can be found here.

Analyze and measure

This was not my idea but we came up with the idea of tracking the progress of our migration. In such situations, gamification works like a charm, as you can provide transparency to other teams and stakeholders, and it helps you in measuring the success of the migration.

Image description

Let's dive into technical aspects of this topic, we've just used manually updated excel table and bash script that calculates JavaScript/Typescript files count on every push to the main branch and posts the results to the Slack channel. Here's a basic sample, but you can configure it for your needs:



#!/bin/bash

REMOVED_JS_FILES=$(git diff --diff-filter=D --name-only HEAD^..HEAD -- '.js' '.jsx')
REMOVED_JS_FILES_COUNT=$(git diff --diff-filter=D --name-only HEAD^..HEAD -- '.js' '.jsx'| wc -l)

echo $REMOVED_JS_FILES
echo "${REMOVED_JS_FILES_COUNT//[[:blank:]]/}"

if [ "${REMOVED_JS_FILES_COUNT//[[:blank:]]/}" == "0" ]; then
echo "No js files removed"
exit 0;
fi

JS_FILES_WITHOUT_TESTS=$(find ./src -name ".js" ! -wholename "tests" -print | wc -l)
JS_FILES_TESTS_ONLY=$(find ./src -name ".js" -wholename "tests" -print | wc -l)
TOTAL_JS_FILES=$(find ./src -name ".js" ! -wholename "*.snap" -print | wc -l)

JS_SUMMARY="Total js: ${TOTAL_JS_FILES//[[:blank:]]/}. ${JS_FILES_WITHOUT_TESTS//[[:blank:]]/} JS(X) files left (+${JS_FILES_TESTS_ONLY//[[:blank:]]/} tests files)"

PLOT_LINK="<excellink>"

echo $JS_SUMMARY

AUTHOR=$(git log -1 --pretty=format:'%an')
MESSAGE=":rocket: ULTIMATE KUDOS to :heart:$AUTHOR:heart: for removing Legacy JS Files from AGENT Repo: :clap::clap::clap:\n```$REMOVED_JS_FILES```"

echo $MESSAGE
echo $AUTHOR

SLACK_WEBHOOK_URL="YOUR_WEBHOOK_URL"
SLACK_CHANNEL="YOUR_SLACK_CHANNEL"
curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$MESSAGE\n$JS_SUMMARY\n\n$PLOT_LINK\",\"channel\":\"$SLACK_CHANNEL\"}" $SLACK_WEBHOOK_URL

Enter fullscreen mode Exit fullscreen mode




Stop the bleeding of the codebase

Migration is a process; you can't do it in one day. People are fantastic, but at the same time, they have habits, they get tired, and so forth. That's why when you start a migration, people have inertia to do things the old way. So, you need to find a way to prevent the bleeding of your codebase when engineers continue to write JavaScript instead of Typescript. You can do so using automation tools. The first option is to write a script that checks that no new JavaScript files are being added to your codebase. The second one is to use the same idea as test coverage and use it for types.

There is a helpful tool that you can use for your needs.

Set up a threshold, add an npm script and you are ready to start:



"type-coverage": "typescript-coverage-report -s --threshold=88"

Enter fullscreen mode Exit fullscreen mode




Results

Today, there is no Flow or JS code in our codebase, and we are pleased with our bet to use TypeScript, successfully migrating more than 200k files of JavaScript to TypeScript.


Feel free to ask questions, express any opinions or concerns, or discuss your point of view. Share, subscribe, and make code, not war. ❤️

If you'll find an error, I'll be glad to fix it or learn from you — just let me know.

If you want to work with me you can apply here, DM me on Twitter or LinkedIn.

Top comments (20)

Collapse
 
rajeshroyal profile image
Rajesh Royal

I made the same design of moving all the projects to TypeScript when I joined, and I'm glad I made that decision. Although some of the developers still has a tendency to use any which makes TS no use 🙁

Collapse
 
frolovdev profile image
Andrey Frolov • Edited

And whats the problem with a small amount of any's in project?

Collapse
 
rajeshroyal profile image
Rajesh Royal

We had big problem. cannot desctructure X from undefined

Thread Thread
 
frolovdev profile image
Andrey Frolov • Edited

But the typescript type system is not Ocaml, Haskell, or Rust like; it doesn't give you any 100% that you don't get any runtime errors or null pointer errors :d

Yes, it prevents a huge surface of errors, but not as much as it looks in theory.

Thread Thread
 
rajeshroyal profile image
Rajesh Royal

agreed 💯

Collapse
 
frolovdev profile image
Andrey Frolov

Good point, but It's not so true, the development is not about black and white and good or bad, you always have semitones - tradeoffs. For us, it was an appropriate decision to make a sharp migration and start to check type coverage to reduce the amount of any's iteratively

Collapse
 
jwp profile image
John Peters

Sometimes our legacy prejudices blind us. For Javascripters, they rejected TypeScript from the start. That's why newfound issues were solved using various Javascript libraries none of which used the same dependencies or coding patterns. Alas NPM became a virtual cesspool as a result.

TypeScript has its roots in over 35 years of other languages such as C#, Java and even Pascal. Many Javascripters and large companies see the value now. Demand is only going to grow. Being a part of that need is a great place to be right now.

Collapse
 
frolovdev profile image
Andrey Frolov

Thank you for your comment. But what do you mean by demand right here?

Collapse
 
jwp profile image
John Peters

TypeScript is becoming very popular in large companies.

Thread Thread
 
frolovdev profile image
Andrey Frolov

de facto standard :d

Collapse
 
drmnk profile image
Denis Romanenko

Thanks God it was not Dart. Canonical still doesn’t have strength to admit that they bet on the wrong horse…

Collapse
 
insidewhy profile image
insidewhy

Using Dart on a flutter project at work. So much disappointment, so much boilerplate, so much having to give up on doing things because the type system is too basic.

Collapse
 
frolovdev profile image
Andrey Frolov

Interesting point. What do you mean by too basic?

Collapse
 
frolovdev profile image
Andrey Frolov

Nowadays most people die of a sort of creeping common sense, and discover when it is too late that the only things one never regrets are one's mistakes :d

Collapse
 
konnikiforov profile image
KonNikiforov

Thx, get some insights for my current project!

Collapse
 
frolovdev profile image
Andrey Frolov

appreciate it

Collapse
 
artemekaterinenko profile image
artemekaterinenko

Great stuff! This will save us a lot of time for the future migration

Collapse
 
frolovdev profile image
Andrey Frolov

Thank you mate

Collapse
 
funsis profile image
Sergey Korovin

Great article, found some interesting ideas for me

Collapse
 
frolovdev profile image
Andrey Frolov

My pleasure