Creating a Simple OCR Application with Electron, Vue.js & Tesseract.js

Originally published on my blog

Nowadays JavasScript is one of the most popular programming languages, and used for a lot of fields and platforms through Web. In this tutorial we're going to learn how to build an OCR desktop application with javascript using Electron and Tesseract.js ... and guess what... our FE will be implemented with Vue.js

More than one year ago, I meet electron.js, from the beginning to me, was an awesome library... After all, create a desktop application with JavaScript is a lovely superpower... doesn't it? So I started my learning with a simple project that I called triton-forms, basically is a dynamic form engine built with AngularJS, you can meet it right here.
This week, I decided return to my electron adventures and create another application, this time with Vue.js.

The OCR Project

Well as you can imagine, when I started this article (and now) the main goal was build something cool with electron and learn on the road, but the "what" we are going to build, in that moment wasn't clear. So I expended a couple of days reading and thinking a simple but cool exercise to do.

Yesterday like a heaven's signal, I found my brother typing on his laptop something for his homework from a printed document, so I told him... "What are you doing dude? you can do that pretty easily and fast with an OCR". That last word stayed like an echo on my mind and then I knew it... "I must create an OCR application".

The idea is pretty simple, basically we are going to have an application with a drop area, in that area we are going to drop the image file to process, then the text is extracted and displayed to user. Sounds good... right? so let's started!

Electron - Getting Started

Electron combine the power of Node.js and a dedicated Chromium Web browser instance in order to run Web/JavaScript applications like desktop application, that's the reason why we are going to use it as container for our Vue.js Web application, so let's started configuring our electron project!

Our project in essence is a Node.js based application so, first of all , we're going to create a new npm project in a new directory. With the -y parameter the package.json will be created with default values:

$ npm init -y

And add the electron dev dependency to the project

$ npm install --save-dev electron

Then in our package.json file, add the following lines:

  "main":"index.js",
  "scripts": {
    "start": "electron ."
  },

This allow us running our electron application with just npm start command

Finally in order to complete our electron setup let's create the index.js file, and there we are going to create the basic script to show the https://lexmartinez.com/ Web site contents. So, in our index.js we should have the following lines:

const electron = require('electron');
const app = electron.app;
const BrowserWindow = electron.BrowserWindow;

let screen;

const renderApp = () => {
 // create the browser window
 screen = new BrowserWindow()
 // render the required website/entrypoint
 screen.loadURL('https://lexmartinez.com/')

 // dereference the screen object when the window is closed
 screen.on('closed', () => {
  screen = null;
 });
}

// call the renderApp() method when Electron has finished initializing
app.on('ready', renderApp);

// when all windows are closed, quit the application on Windows/Linux
app.on('window-all-closed', () => {
 // only quit the application on OS X if the user hits cmd + q
 if (process.platform !== 'darwin') {
  app.quit();
 }
});

app.on('activate', () => {
 // re-create the screen if the dock icon is clicked in OS X and no other
 // windows were open
 if (screen === null) {
  renderApp();
 }
});

And we are going to get this result ...

Vue-lize it!!

As I announced above, we are going to use Vue.js for all front-end stuff, so the first step, will be setup webpack in order to bundle our Vue.js Web Application and display it into our electron container.

To do that and configure our Vue.js we are going to use the vue-cli scaffolding:

# In case that you dont have the vue-cli installed yet
$ npm install -g vue-cli

$ vue init webpack app

# Use this answers for the wizard
? Project name app
? Project description xxxxxxxxxx
? Author xxxxxx <xxxxxx@domain.com>
? Vue build standalone
? Install vue-router? Yes
? Use ESLint to lint your code? Yes
? Pick an ESLint preset Standard
? Set up unit tests No
? Setup e2e tests with Nightwatch? No

That will create a new Vue.js project into the app folder, the next step would be merge our Vue.js and electron projects, We need to do the following four steps:

• We have to combine our package.json files, copying the Vue.js dependencies and engines setup to electron project.
• Remove the README.md, .gitignore and package.json files from app folder
• Move the rest of app folder contents to the electron project root (including .xxx files like .babelrc and so on)
• Finally, update the package.json scripts section like this:

"scripts": {
    "start": "electron .",
    "dev": "webpack-dev-server --inline --progress --config build/webpack.dev.conf.js",
    "lint": "eslint --ext .js,.vue src",
    "build": "node build/build.js"
  }

Now you can delete the app folder and test the new setup with npm run dev command and navigate to http://localhost:8080

At this point we got the electron and Vue.js setup working separately, let's put all together, the idea is run the webpack-dev-server with our Vue.js application contained on electron with just one command for that we are going to use concurrently package:

$ npm install concurrently --save-dev

And then update the npm start command with this :

  "start": "concurrently --kill-others \"npm run dev\" \"electron .\"",

Finally update the index.js file in order to wait a bit to webpack-dev-server, let's rewrite the on ready handler:

// The loaded URL must be changed too
 screen.loadURL('http://localhost:8080/')

app.on('ready', ()=>{
 setTimeout(renderApp, 3000);
});

This is just for development purposes, this setTimeout should be removed when we are going to package the electron application

And this is the result ...

Now our workspace will be src directory, the first thing that we are going to do is refactor the default HelloWorld component, let's call it OCR.vue, then we must fix the imports on our routing file and remove the unused assets (Vue.js logo) from assets directory and App.vue file.

Component Template

Our OCR component template will be divided in three panels: The image file input panel with drag-drop zone in order to select the target file, progress panel with process status updates and results panel with the extracted text. Additionally we are going to use Vuetify as our application look n' feel:

$ npm install vuetify --save

Then in our main.js file:

import Vue from 'vue'
import Vuetify from 'vuetify'
import('vuetify/dist/vuetify.min.css')

Vue.use(Vuetify)

And finally using the Vuetify components this will be our OCR.vue component layout:

<template>
  <v-app id="inspire" dark>
    <v-toolbar app fixed clipped-left>
      <v-toolbar-title>Simple OCR</v-toolbar-title>
      <v-spacer></v-spacer>
      <span v-if="isSuccess || isFailed">
      <v-btn icon @click="reset">
        <v-icon>refresh</v-icon>
      </v-btn>
      <v-btn icon @click="save">
        <v-icon>save</v-icon>
      </v-btn>
      <v-btn icon @click="drive">
        <v-icon>file_upload</v-icon>
      </v-btn></span>
    </v-toolbar>
    <v-content>
      <v-container fluid fill-height>
        <v-layout justify-center align-center>
          <div class="container" v-if="isInitial">
            <form enctype="multipart/form-data" novalidate>
              <h1>Upload image</h1>
              <div class="dropbox">
                <input type="file" :name="'document'" :disabled="isSaving" @change="filesChange($event.target.files);" accept="image/*" class="input-file">
                <p v-if="isInitial">
                  Drag your file here to begin<br> or click to browse
                </p>
              </div>
            </form>
          </div>
          <div class="container text-xs-center" v-if="isSaving">
            <v-progress-circular v-bind:size="200" v-bind:width="15" v-bind:rotate="-90"
              v-bind:value="(status.progress * 100)" color="primary">
                  {{progress}}
            </v-progress-circular>
            <h2>{{status.status}}</h2>
          </div>
            <v-layout row wrap v-if="isSuccess || isFailed">
            <v-flex xs12>
              <v-divider></v-divider>
              <v-text-field label="Result" v-model="status.text" counter full-width multi-line single-line :auto-grow="true"></v-text-field>
            </v-flex>
            </v-layout>
        </v-layout>
      </v-container>
    </v-content>
    <v-footer app fixed>
      <span>&copy; 2017 - Lex Martinez &lt;@lexmartinez&gt;</span>
    </v-footer>
  </v-app>
</template>

<style>
  .dropbox {
    outline: 2px dashed grey; /* the dash box */
    outline-offset: -10px;
    background: transparent;
    color: dimgray;
    padding: 10px 10px;
    min-height: 200px; /* minimum height */
    position: relative;
    cursor: pointer;
  }

  .input-file {
    opacity: 0; /* invisible but it's there! */
    width: 100%;
    height: 200px;
    position: absolute;
    cursor: pointer;
  }

  .dropbox:hover {
    background: rgba(255,255,255,0.1); /* when mouse over to the drop zone, change color */
  }

  .dropbox p {
    font-size: 1.2em;
    text-align: center;
    padding: 50px 0;
  }
</style>

In that fragment we are including the mentioned three panels with some flags in order to switch them when the OCR process status changes, here you can found more information about Vuetify components.

OCR engine

Other awesome Node.js library is Tesseract.js, which provides a complete but simple text detection framework with a few code lines we are going to create our OCR feature:

const Tesseract = require('tesseract.js');

Tesseract.recognize('/Users/john-doe/Desktop/text.jpg')
 .progress(function(packet){
     console.info(packet)
    })
 .then(function(result){
  console.log(result.text)
 })

The Tesseract.js dependency could be installed with this command npm install tesseract.js --save, also you're going to need the language traineddata file, which can be found here

Let's include that on our Vue.js component script: On methods section we are going to create a ocr function :

methods: {
    ocr: function (event) {
      Tesseract.workerOptions.workerPath = 'http://localhost:8080/static/worker.js'
      Tesseract.workerOptions.langPath = 'http://localhost:8080/static/'
      Tesseract.recognize(event)
        .progress((status) => {
          this.status = status
        })
        .then((result) => {
          this.currentStatus = STATUS_SUCCESS
          this.status = result
        }).catch((error) => {
          this.currentStatus = STATUS_FAILED
          this.status = error
        })
    },

As we see the final ocr function is not very different to our initial snippet, we just add a few lines in order to setup the languages path and worker path.

You can found the worker.js file in your node_modules/tesseractjs folder, we put it with the traineddata in the static folder in order to easy the handle and configuration

Uploading Behaviour

If you see carefully the template above we are using a few flags in order to switch the component panel, and other more functions in order to handle the user events. There the main behaviour is the upload task a.k.a filesChange function let's see the simple implementation of that:

filesChange (fileList) {

     if (!fileList.length) return

     this.currentStatus = STATUS_SAVING

     this.ocr(fileList[0])

   }

Hell Yeah!!! Our Simple OCR App is working 8-)

• Electron.js documentation could be found right here! and Tesseract.js docs here!
• Complete source code for this exercise could be found on this Github repo
• Thanks for reading! comments, suggestions and DMs are welcome!
• Finally, for me, this application seems to have potential, thus if you want contribute and improve this simple app I'm able to do that...