Takuya Matsuyama

Posted on Apr 4, 2019 • Edited on Feb 12, 2021

Hacking PouchDB to Use It on React Native

#reactnative #pouchdb #couchdb

UPDATE(2021/02/12)

Created PouchDB@7 for React Native

Takuya Matsuyama ・ Feb 12 '21

#pouchdb #reactnative

Hi, it's Takuya here.

I'm developing a React Native app which needs to sync data with CouchDB.
PouchDB, a well-designed database that can sync with CouchDB written in JavaScript for browsers, can be also used on React Native.
I have built react-native-sqlite-2 and pouchdb-adapter-react-native-sqlite to let it work on RN.

But there are still some issues to make it work.
For example, it has a big problem with storing attachments since RN doesn't support FileReader.readAsArrayBuffer yet (I hope they will support in the future though).

It seems like pouchdb-react-native project is inactive these days.
So I've been working on making PouchDB possible to run perfectly on RN and successfully did it. Here is what I did.

How to run PouchDB on RN

First, I would like to show you how to use PouchDB on RN.
I've made some packages that I hacked based on the official PouchDB core modules.
Here is a working demo app.
I am planning to use it in my production app called Inkdrop.

Install deps

Install PouchDB core packages:



npm i pouchdb-adapter-http pouchdb-mapreduce

And install hacked packages for React Native:



npm i @craftzdog/pouchdb-core-react-native @craftzdog/pouchdb-replication-react-native

Next, install SQLite3 engine modules:



npm i pouchdb-adapter-react-native-sqlite react-native-sqlite-2
react-native link react-native-sqlite-2

Then, install some packages to polyfill functions that PouchDB needs:



npm i base-64 events

Create polyfills

Make a js file to polyfill some functions that PouchDB needs:



import {decode, encode} from 'base-64'

if (!global.btoa) {
    global.btoa = encode;
}

if (!global.atob) {
    global.atob = decode;
}

// Avoid using node dependent modules
process.browser = true

Import it at the first line of your index.js.

Load PouchDB

Make pouchdb.js like so:



import PouchDB from '@craftzdog/pouchdb-core-react-native'
import HttpPouch from 'pouchdb-adapter-http'
import replication from '@craftzdog/pouchdb-replication-react-native'
import mapreduce from 'pouchdb-mapreduce'

import SQLite from 'react-native-sqlite-2'
import SQLiteAdapterFactory from 'pouchdb-adapter-react-native-sqlite'

const SQLiteAdapter = SQLiteAdapterFactory(SQLite)

export default PouchDB
  .plugin(HttpPouch)
  .plugin(replication)
  .plugin(mapreduce)
  .plugin(SQLiteAdapter)

If you need other plugins like pouchdb-find, just add them to it.

Use PouchDB

Then, use it as usual:



import PouchDB from './pouchdb'

function loadDB () {
  return new PouchDB('mydb.db', { adapter: 'react-native-sqlite' })
}

How I hacked PouchDB

To get it to work on React Native, we need to avoid calling FileReader.readAsArrayBuffer from PouchDB core modules.
That means we always process attachments in Base64 instead of Blob.
It can be done with a few lines of code hacks.

Where `readAsArrayBuffer` is called

PouchDB tries to calclulate MD5 digest for every document, which needs to call readAsArrayBuffer.

In pouchdb-binary-utils/lib/index-browser.js:



72 function readAsBinaryString(blob, callback) {
73   if (typeof FileReader === 'undefined') {
74     // fix for Firefox in a web worker
75     // https://bugzilla.mozilla.org/show_bug.cgi?id=901097
76     return callback(arrayBufferToBinaryString(
77       new FileReaderSync().readAsArrayBuffer(blob)));
78   }
79
80   var reader = new FileReader();
81   var hasBinaryString = typeof reader.readAsBinaryString === 'function';
82   reader.onloadend = function (e) {
83     var result = e.target.result || '';
84     if (hasBinaryString) {
85       return callback(result);
86     }
87     callback(arrayBufferToBinaryString(result));
88   };
89   if (hasBinaryString) {
90     reader.readAsBinaryString(blob);
91   } else {
92     reader.readAsArrayBuffer(blob);
93   }
94 }

This function is called from pouchdb-md5/lib/index-browser.js:



24 function appendBlob(buffer, blob, start, end, callback) {
25   if (start > 0 || end < blob.size) {
26     // only slice blob if we really need to
27     blob = sliceBlob(blob, start, end);
28   }
29   pouchdbBinaryUtils.readAsArrayBuffer(blob, function (arrayBuffer) {
30     buffer.append(arrayBuffer);
31     callback();
32   });
33 }

Well, how to avoid that?

Storing Attachments

Disable binary option of getAttachment method in pouchdb-core/src/adapter.js like so:



714     if (res.doc._attachments && res.doc._attachments[attachmentId]
715       opts.ctx = res.ctx;
716       // force it to read attachments in base64
717       opts.binary = false;
718       self._getAttachment(docId, attachmentId,
719                           res.doc._attachments[attachmentId], opts, callback);
720     } else {

With this change, you will always get attachments encoded in base64.

Pull Replication

We have to convert blob to base64 when fetching attachments from remote database in pouchdb-replication/lib/index.js like so:



function getDocAttachmentsFromTargetOrSource(target, src, doc) {
  var doCheckForLocalAttachments = pouchdbUtils.isRemote(src) && !pouchdbUtils.isRemote(target);
  var filenames = Object.keys(doc._attachments);

  function convertBlobToBase64(attachments) {
    return Promise.all(attachments.map(function (blob) {
      if (typeof blob === 'string') {
        return blob
      } else {
        return new Promise(function (resolve, reject) {
          var reader = new FileReader();
          reader.readAsDataURL(blob);
          reader.onloadend = function() {
            const uri = reader.result;
            const pos = uri.indexOf(',')
            const base64 = uri.substr(pos + 1)
            resolve(base64)
          }
        });
      }
    }));
  }

  if (!doCheckForLocalAttachments) {
    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  }

  return target.get(doc._id).then(function (localDoc) {
    return Promise.all(filenames.map(function (filename) {
      if (fileHasChanged(localDoc, doc, filename)) {
        return src.getAttachment(doc._id, filename);
      }

      return target.getAttachment(localDoc._id, filename);
    }))
      .then(convertBlobToBase64);
  }).catch(function (error) {
    /* istanbul ignore if */
    if (error.status !== 404) {
      throw error;
    }

    return getDocAttachments(src, doc)
      .then(convertBlobToBase64);
  });
}

That worked!
And that's why I made both @craftzdog/pouchdb-core-react-native and @craftzdog/pouchdb-replication-react-native.
If you found any problem on them, pull requests would be welcomed here.

Top comments (18)

badbod99 • Apr 4 '19

Looks good. I wonder if it's possible to polyfill the buffer logic rather than forking pouchdb to sort the attachment issue. Otherwise, might be worth putting together a PR for a new option to the PouchDB repo for forcing base64 when buffers are not supported/available.

Takuya Matsuyama • Apr 4 '19

I've tried rn-nodeify to polyfill Buffer but it didn't solve the issue.

Nathan Glenn • Apr 6 '19

Thanks so much for putting this together! I tried out the demo and it worked great. I was surprised to see that you have a simple "hello, world!" text encoded as base64. Why did you do that? Do all attachments have to be base64-encoded with your method?

Takuya Matsuyama • Apr 7 '19

That's because I wanted to demonstrate storing attachments works fine on RN apps. "Hello, world" text is useful for that.
Right. All attachments must be encoded in base64 to let PouchDB avoid calling readAsBinaryString.

Julio Márquez • Apr 30 '19 • Edited

Hi takuya,

I apply your changes but still I cannot see the base64 image. I can see the _attachment field but with a weird format (maybe blob)

I also set the encoding to plain/text
In couchdb server I can see the image

But when the synchronization tries to download the image back to the device its in a wrong format.

I can see the attachment has this form: 77+977+977+....9AAAA....

Could you please advise how did you solve this?

Thanks a lot in advance

Duy Tran • Jun 10 '19 • Edited

I have the same problem, image attachment has a weird format with 77+9UE5HDQoaCgAAAAAAAAAA...
I think it still a blob type since the structure of this object is like this:

Rom • Aug 21 '19

Thanks, wanted to mention the same. You're right, except there are some issues with attachments (see github.com/stockulus/pouchdb-react...), but they are working on it.

Alexander • Oct 19 '19

Sorry to say, but it doesn't and they dont, unfortunately. Project seems to be abandoned, look at open issues.
Sure you can still use it if you don't need replication (what?), but asyncstorage would be easier.

Rom • Oct 28 '19

Replication does work, but not attachment.

vittal • Aug 11 '20

I am getting this error
TypeError: WebSqlPouchCore.call is not a function

productivix • Oct 29 '22

hi, how did you solve this issue please ?

Mohaimin Islam • May 30 '20

Where are these SQLite DBs located on the phone? Does it exist in the app installation directory and will get removed upon app un-installation? If so, is there any DB that I can store outside the scope of the app installation so that it doesn't get removed after un-installation?

Asking because building a very offline first accessible app where the user might not be aware enough to know that uninstalling the app might lose all offline data.