To build the best search experience for your application you need Full-text search.

Full-text search enables you to find documents that contain a set of words or phrases within their text.

MongoDB has a $text index to provide a limited full-text search in a single collection, more info here $text
If you are using MongoDB Atlas, you can also use MongoDB Atlas Search

If you are using a self-hosted deployment and want to search in more than one collection, elastic search is the best option for you

MongoDB + ElasticSearch Architecture

We are going to use MongoDB Change Streams (CDC - Change Data Capture) to sync MongoDB data to Elastic Search.

You can read more about it here The right decisions can simplify and make it easier to scale.

This is the simplest implementation to sync a MongoDB collection using change stream to elastic search.

import { Client } from '@elastic/elasticsearch';

const esClient = new Client({ node: process.env.ELASTICSEARCH });

const transactionStreamHandler = async (data: ChangeStreamDocument<ITransaction>) => {
  // map mongodb transaction to elasticsearch data
  const esPayload = {
    identifier: data.fullDocument.identifier,
    customer: {
      name: data.fullDocument.customer.name,
      email: data.fullDocument.customer.email,
      taxID: data.fullDocument.taxID?.taxID,
    },
  }

  const document = {
    index: 'woovi-transaction',
    id: data.fullDocument._id,
    document: esPayload,
  };

  await esClient.index(document);
}

const stream = Transaction.watch([], {
    fullDocument: 'updateLookup',
  });

stream.on('change', transactionStreamHandler);

Transaction.watch will watch for any changes to the Transaction collection.

transactionStreamHandler will map the Transaction collection data to the elastic search payload that will be used in the full-text search.
You don't need to save everything on the elastic search, only the fields that will be used on the search.
We are saving Transaction on the woovi-transaction elastic search index, we are using woovi-* prefix to enable us to search in many collections at once.

Retrieving data from Elastic Search

To search, we call the elastic search client search method with our query.

const search = 'Sibelius'

const searchQuery = {
    index: 'woovi-*',
    size: 200,  
    query: {
      bool: { 
        must: [
          { query_string: { query: `${search}*` } },
        ],
      }
    }
  }

const search = await esClient.search(searchQuery);

In the above query, we search every woovi-* index that matches the query Sibelius, any Transaction, Charge, User, Customer.
You could search only transaction passing, woovi-transaction, or passing a list of indexes to search like woovi-transaction,woovi-customer.

To conclude

Syncing to a database sounds like a hard problem, but MongoDB change streams greatly simplify this task.
This approach is also lightweight.
Try to add full-text search in your application to provide the best search experience for your users.

Woovi is an innovative startup revolutionizing the payment landscape. With Woovi, shoppers can enjoy the freedom to pay however they prefer. Our cutting-edge platform provides instant payment solutions, empowering merchants to accept orders and enhance their customer experience seamlessly.

If you're interested in joining our team, we're hiring! Check out our job openings at Woovi Careers.

Photo by Amélie Mourichon on Unsplash