Part 4: Securely connect Node.js server to Elastic Cloud

Table of Content | Read Next: Part 5: Plan for efficient data storage and search performance in Elasticsearch

Resources

Would you rather watch a video to learn this content? Click on the link below!

• Episode 4: Securely connect Node.js server to Elastic Cloud

The following are the project directories shared in the blog. Click on the link below to access it!

• Code for connecting to Elastic Cloud with basic authentication
• Code for connecting to Elastic Cloud with an API Key

Connect Node.js Server to Elastic Cloud

In part 3, we signed up for a free trial of Elastic Cloud and created a deployment.

This deployment consists of Elasticsearch cluster, Kibana, Fleet, three Elastic Solutions(Enterprise Search, Observability, Security) and more!

Connecting to this deployment will give us access to everything mentioned above, including our Elasticsearch cluster.

We can connect our server to Elastic Cloud deployment via two different authentication mechanisms:

1. basic authentication
2. API key

API key is the best practice for production because it gives users flexibility in creating different API keys for different systems or applications.

In order to generate an API key, we need to log into Elastic Cloud as a user first(basic authentication).

Let's get started!

Step 1: Pull up the earthquake_app directory with your code editor

In part 2, we created a project directory called the earthquake_appand we built a Node.js server within it.

Next, we will create a file where we will store Elastic Cloud access credentials.

Step 2: Set up the config directory

In part 2, we have installed a dependency called config.

The config dependency allows you to store node configuration in an external configuration file. It can be customized based on different environments(local/dev/qa/prod/etc).

In our case, we will be using config to store access credentials such as Elastic cloud ID, username, password, and API key.

Within the earthquake_app directory, create a config directory. Within the config directory, create a default.json file as shown below.

Step 3: Enter the Elastic Cloud access credentials in the default.json file

Copy and paste the following code into the default.json file.

//in config/default.json
{
    "elastic": {
      "cloudID": "DEPLOYMENT_NAME:CLOUD_ID_DETAILS",
      "username": "elastic",
      "password": "LONGPASSWORD"
    }
}

Your default.json file should look like the following screenshot.

We will replace "DEPLOYMENT_NAME:CLOUD_ID_DETAILS"(line 3) and "LONGPASSWORD"(line 5) with our Elastic Cloud deployment access credentials.

In part 3, we have downloaded and saved the access credentials.

Find and open the saved credentials.

You should see something like this.

Copy your password and paste it into line 5 "password" as shown below.

Next, let's find our "DEPLOYMENT NAME:CLOUD_ID_DETAILS".

In part 3, we learned how to get to the Kibana home page.

From the Kibana home page, click on the menu icon(red box).

It will display a drop down menu as shown below.

From the drop down menu, click on the Manage this deployment option(pink box).

You will see the following on your screen.

Look at the region highlighted with a red box. This is where your deployment name and cloud id information are displayed.

Click on the copy button(blue box) then paste the info into line 3 of the default.json file.

Your default.json file should look like the following screenshot with your credentials pasted into line 3 and line 5.

Step 4: Connect to Elastic Cloud using basic authentication

Within the server directory, create a elasticsearch directory. Within the elasticsearch directory, create a client.js file as shown below.

At this point, you might be wondering how we will get our server to communicate with Elastic Cloud so we can establish a secure connection?

It's actually quite simple!

Elasticsearch Client is the simplest and easiest way to communicate with your Elasticsearch cluster from your app, in a programming language of your choice.

Elasticsearch Client allows you to easily access every endpoint of Elasticsearch API to perform a variety of tasks.

Since we are building a server using Node.js with Express, we will be using the Elasticsearch Javascript Client to connect to our Elasticsearch cluster!

In client.js, we will create an instance of Elasticsearch Client. Elasticsearch Client will contain our Elastic Cloud access credentials that point to our Elasticsearch cluster.

Then, we will set up our server to connect to Elastic Cloud and display a message in the terminal of its connection status.

In the client.js file, copy and paste the following code.

//in server/elasticsearch/client.js
const { Client } = require('@elastic/elasticsearch');
const config = require('config');

const elasticConfig = config.get('elastic');

const client = new Client({
  cloud: {
    id: elasticConfig.cloudID,
  },
  auth: {
    username: elasticConfig.username,
    password: elasticConfig.password
  },
});

client.ping()
  .then(response => console.log("You are connected to Elasticsearch!"))
  .catch(error => console.error("Elasticsearch is not connected."))

module.exports = client;  

Let's go over the code!

Heads up!
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for connecting to Elastic Cloud with basic authentication.

Line 1

We require @elastic/elasticsearch library we installed in part 2 and set this equal to a constant Client.

This library is the official Node.js client for Elasticsearch and it comes with a lot of functionalities. You will see this coming into play in line 6 when we create a new instance of Elasticsearch Client.

Line 2
We create a constant called config and require the config dependency we installed in part 2. You will see this come into play in line 4 when you use this dependency to get access credentials stored in the config directory.

Line 4
The access credentials were stored under 'elastic' in config/default.json as shown below.

We use config to get credentials stored under 'elastic' and set it equal to a constant elasticConfig.

This allows us to securely store and access sensitive data and not store these in the source code.

Lines 6-14

We create a new instance of Elasticsearch Client with our access credentials(cloudID, username, and password) that points to our Elasticsearch cluster. We set this equal to a constant called client.

Lines 16-18

We use the ping() method on our Elasticsearch Client to check if the Elasticsearch cluster is up and available to process requests.

We have it set up so that if we receive a response(meaning we are connected to Elasticsearch), our terminal will display the message "You are connected to Elasticsearch!".

If an error occurs during this process, our terminal should display the message "Elasticsearch is not connected".

Line 20
We expose our Elasticsearch Client via Node.js module exports as this will be used in server.js.

Go to the server.js file within the server directory. In line 2, add the following snippet.

//in server/server.js
const client = require('./elasticsearch/client');

Your server.js should look like the following screenshot.

In line 2, we require client.js from the elasticsearch directory. We created an instance of Elasticsearch Client in this file.

We do this as we will need to access our Elasticsearch Client from server.js.

Let's test if we are able to connect to Elastic Cloud as a user(basic authentication)!

Start your server by running the following command in the terminal.

//in terminal earthquake_app directory
npm start

You will see that the server is listening on port 3001 and that you are connected to Elasticsearch!

Step 5: Generate an API key
Now that we have logged into Elastic Cloud as a user, we can generate an API key.

From this point on, the API key will be used to connect to Elastic Cloud.

Within the server directory, create a file called create-api-key.js.

In create-api-key.js, copy and paste the following code.

//in server/create-api-key.js
const client = require('./elasticsearch/client');

async function generateApiKeys(opts) {
  const body = await client.security.createApiKey({
    body: {
      name: 'earthquake_app',
      role_descriptors: {
        earthquakes_example_writer: {
          cluster: ['monitor'],
          index: [
            {
              names: ['earthquakes'],
              privileges: ['create_index', 'write', 'read', 'manage'],
            },
          ],
        },
      },
    },
  });
  return Buffer.from(`${body.id}:${body.api_key}`).toString('base64');
}

generateApiKeys()
  .then(console.log)
  .catch((err) => {
    console.error(err);
    process.exit(1);
  });

Your create-api-key.js should look like the following.

Let's go over the code!

Heads up!
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for connecting to Elastic Cloud with API key.

Line 1
We require the instance of Elasticsearch Client we created in the previous step.

Lines 3-21
We define the generateApiKeys() function. This function creates an API key that gives access to our Elastic Cloud deployment.

Line 4
We define a constant body and set it equal to client.security.createAPIKey() method.

Lines 5-18
Within this method, we define what Elasticsearch cluster privileges are granted to the server connecting with the generated API key.

Line 6
We specify the name of the server using this API key.

Line 7
Under role_descriptors, we define a role(access privileges) granted to this server.

Line 8
We name this role as earthquakes_example_writer. You could name this role any way you see fit.

Line 9
We specify that this API key comes with cluster monitor privilege, which gives read-only access for determining cluster state.

Lines 12-13
This key also grants additional privileges such as create_index, write, read, and manage operations for an index named earthquakes.

Line 20
The security.createApiKey function(line 4) returns an id and api_key value which are concatenated and encoded in base64.

Lines 23-28
We call the generateApiKeys() function here which prints the API key in the terminal. If an error occurs, it prints the error instead.

Let's generate an API key by running the create-api-key.js file.

In the terminal, create a separate tab. cd into the earthquake_app directory and run the following command.

//in terminal within the earthquake_app directory
node server/create-api-key.js

You will see the following message in the terminal along with your API key.

Step 6: Connect to Elastic Cloud via API key

We will start by storing the API key in the default.json file of our config directory.

Go to default.json file and add a comma at the end of *line 5. In line 6, copy and paste the following.*

//in config/default.json
"apiKey": "paste your generated API key here"

Copy the API key from your terminal and paste it into *line 6 in default.json.

Your default.json file should look like this.

Open the client.js file in the elasticsearch directory.

Replace username and password(lines 11 and 12) with the following code.

//in elasticsearch/client.js
apiKey: elasticConfig.apiKey

Your client.js file should now look like the following screenshot(lines 11).

Go back to the terminal tab where you are running the server.

You will see that the server is connected to Elasticsearch except that this time we are connected to Elastic Cloud deployment via the API key!

Now that our server is connected to our Elasticsearch cluster, it is time to think about the data we want to store in Elasticsearch.

Move on to Part 5 to plan for efficient data storage and search performance in Elasticsearch!