Setup/configure Elasticsearch on CentOS 7

Elasticsearch is flexible and powerful open-source, distributed real-time search and analytics engine. Using a simple set of APIs provides the ability for full-text search. Elastic search is freely available under the Apache 2 license, which provides the most flexibility.

This tutorial will help you to setup Elasticsearch single node cluster on CentOS, Red Hat, and Fedora systems.

Elasticsearch is a widely using Search Engine and it’s other use cases are log analytics, full-text search, security intelligence, business analytics etc. It’s open source, you can set it up as a cluster on your own servers. In this article, we will discuss about the basics of Elasticsearch and it’s use cases. How to setup a three node Elasticsearch cluster on CentOS servers.
Little bit history

Shay Banon is the founder of Elasticsearch. The first version of Elasticsearch was released on 2010 February. Here I am adding few words from Wiki…

While thinking about the third version of Compass he realized that it would be necessary to rewrite big parts of Compass to "create a scalable search solution". So he created "a solution built from the ground up to be distributed" and used a common interface, JSON over HTTP, suitable for programming languages other than Java as well.[6] Shay Banon released the first version of Elasticsearch in February 2010.

Since its release in 2010, Elasticsearch has quickly become the most popular search engine.
What is Elasticsearch?

Elasticsearch is an open-source, RESTful, distributed search and analytics engine built on Apache Lucene. We can use Elasticsearch in many areas to improve the performance of your infra. Apart from Search Engine, It’s a good option in analytics area. It’s a core component in RELK stack. To analyse the logs and metrics you can use the Elasticsearch cluster as the data store.

Prerequisites

1, Three CentOS servers for setting up the Elasticsearch cluster. Elasticsearch cluster should have a minimum of 3 master-eligible nodes.

2, If possible attach a separate disk for data storage.

3, Memory: Use a minimum 2 GB, the more heap available to Elasticsearch, the more memory it can use for its internal caches, but the less memory it leaves available for the operating system to use for the filesystem cache. Refer this official documentation: Setting the heap size

4, Don’t expose the Elasticsearch process to Public. Make sure you have a private network for inter node communication. For a cluster setup, nodes need to communicate each other.

5, Enable port 9200 and 9300 on all nodes for other nodes in the cluster.

6, Java: Install Java on all the servers.

That’s it. You’re all set to start setting up the three node Elasticsearch cluster.
Steps to setup three node Elasticsearch cluster on CentOS 7

Step 1: Install Java

As I mentioned in prerequisites, Elasticsearch needs Java, so we need to install Java first. To install Java on CentOS, please execute the following command:

 yum install java-1.8.0-openjdk

Execute “java -version” and make sure the Java is installed correctly.

Step 2: Download the Elasticsearch RPM

 curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.2.rpm

You can download the latest version from here >> Download Elasticsearch << In this page you can see all the packages, RPM, DEB etc…
Step 3: Install using RPM

 rpm -i elasticsearch-6.7.2.rpm

OR

First of all, install GPG key for the elasticsearch rpm packages.

 sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Then create yum repository file for the elasticsearch. Edit /etc/yum.repos.d/elasticsearch.repo file:

 sudo vi /etc/yum.repos.d/elasticsearch.repo

Add below content:

[Elasticsearch-7]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

then

sudo yum install elasticsearch

Step 4: Start / Enable service

systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Installation part is done. Once you installed it on all three servers, you can start editing the configuration to setup the cluster using these three nodes.

The Elasticsearch configuration file is located here:
/etc/elasticsearch/elasticsearch.yml

Before making changes in the configuration make sure that the port 9200 and 9300 are open between the nodes in the cluster. Add firewall rules accordingly. Try telnet / nc and make sure that the connections are okay between nodes..

The ElasticSearch has been successfully installed and running on your CentOS or RHEL system.

Run the following command to verify service:

 curl -X GET "localhost:9200/?pretty"

You will see the results like below:

{
"name" : "tecadmin",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "HY8HoLHnRCeb3QzXnTcmrQ",
"version" : {
"number" : "7.4.0",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
"build_date" : "2019-09-27T08:36:48.569419Z",
"build_snapshot" : false,
"lucene_version" : "8.2.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

Step 5 – Elasticsearch Examples (Optional)

The following examples will help you to add, fetch and search data in the Elasticsearch cluster.
Create New Bucket

 curl -XPUT http://localhost:9200/mybucket

Output:

  {"acknowledged":true}

Adding Data to Elasticsearch

Use following commands to add some data in Elasticsearch.
Command 1:

  curl -XPUT 'http://localhost:9200/mybucket/user/johny' -d '{ "name" : "Rahul Kumar" }'

Output:

{"_index":"mybucket","_type":"user","_id":"johny","_version":1,"created":true}

Command 2:

curl -XPUT 'http://localhost:9200/mybucket/post/1' -d '
{
"user": "Rahul",
"postDate": "01-15-2015",
"body": "This is Demo Post 1 in Elasticsearch" ,
"title": "Demo Post 1"
}'

Output:

{"_index":"mybucket","_type":"post","_id":"1","_version":1,"created":true}

Command 3:

curl -XPUT 'http://localhost:9200/mybucket/post/2' -d '
{
"user": "TecAdmin",
"postDate": "01-15-2015",
"body": "This is Demo Post 2 in Elasticsearch" ,
"title": "Demo Post 2"
}'

Output:

{"_index":"mybucket","_type":"post","_id":"2","_version":1,"created":true}

Fetching Data from Elasticsearch

Use the following command to GET data from ElasticSearch and read the output.

curl -XGET 'http://localhost:9200/mybucket/user/johny?pretty=true'
curl -XGET 'http://localhost:9200/mybucket/post/1?pretty=true'
curl -XGET 'http://localhost:9200/mybucket/post/2?pretty=true'

Searching in Elasticsearch

Use the following command to search data from elastic search. Below command will search all data associated with user johny.

 curl 'http://localhost:9200/mybucket/post/_search?q=user:TecAdmin&pretty=true'

Output:

{
"took" : 145,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
"_index" : "mybucket",
"_type" : "post",
"_id" : "2",
"_score" : 0.30685282,
"_source":
{
"user": "TecAdmin",
"postDate": "01-15-2015",
"body": "This is Demo Post 2 in Elasticsearch" ,
"title": "Demo Post 2"
}
} ]
}
}