How to install Elasticsearch 7 with Kibana using Docker Compose

This tutorial will help you setup a single node Elasticsearch cluster with Kibana using Docker Compose.

Pre-requisites

• Install Docker (refer to official docs if not installed https://docs.docker.com/engine/install/)
• Install Docker Compose (refer to official docs if not installed https://docs.docker.com/compose/install/)

This tutorial assumes you are comfortable with Docker and Docker Compose. If you are not, you can go through this article of mine which is kind of a crash course with Docker Compose (https://medium.com/swlh/simplifying-development-on-your-local-machine-using-docker-and-docker-compose-2b9ef31bdbe7?source=friends_link&sk=240efed3fd3a43a1779e7066edb37235)

Video Lesson

I have also created a video tutorial for this on my YouTube channel. If you prefer that, you may visit the link below and check it out

https://www.youtube.com/watch?v=EClKhOE0p-o

Step 1: Create docker-compose.yml file

Create a directory on your machine for this project

mkdir $HOME/elasticsearch7-docker
cd $HOME/elasticsearch7-docker

Inside that directory create a docker-compose.yml file with contents as shown below

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2-amd64
    env_file:
      - elasticsearch.env
    volumes:
      - ./data/elasticsearch:/usr/share/elasticsearch/data

  kibana:
    image: docker.elastic.co/kibana/kibana:7.9.2
    env_file:
      - kibana.env
    ports:
      - 5601:5601

Step 2: Create the env files

Both Elasticsearch and Kibana docker images allow us to pass on environment variables which are passed on to the configuration as defined in elasticsearch.yml and kibana.yml files. For passing the environment variables to container, we can use the env_file setting of the docker compose file.

Create the elasticsearch.env file:

cluster.name=my-awesome-elasticsearch-cluster
network.host=0.0.0.0
bootstrap.memory_lock=true
discovery.type=single-node

Note: With latest version of Elasticsearch, it is necessary to set the option discovery.type=single-node for a single node cluster otherwise it won't start

Create kibana.env file

SERVER_HOST="0"
ELASTICSEARCH_URL=http://elasticsearch:9200
XPACK_SECURITY_ENABLED=false

Step 4: Create Elasticsearch data directory

Navigate to the directory where you have created your docker-compose.yml file and create a subdirectory data. Then inside the data directory create another directory elasticsearch.

mkdir data
cd data
mkdir elasticsearch

We will be mounting this directory to the data directory of elasticsearch container. In your docker-compose.yml file there are these lines:

    volumes:
      - ./data/elasticsearch:/usr/share/elasticsearch/data

This ensures that the data on your Elasticsearch container persists even when the container is stopped and restarted later. So, you won't lose your indices when you restart the containers.

Step 4: Run the setup

We're good to go now. Open your terminal and navigate to the folder containing your docker-compose.yml file and run the command:

docker-compose up -d

This will start pulling the images from docker.elastic.co and depending on your internet speed, this should take a while. Once the images are pulled, it will start the containers.

You can run the following command to see if both the containers are running:

docker-compose ps

The output should look something like this

                Name                              Command               State           Ports         
------------------------------------------------------------------------------------------------------
docker-elasticsearch-setup_elasticsearch_1   /tini -- /usr/local/bin/do ...   Up      9200/tcp, 9300/tcp    
docker-elasticsearch-setup_kibana_1          /usr/local/bin/dumb-init - ...   Up      0.0.0.0:5601->5601/tcp

Notice the State field. It should be Up for both the containers. If it is not, then check the logs using the command (replace {serviceName} with the name of the service, eg elasticsearch or kibana)

docker-compose logs -f {serviceName}

A common error that you might encounter is related to vm.max_map_count being too low. You can fix it by running the command

sysctl -w vm.max_map_count=262144

Check this link for more details https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html

If both the services are running fine, you should be able to see kibana console on http://localhost:5601 on your web browser. Give it a few minutes as it takes some time for Elasticsearch cluster to be ready and for Kibana to connect to it. You can get more info by inspecting the logs using docker-compose logs -f kibana command.

This completes our setup. Hope you found it helpful. Happy coding :-)