DEV Community

Cover image for Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana
shun
shun

Posted on • Edited on

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

#logstash #elasticsearch #kibana #docker

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

Create docker-compose.yml 

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.9.0
    environment:
      - discovery.type=single-node
    volumes:
      - esdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  logstash:
    image: docker.elastic.co/logstash/logstash:8.9.0
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    ports:
      - 5044:5044
    depends_on:
      - elasticsearch

  kibana:
    image: docker.elastic.co/kibana/kibana:8.9.0
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch

volumes:
  esdata:
Enter fullscreen mode Exit fullscreen mode

If you want to mount log files, use the following code:

volumes:
  - ./logstash/logs:/usr/share/logstash/logs
Enter fullscreen mode Exit fullscreen mode

Creating logstash/config/logstash.yml and logstash/pipeline/logstash.conf

Create a folder named "logstash" in the same directory as the docker-compose.yml. Inside the "logstash" folder, create two subfolders named "config" and "pipeline". Save the following logstash.yml and logstash.conf files into their respective folders.
File Structure:

/my_project
  docker-compose.yml
  /logstash
    /config
      logstash.yml
    /pipeline
      logstash.conf
Enter fullscreen mode Exit fullscreen mode

logstash/config/logstash.yml:

http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline
Enter fullscreen mode Exit fullscreen mode

logstash/pipeline/logstash.conf:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
  stdout {
    codec => rubydebug
  }
}
Enter fullscreen mode Exit fullscreen mode

Here is the basic Logstash configuration to read the content of a text file and output it to standard output (stdout):

input {
  file {
    path => "/path/to/your/file.txt"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

output {
  stdout { codec => rubydebug }
}
Enter fullscreen mode Exit fullscreen mode

Starting the Docker Container 

docker-compose up
Enter fullscreen mode Exit fullscreen mode

Stopping and Removing All Running Containers 

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
Enter fullscreen mode Exit fullscreen mode

Displaying Docker Container Logs 

docker logs <your-container-id>
Enter fullscreen mode Exit fullscreen mode

or

docker-compose logs logstash
Enter fullscreen mode Exit fullscreen mode

Stopping Docker Containers 

docker-compose down
Enter fullscreen mode Exit fullscreen mode

Restarting Containers 

docker-compose up
Enter fullscreen mode Exit fullscreen mode

Entering a Running Docker Container 

docker exec -it <container_id> /bin/bash
Enter fullscreen mode Exit fullscreen mode

or

docker exec -it <container_id> /bin/sh
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
askrodney profile image
Rodney Lab •

Thanks for sharing this shun, and welcome to dev.to!

Read next

oluchiedeh profile image

How to Dockerize a Python Script for Data Processing from a CSV File: A Step-by-Step Guide.

Oluchukwu Edeh -

siddhantkcode profile image

Building smarter Docker Images for your development environments with Pants 🚀

Siddhant Khare -

codewith_ili profile image

How to Run MongoDB in Docker

Ili Aliaj -

sukkergris profile image

Run devcontainers as a non-root user

Theodor Heiselberg -