Elasticsearch - Winlogbeat Creating Query for an Index to Get a Specific String

Hello there, on this post I will share requesting method to get specific string from an API in Elastic - Winlogbeat.

I've searched to find an example of the query on internet but didn't find anything. Then I created a query by myself and I wanted to share it with you. So let's begin :)

System's Set Up and Configurations

We are using 2 type of beats which are Filebeat (for Linux clients) and Winlogbeat (for Windows clients).

Filebeat collects the logs of Linux client and sends them to elastic search on port 5044. Winlogbeat also collects the logs of Windows client and sends to elastic search on port 5043. The schema of system is:

Beat Configurations

Beat config files are the files that taking place on client machines to send the collected data to elastic server.

Filebeat Configuration File:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
    - /var/log/*/*.log

output.logstash:
  hosts: ["x.x.x.x:5044"]

Winlogbeat Configuration File:

winlogbeat.event_logs:
  - name: Application
    fields:
      log_type: application

  - name: System
    fields:
      log_type: system

  - name: Security
    fields:
      log_type: security

output.logstash:
  hosts: ["x.x.x.x:5043"]

Configuration Files on Elastic Server

These beats are welcomed by some config files on elastic server. The config files must be under /etc/logstash/conf.d/ directory.

filebeat.conf:

# INPUT HERE
input {
   beats {
      port => 5044
      client_inactivity_timeout => 1200
    }
  }

# FILTER HERE
filter{
}

#OUTPUT HERE
output {
      elasticsearch {
        index => "linux"
      }
}

winlogbeat.conf:

# INPUT HERE
input {
   beats {
      port => 5043
      client_inactivity_timeout => 1200
    }
  }

# FILTER HERE
filter{
}

#OUTPUT HERE
output {
      elasticsearch {
        index => "win10"
      }
}

With the help of this config files and adjustment, we are able to index Linux logs as "linux", Windows logs as "win10" on elastic server.

REST APIs of Elasticsearch

If we want to create a request to an index, we have to know which index we are messing. As you know Elasticsearch has some REST APIs. You can list your indexes with _cat API like this:

GET http://your_host_ip:9200/_cat/indices

Response of this request is:

yellow open win10                             004M_Z8HRhOOF9LLgP4Vkw 1 1     245     0 665.9kb 665.9kb
yellow open filebeat-7.17.1-2022.03.02        vWCH-_71QAubBvHXVtgz2g 1 1     392     0 158.8kb 158.8kb
green  open .apm-agent-configuration          QTZfTFNZSXOGcH0P2588YQ 1 0       0     0    226b    226b
yellow open filebeat-7.17.1-2022.03.01        MCb65OGTTECbn9xw1tJ87w 1 1    7271     0   1.1mb   1.1mb

As you can see above, I have 4 indices (to make it short) and win10 index has been created with Winlogbeat. That's why I named it as "win10".

What's Inside of an Index

If we wanted to look inside of an index to see what he has, the search API comes to help. We can see the beats with this request schema:

GET http://host_ip:9200/your_index/_search 

In my case, the request is:

GET http://x.x.x.x:9200/win10/_search

Response of this request should looks like this:

"took": 721,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 299,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "win10",
        ...

There are lots of things in the response, How are we going to choose what actually we want?
Well you can do it with "q=" parameter in request. In my case I want to get response of beats that its winlog.event_id is equals to "4624" (which refers to "An account was successfully logged on")
So I need to use this request:

GET http://x.x.x.x:9200/win10/_search?q=query.bool.must.filter.bool.should.match.winlog.event_id="4624"

Because the winlog.event_id is under lots of things. Here:

"query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "winlog.event_id": "4624"

If I want to reach winlog.event, then I need to write whole path seperated by dots. You need to adjust it for your needings.

That's all,
Hope it would be helpful for you :)