Create indexs in elasticsearch using logstash & filebeat

Elastic Stack Elasticsearch
1

I was not able to see any data or any indexs in elasticsearch . I was using filebeat --> logstash --> elasticsearch

filebeat.yml

filebeat.inputs:

-> type: log

paths:
- /var/log/hadoop/hadoop/hive/gc.log*
document_type: hivemetastore

  • type: log

    paths:

    • /var/log/hadoop/hadoop/hdfs/gc.log*
      document_type: hdfs

logstash.conf

input {
beats {
port => 5044
}
}

output{

if [document_type] == "hdfs" {
elasticsearch {
hosts => ["endpoint:443"]
action => "index"
index => "hdfs"
}
}

if [document_type] == "hivemetastore" {
elasticsearch {
hosts => ["endpoint:443"]
action => "index"
index => "hive"
}
}

}

I'm not sure where i was doing wrong. without if condition i was able to see logs.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.