Not able to create /see any indexs in elasticsearch

[bob_ve (Bhargav Madala)]

I was not able to see any data or any indexs in elasticsearch . I was using filebeat --> logstash --> elasticsearch

filebeat.yml

filebeat.inputs:

-> type: log

paths:

• /var/log/hadoop/hadoop/hive/gc.log*
  document_type: hivemetastore

• type: logpaths:
  • /var/log/hadoop/hadoop/hdfs/gc.log*
    document_type: hdfs

logstash.conf

input {
beats {
port => 5044
}
}

output{

if [document_type] == "hdfs" {
elasticsearch {
hosts => ["endpoint:443"]
action => "index"
index => "hdfs"
}
}

if [document_type] == "hivemetastore" {
elasticsearch {
hosts => ["endpoint:443"]
action => "index"
index => "hive"
}
}

}

I'm not sure where i was doing wrong. without if condition i was able to see logs.

[steffens (Steffen Siering)]

I don't think filebeat has any field named document_type. That's why the conditions fail. If you insist on document_type, you can configure filebeat this way:

paths:
- /var/log/hadoop/hadoop/hive/gc.log*
fields.document_type: hivemetastore
fields_under_root: true