Hi all!
I'm having trouble setting up Filebeat to send its data to Elasticsearch via Logstash. ELK stack is properly setup and is running without issues. After configuration, I can see the indices being created in the format filebeat-6.6.2-2019.MM.DD as expected but another index is also automatically created as "%{[@metadata][beat]}-%{[@metadata][version]}-2019.MM.DD" and I can't seem to figure out why. I've tried enable debug in both LS and ES to try to determine what is causing it but haven't had any chance so far. This index gets automatically recreated for some reason even after being deleted several times. I've confirmed that the stack element as well as filebeat are all at the same version (6.6.2), running on Ubuntu 16.04. Here are my logstash and filebeat settings. Any insight will be greatly appreciated.
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
filebeat.inputs:
- type: log
enabled: true
paths:- /var/log/*.log
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.dashboards.enabled: true
setup.kibana:
host: "xx.xx.xx.xx:5601"
output.logstash:
hosts: ["xx.xx.xx.xx:5044"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
-Bob