I have installed elasticsearch, kibana and logstash on 3 separate machines. I am able to use metricbeat, packetbeat and auditbeat without any issue.
But when I install filebeat, it does not create an index on elasticsearch. I have manually loaded the template and I can see the template in ES. But the index does not get created automatically, the way it did for metricbeat and packetbeat. My setup is filbeat to push to logstash and logstash to ES.
I have tried both automatically and manually to put the template.
curl -XPUT -H 'Content-Type: application/json' http://X.X.X.Y:9200/_template/filebeat-6.4.1 -d@filebeat.template.json
My filebeat configuration:
filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/secure
- /var/log/messages
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
reload.period: 10s
setup.template.settings:
index.number_of_shards: 1
index.number_of_replicas: 1
setup.template.name: "filebeat"
setup.template.pattern: "filebeat-*"
setup.template.overwrite: true
output.logstash:
# The Logstash hosts
hosts: ["X.X.X.X:5044"]
The template as seen in ES:
curl -XGET http://X.X.X.Y:9200/_template?pretty
{
".monitoring-logstash" : {
"order" : 0,
"version" : 6040099,
"index_patterns" : [
".monitoring-logstash-6-*"
],
"settings" : {
"index" : {
"format" : "6",
"codec" : "best_compression",
The rest of the indices are seen
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open auditbeat-6.4.1-2018.09.24 oxhl6enaRDy7q_gACYFlrw 5 1 9 0 91.4kb 91.4kb
yellow open packetbeat-6.4.1-2018.09.25 otBpda3OQ_uUX8Jr0y66tA 5 1 72681 0 16.4mb 16.4mb
green open .kibana uIva3SKOSsOhNhRWhOGHXA 1 0 25 4 93.2kb 93.2kb
yellow open auditbeat-6.4.1-2018.09.23 jt9p5K4hTDC6E-gOSoR4yA 5 1 1476 0 1000.7kb 1000.7kb
yellow open packetbeat-6.4.1-2018.09.26 4Jcc3xSzSuethsjWVYsl-A 5 1 61922 0 24.2mb 24.2mb
yellow open auditbeat-6.4.1-2018.09.25 jYBSCT_MQouYKFFtfV3c2w 5 1 16 0 166kb 166kb
yellow open metricbeat-6.4.1-2018.09.23 ytPYG9YbRaWxYRUAZyvmEA 5 1 128950 0 40.7mb 40.7mb
yellow open packetbeat-6.4.1-2018.09.24 bEKLAiOPSYSqkygHWmV1rg 5 1 39143 0 9.6mb 9.6mb
yellow open metricbeat-6.4.1-2018.09.25 Gp3HbEOiSgi7n_pxCFBJyg 1 1 213274 0 33.1mb 33.1mb
yellow open metricbeat-6.4.1-2018.09.26 Krbp5E_6Qo6jZtOML4aojg 1 1 202551 0 39mb 39mb
yellow open metricbeat-6.4.1-2018.09.22 kvAq_QlXT0-J1q_rxiUtqw 5 1 35689 0 12.2mb 12.2mb
yellow open auditbeat-6.4.1-2018.09.26 umGzK1-0Qv-sMLWTKEfC-A 5 1 4 0 38.1kb 38.1kb
yellow open metricbeat-6.4.1-2018.09.24 6O3vGJAISUCI89iyHkEUWQ 5 1 181624 0 54.7mb 54.7mb
Thanks and Regards