Good afternoon all.
I've recently installed a filebeat and enabled the system and apache modules.
After that I set the filebeat.yml to point to ES and Kibana and run the
'filebeat setup -e'
Everything went as expected. With an apache2 running I started the filebeat and I saw the number of documents increased in my datastream:
.ds-filebeat-8.4.3-2022.11.09-000001
I opened Kibana to select both, '[Filebeat System] ECS' and '[Filebeat Apache] Access and error logs ECS'. In both of them I saw how data was showed.
I stopped filebeat and I edited the filebeat.yml to point to Logstash.
In Logstash I created the following pipeline:
input {
beats {
port => "5044"
}
}
output {
if [@metadata][pipeline] {
elasticsearch {
hosts => ["https://192.168.0.111:9200","https://192.168.0.112:9200","https://192.168.0.113:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
action => "create"
pipeline => "%{[@metadata][pipeline]}"
cacert => '/certs/elastic/http_ca.crt'
user => "${LS_USER}"
password => "${LS_PWD}"
}
} else {
elasticsearch {
hosts => ["https://192.168.0.111:9200","https://192.168.0.112:9200","https://192.168.0.113:9200"]
manage_template => false
cacert => '/certs/elastic/http_ca.crt'
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
action => "create"
user => "${LS_USER}"
password => "${LS_PWD}"
}
}
}
Once I saved the file I started Logstash which run properly and I restarted filebeat.
After this more data was added in the datastream but when I checked the same dashboards, '[Filebeat System] ECS' showed data but '[Filebeat Apache] Access and error logs ECS' didnt.
At the moment I make the filebeat point to the ES again I see data back in both when it point to Logstash '[Filebeat Apache] Access and error logs ECS' stops showing data.
Any idea about what could be wrong?
Am I missing something?
Thank you in advance and best regards.
