Hi Team,
I am using apache module and fileset in Beats+ELK stack where Filebeat is sending logs to logstash, logstash is using Ingest pipeline(we get module ingest pipeline - filebeat-8.7.1-apache-access-pipeline) , but it is somehow not utilizing the pipeline when ingesting data into elasticsearch. The same ingest pipeline works perfectly when I use filebeat directly to elasticsearch but not when logstash is used in between.
Expectation: Data plugin/geoip/useragent plugins should be applied to the data which is part of ingest pipeline. The _simulate api gives the expeted output but not via logstash.
Logstash logs: Data is coming to logstash, and I can see pipeline name is also mentioned, but it is not being utilized in Elasticsearch- No geoip or user agent filter got applied
Kibana output: Faulty output - 17th july data is coming from filebeat-->logstash->elasticsearch
Exppected output - 30th may 2014 - data is pushed directly through filebeat(with output set to elasticsearch and pipeline name mentioned in filebeat.yml)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
