Filebeat is not using custom pipeline mentioned in the output.elasticsearch

Hello all,

Currently, I am using filebeat, elastic search 7.9.1 version and I have added the custom Ingest pipeline to push the logs but still, filebeat creates a new Ingest pipeline as filebeat-7.9.1-haproxy-log-pipeline. But whenever filebeat pushes the logs I want it to push the pipeline I have mentioned in the configuration, can someone help, please?



  hosts: ['localhost:9200']

  username: 'abc'

  password: 'abc'

  index: "mytestindex-%{[agent.version]}-%{+yyyy.MM.dd}"

  pipeline: "my_test_pipeline"

Here I am expecting it should use my_test_pipeline to push the logs instead of filebeat-7.9.1-haproxy-log-pipeline


Get node stats

GET _nodes/your-ingest-node/stats/ingest

Check the count fields in "my_test_pipeline"

How this will help me to solve the problem, please ?

If the count is > 0 entering the pipeline, the data is getting there. Each process in the pipeline has a count, so you can see some of what is happening inside the pipeline.

But I already know that none of the logs are processing through my pipeline.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.