Filebeat is not using custom pipeline mentioned in the output.elasticsearch

Elastic Stack Beats
,
1

Hello all,

Currently, I am using filebeat, elastic search 7.9.1 version and I have added the custom Ingest pipeline to push the logs but still, filebeat creates a new Ingest pipeline as filebeat-7.9.1-haproxy-log-pipeline. But whenever filebeat pushes the logs I want it to push the pipeline I have mentioned in the configuration, can someone help, please?

filebeat.yml

output.elasticsearch:

  hosts: ['localhost:9200']

  username: 'abc'

  password: 'abc'

  index: "mytestindex-%{[agent.version]}-%{+yyyy.MM.dd}"

  pipeline: "my_test_pipeline"

Here I am expecting it should use my_test_pipeline to push the logs instead of filebeat-7.9.1-haproxy-log-pipeline

Regards,
Mahesh

2

Get node stats

GET _nodes/your-ingest-node/stats/ingest

Check the count fields in "my_test_pipeline"

3

How this will help me to solve the problem, please ?

4

If the count is > 0 entering the pipeline, the data is getting there. Each process in the pipeline has a count, so you can see some of what is happening inside the pipeline.

5

But I already know that none of the logs are processing through my pipeline.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.