How to set up each pipeline for each index in filebeats?

Elastic Stack Beats
1

I use one filebeats, to ship two log
1 hightec.log
2 ghslog.log

they are different format ,
I want to set up different pipeline for the each log ,
can you tell me how to config it ?

Thanks!

- type : log

  enabled : true

  paths :

    #- /var/log/*.log

- D:\Software\HDP\test_log\hightec.log

  fields :

    type : "hightec"

- type : log

  enabled : true

  paths :

    #- /var/log/*.log

    - D:\Software\HDP\test_log\GHSLOG.log

  fields :

type : "ghs"

 

setup.template.name : "index2-%{[agent.version]}"

setup.template.pattern : "index2-%{[agent.version]}-*"

 

output.elasticsearch :

  # Array of hosts to connect to.

  hosts : [ "174.34.93.62:9200" ]

  # pipeline: "ne_user_lookup"

  index : "index2-%{[agent.version]}-%{[fields.type]:other}-%{+yyyy.MM.dd}"
2

You can define a pipeline in each input and the appropriate pipeline will be used in the output.

See here

- type : log

  enabled : true

  pipeline: my-pipeline
...
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.