Direct different log to different Kafka topic

Elastic Stack Beats
1

Hi,
How can I direct logs from different inputs (all same type: log) to different topics in my kafka cluster.

Thanks
N.

2

The topic can be set by referencing a variable topic: '%{[fields.topic]}'. Or there is a more expressive syntax that can be used with topics. See kafka output config.

Example 1:

filebeat.inputs:
- paths: [/web/*.log]
  fields:
    topic: web

output.kafka.topic: '%{[fields.topic]}'

Example 2:

filebeat.inputs:
- paths: [/web/*.log]
  fields:
    topic: web

output.kafka.topic: mydefault
output.kafka.topics:
- when.contains.source: web
  topic: web
- when.contains.message: login
  topic: authentication
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.