Output to more than one Kafka topics

Elastic Stack Beats
1

Hi,

We are trying to send output to more than one Kafka topics using filebeat. It is not working as expected.
Output section:

output.kafka:
  enabled: true

  hosts: [ "---" ]

 topic: "%{[fields.topic_name]}"
  
  topics:
    - topic: "topic_A"
      when:
      and:
      - equals:
      apptype: "app"
    - topic: "topic_B"
      when:
      and:
      - equals:
      apptype: "book"

  # Use SSL settings for HTTPS.
  ssl.enabled: true

  # Use TLS settings for encryption
  tls.certificate_authorities: ['D:\tmp\kafka.client.truststore.jks']
  
  #================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

  processors:
  - fingerprint:
      fields: ["message"]
#================================ Logging =====================================


logging.level: debug
logging.to_files: true
logging.files:
  path: D:\Logs\filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

Could anyone please help on this?

2

HI,

the problem can be the indentation.

Try with this config:

output.kafka:
  enabled: true
  hosts: ["---"]
  topic: "%{[fields.topic_name]}"
  topics:
    - topic: "topic_A"
      when.equals:
        apptype: "app"
    - topic: "topic_B"
      when.equals:
        apptype: "book"
  ssl.enabled: true
  tls.certificate_authorities: ['D:\tmp\kafka.client.truststore.jks']
processors:
  - fingerprint:
      fields: ["message"]
logging.level: debug
logging.to_files: true
logging.files:
  path: D:\Logs\filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

Regards

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.