I want to read key value kafka headers

Elastic Stack Logstash
1

Hello I want to read key value kafka headers but it is not giving me any output Can anyone please help me with the configuration or piece of code to read key value from kafka headers.
My old ticket reference -

2

Hello,

Can you provide more context? Your old post was about creating headers with the Kafka output, is this the same issue?

If so, nothing changes since then, this is still not supported.

3

Hello Currently we are consuming data from from kafka topic which has headers also configured in that kafka topic .. I want to read the headers with key and value using logstash configuration

input {
  kafka {
    bootstrap_servers => "your_kafka_broker"
    topics => ["your_topic"]
    codec => "json"
    consumer_threads => 1
    decorate_events => true
  }
}

filter {
  ruby {
    code => '
      headers = event.get("[@metadata][kafka][headers]")
      if headers
        headers.each { |key, value|
          event.set("header_#{key}", value)
        }
      end
    '
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

Above is my code which i am using to read headers from kafka topic but i see no results.

5

It seems that you need to use set this as extended to get the headers field.

From the documentation:

  • extended: record’s attributes, headers are added
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.