Add field with logstash (using kafka input plugin)

Elastic Stack Logstash
1

I'm using the kafka input plugin for Logstash and there are metadata fields described in https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html.

I want to copy some or all the fields into my data for reference/debugging. The use case is basically the same as Sending @metadata from logstash to elastic search

I tried the following in my filter:

mutate {
   add_field => { "kafka_topic"=> "%{[@metadata][kafka][topic]}" }
   add_field => { "kafka_consumer_group"=>"%{[@metadata][kafka][consumer_group]}"  }
   add_field => { "kafka_timestamp"=>"%{[@metadata][kafka][timestamp]}" }
}

But what I see in Elasticsearch is the literal text for the value (eg. "%{[@metadata][kafka][consumer_group]}")

I tried the ruby script in the question above, but had some error. It seems like this is a simple thing but I can't figure out how to make it work.

Can someone help?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.