Kafka Input JSON Codec

[lifeofguenter (Gunter Grodotzki)]

From version 3.0.0 the default codec for kafka-input-plugin has been changed from json to plain (see: https://github.com/logstash-plugins/logstash-input-kafka/blob/master/CHANGELOG.md#300beta2-1).

One possibility would be to just set "codec" back to json, which I also did and it works.

However, what is the "recommended" way? The payloads coming from kafka are json strings (ecs-format: https://www.elastic.co/guide/en/ecs/current/index.html), so I am having difficulty applying a json-filter on all of it, as json-filter works with single fields - not the whole row/line/document.

Any advice?

[Badger]

@lifeofguenter

I am having difficulty applying a json-filter on all of it, as json-filter works with single fields - not the whole row/line/document

I do not understand the issue. If the input is using the plain codec then the entire message is put in the [message] field, and you can parse it using

json { source => "message" }