Hi.
I have a FileBeat -> Kafka -> Logstash -> Elasticsearch setup, and the file that FileBeat monitors produces JSON lines with { "index" : "myIndex" }. Everything is sent to Kafka to the same topic and from there to Logstash. I would like Logstash to determine the Elasticsearch index according to this JSON field. The thing is FileBeat and Kafka wraps the message in such a way that my JSON is triple encoded since it is nested inside a message -> beat -> message hierarchy.
What can I do to solve this?
Thanks.