Filebeat json decode dynamic target uses kubernetes pod name

Elastic Stack Beats
1

Hello,

is it possible to use a dynamic target in the decode_json_fields processor?
https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html

I would like to have a target as follows:
processors:

  • decode_json_fields:
    fields: ["message"]
    process_array: false
    max_depth: 1
    target: "pmsg.kubernetes.pod.name"

pmsg.kubernetes.pod.name -> message holds a json string which is decoded an put behind the target.

pmsg -> is a fixed string.
kubernetes.pod.name -> has the kubernetes pod name where the message is coming from -> that name changes depending on the pod name.

Thanks for your help

2

Hi @juka,

I think there is no way to do that with the decode_json_fields. But something you could do is using the rename processor to rename after decoding.

Would that help in this case?

Best regards

3

Thank you for your reply.
I can imagine using the rename processor if it gives me the option of accessing the kubernetes.pod.name for renaming

Currently we have a kubernetes cluster with several pods who log in json format. That json string is getting decoded and receives the prefix "pmsg" via the target option. My problem is that the pods have the same key names in the logging output but the values are sometimes strings, integers, etc. which leads to mapping conflicts.

I would like to parse the json string, decode it and write into different key names which include kubernetes pod information.

Is it possible to access the kubernetes.pod.name in the rename processor? By that I mean:

processors:
- rename:
     fields:
        - from: "pmsg.error_message"
           to: "nameofthekubernetespod.error_message"

nameofthekubernetespod.error_message -> this would be a dynamic field because the name of the kubernets pod changes.

Thank you for your help!

4

Yes, as far as I know that should be possible

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.