Json.ignore_decoding_error: true automatically enables json processing for everything?

[Kosodrom (Kosmodrom)]

Hi folks,

On our openshift clusters we have a wide range of applications and workloads running, which write logs in different ways.

If some developers write logs in json we want to enable them to annotate their workloads and to indicate that they want to have their json logs parsed into fields. At the same time, we don't want to know and to be involved in this configuration. Therefore we want to use hint based autodiscover. As far as I understand this is exactly what it is made for.

The reality is just: It does happen that some pods are still annotated with json annotations, but do not write json logs anymore for some reason. This results in spamming errors:

ERROR [reader_json] readjson/json.go:74 Error decoding JSON

You can suppress this error by giving: json.ignore_decoding_error: true but it seems that once you set it, Filebeat starts to parse ALL json logs that are coming, even from pods that are not annotated. This leads to mapping explosions, mapping conflicts and crazy management overhead.

1. Question: Do I understand correct that json.ignore_decoding_error: true, enabled json processor for all logs?
2. Question: Is it possible to suppress ERROR [reader_json] readjson/json.go:74 Error decoding JSON without enabling json processor for everything and just let hint based autodiscover work.

Thanks

[jsoriano (Jaime Soriano)]

Hey @Kosodrom,

What is the input configuration you are using? Would it be possible to use two different inputs, one for the json files and another one for the rest?

[Kosodrom (Kosmodrom)]

Hi,

Thanks for the response. Yes it would be possible of course. What I am doing now is: I drop all logs from my container, which contain this message:

  - drop_event:
      when:
        contains:
          message: "Error decoding JSON"

I was just curious if the described behavior is an expected one.

From my point of view setting json.ignore_decoding_error: true should not enable decode_json_fields processor.