Json processor fails when processing some fields without quotes

Elastic Stack Kibana
1

Hello,

We are sending logs to escloud using fluentbit 2.0.6 with the field message in JSON format but when we use the JSON processor in that field it fails in different ways.

  1. As some field values come without quotes and starting with 0 it throws an error.
    "time_local": 06/Feb/2023:16:29:43 +0000,
    Invalid numeric value: Leading zeroes not allowed

  2. Other fields with empty values and without quotes also fail
    "remote_user": ,
    failed with message Unexpected character (',' (code 44)): expected a value

Is there a way to process those type of json logs? Or do you know if it's possible to add quotes to all fields?

regards

2

Can you share a sample of your message?

It doesn't seem to be a valid JSON as the values do not have double quotes, which is needed by the JSON processor.

For example, if you have something like this:

{ "remote_user": ,"time_local": 06/Feb/2023:16:29:43 +0000 }

Then the processor will not work as this is not a valid JSON.

The best option is to fix the message before sending to elasticsearch, I do not use fluentbit, but have you checked if there is any option in the output to send it as a valid json?