[Logstash] <Java::JavaUtil::ArrayList:31 [nil]> on json with [null]

[zenkovac]

hi, i'm experiencing an error when processing a json log which has a [null] value on a field.

■The log is the following (tested as valid json):

{"message":{"errorMessage":"INTERNAL_SERVER_ERROR","errorDetail":{"message":"Request failed with status code 500","name":"Error","stack":"xxx","config":{"url":"xxx","method":"post","data":"xxx","headers":{"Accept":"application/json, text/plain, */*","Content-Type":"application/json;charset=utf-8","User-Agent":"xxx","Content-Length":123},"field1":[null],"field2":[null],"requestStartedAt":1629355057329}}},"level":"error","version":"123","timestamp":"2021-08-19T06:37:41.658Z"}

■my filebeat config looks like this:

- type: log
  paths:
    - /tmp/file.log
  fields_under_root: true
  json.keys_under_root: true
  json.add_error_key: true
  enabled: true

output.logstash:
  hosts: ["127.0.0.1:5044"]

■my logstash config which i reduced to the minimum to debug

input {
    beats {
        port => 5044
    }
}
output {
    file {
        path => "/tmp/output.log"
        codec => rubydebug
  }
}

■content of /tmp/output.log

{
     "timestamp" => "2021-08-19T06:37:41.658Z",
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
       "version" => "123",
    "@timestamp" => 2021-08-24T15:02:22.594Z,
      "@version" => "1",
         "level" => "error",
       "message" => "{\"errorMessage\"=>\"INTERNAL_SERVER_ERROR\", \"errorDetail\"=>{\"config\"=>{\"field2\"=><Java::JavaUtil::ArrayList:31 [nil]>, \"requestStartedAt\"=>1629355057329, \"url\"=>\"xxx\", \"method\"=>\"post\", \"data\"=>\"xxx\", \"headers\"=>{\"User-Agent\"=>\"xxx\", \"Content-Length\"=>123, \"Accept\"=>\"application/json, text/plain, */*\", \"Content-Type\"=>\"application/json;charset=utf-8\"}, \"field1\"=><Java::JavaUtil::ArrayList:31 [nil]>}, \"message\"=>\"Request failed with status code 500\", \"name\"=>\"Error\", \"stack\"=>\"xxx\"}}"
}

As you can see i'm getting the [null] value translated to <Java::JavaUtil::ArrayList:31 [nil]>.
I validated that the java error is being introduced by logstash.
Using stack 7.8.0

[Badger]

Definitely looks like a bug to me. With 7.14.0, if you use

processors:
  - decode_json_fields:
      fields: [ "message" ]
      process_array: true
      target: ""

then you get an array containing nil, as expected

       "foo" => [
    [0] nil
],
   "message" => "{ \"foo\": [null] }",

If you remove the target option, so that [message] is overwritten, then you get

   "message" => "{\"foo\"=><Java::JavaUtil::ArrayList:31 [nil]>}",

[zenkovac]

Thanks badger, I reported it in github:

[Logstash] <Java::JavaUtil::ArrayList:31 [nil]> on json with [null]