hi, i'm experiencing an error when processing a json log which has a [null] value on a field.
■The log is the following (tested as valid json):
{"message":{"errorMessage":"INTERNAL_SERVER_ERROR","errorDetail":{"message":"Request failed with status code 500","name":"Error","stack":"xxx","config":{"url":"xxx","method":"post","data":"xxx","headers":{"Accept":"application/json, text/plain, */*","Content-Type":"application/json;charset=utf-8","User-Agent":"xxx","Content-Length":123},"field1":[null],"field2":[null],"requestStartedAt":1629355057329}}},"level":"error","version":"123","timestamp":"2021-08-19T06:37:41.658Z"}
■my filebeat config looks like this:
- type: log
paths:
- /tmp/file.log
fields_under_root: true
json.keys_under_root: true
json.add_error_key: true
enabled: true
output.logstash:
hosts: ["127.0.0.1:5044"]
■my logstash config which i reduced to the minimum to debug
input {
beats {
port => 5044
}
}
output {
file {
path => "/tmp/output.log"
codec => rubydebug
}
}
■content of /tmp/output.log
{
"timestamp" => "2021-08-19T06:37:41.658Z",
"tags" => [
[0] "beats_input_codec_plain_applied"
],
"version" => "123",
"@timestamp" => 2021-08-24T15:02:22.594Z,
"@version" => "1",
"level" => "error",
"message" => "{\"errorMessage\"=>\"INTERNAL_SERVER_ERROR\", \"errorDetail\"=>{\"config\"=>{\"field2\"=><Java::JavaUtil::ArrayList:31 [nil]>, \"requestStartedAt\"=>1629355057329, \"url\"=>\"xxx\", \"method\"=>\"post\", \"data\"=>\"xxx\", \"headers\"=>{\"User-Agent\"=>\"xxx\", \"Content-Length\"=>123, \"Accept\"=>\"application/json, text/plain, */*\", \"Content-Type\"=>\"application/json;charset=utf-8\"}, \"field1\"=><Java::JavaUtil::ArrayList:31 [nil]>}, \"message\"=>\"Request failed with status code 500\", \"name\"=>\"Error\", \"stack\"=>\"xxx\"}}"
}
As you can see i'm getting the [null] value translated to <Java::JavaUtil::ArrayList:31 [nil]>.
I validated that the java error is being introduced by logstash.
Using stack 7.8.0