Filebeat output empty JSON array as "null" instead of "[]"

ijingo · June 22, 2021, 4:45am

Filebeat output empty JSON array as "null" instead of [ ].

The original line of JSON is as follows, in which "array_field" is an empty array.

{"body": {"array_field": []}}

However, after output, the "array_field" is converted to "null".

2021-06-22T12:30:21.670+0800    INFO    log/harvester.go:302    Harvester started for file: /Users/wangji/example.log
{"array_field":null}

I expected the output should be the same as the original input.

Example configuration YAML:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /Users/wangji/example.log
  json:
    message_key: json

output.console:
  codec.format.string: '%{[json.body]}'

The version of filebeat I tried is :

filebeat version 7.13.2 (amd64), libbeat 7.13.2 [686ba416a74193f2e69dcfa2eb142f4364a79307 built 2021-06-10 21:04:13 +0000 UTC]

I am suspecting that this behavior is a bug, which is caused by this line:

github.com

elastic/beats/blob/master/libbeat/common/event.go#L134

    
      
          		}
          		out = append(out, normalizedMap)
          	}
          
          
	return out, errs
          }
          
          
// normalizeSlice normalizes each element of the slice and returns a []interface{}.
          func (e *GenericEventConverter) normalizeSlice(v reflect.Value, keys ...string) (interface{}, []error) {
          	var errs []error
          	var sliceValues []interface{}
          
          
	n := v.Len()
          	for i := 0; i < n; i++ {
          		sliceValue, err := e.normalizeValue(v.Index(i).Interface(), append(keys, strconv.Itoa(i))...)
          		if len(err) > 0 {
          			errs = append(errs, err...)
          		}
          
          
		sliceValues = append(sliceValues, sliceValue)
          	}

When the v.Len() equals 0, this method will return a "nil" instead of an empty interface{} array.
Change the line as follows can fix the anomaly behavior. This change will explicitly initialize the return value as an empty array of interface{} type.

sliceValues := []interface{}{}

ijingo · June 30, 2021, 7:49am

hello?

system · July 28, 2021, 9:49am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.