Filebeat NGINX-Plugin sending multiples separate lines as one event

[iceman91176 (Carsten Buchberger)]

We have deployed filebeat as kubernetes daemon set and are using hint based configuration to send logs to logstash. Here is the filebeat configuration

filebeat.autodiscover:
  providers:
   - type: kubernetes
     host: ${HOSTNAME}
     hints.enabled: true
     hints.default_config.enabled: false
     labels.dedot: true
     annotations.dedot: true
     add_resource_metadata:
       namespace:
         enabled: true

processors:
  - add_cloud_metadata:

output.logstash:
  hosts: ["xxx:5044"]

Our nginx-deployments tag the pods like this

co.elastic.logs/enabled: "true"
co.elastic.logs/fileset.stderr: error
co.elastic.logs/fileset.stdout: ingress_controller

For the stdout strem everything is working fine, but with the stderr stream multiple separate lines get combined.

Example:

Container logs

{"log":"I0821 14:25:19.736405       6 flags.go:204] Watching for Ingress class: nginx-private\n","stream":"stderr","time":"2020-08-21T14:25:19.73649997Z"}
{"log":"W0821 14:25:19.736474       6 flags.go:207] Only Ingresses with class \"nginx-private\" will be processed by this Ingress controller\n","stream":"stderr","time":"2020-08-21T14:25:19.736514707Z"}
{"log":"W0821 14:25:19.737100       6 flags.go:249] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)\n","stream":"stderr","time":"2020-08-21T14:25:19.737170774Z"}
{"log":"W0821 14:25:19.737189       6 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.\n","stream":"stderr","time":"2020-08-21T14:25:19.737225145Z"}
{"log":"I0821 14:25:19.737410       6 main.go:220] Creating API client for https://10.127.32.1:443\n","stream":"stderr","time":"2020-08-21T14:25:19.737448109Z"}
{"log":"I0821 14:25:19.749068       6 main.go:264] Running in Kubernetes cluster version v1.16 (v1.16.10) - git (clean) commit f3add640dbcd4f3c33a7749f38baaac0b3fe810d - platform linux/amd64\n","stream":"stderr","time":"2020-08-21T14:25:19.749788766Z"}
{"log":"I0821 14:25:22.167797       6 status.go:86] new leader elected: ngix-ingress-private-nginx-ingress-controller-57f7bf5b8b-pms4w\n","stream":"stderr","time":"2020-08-21T14:25:22.167865205Z"}

Appear in Logstash as one long line with newline characters in between

"message" => "I0821 14:25:19.736405       6 flags.go:204] Watching for Ingress class: nginx-private\nW0821 14:25:19.736474       6 flags.go:207] Only Ingresses with class \"nginx-private\" will be processed by this Ingress controller\nW0821 14:25:19.737100       6 flags.go:249] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)\nW0821 14:25:19.737189       6 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.\nI0821 14:25:19.737410       6 main.go:220] Creating API client for https://10.127.32.1:443\nI0821 14:25:19.749068       6 main.go:264] Running in Kubernetes cluster version v1.16 (v1.16.10) - git (clean) commit f3add640dbcd4f3c33a7749f38baaac0b3fe810d - platform linux/amd64\nI0821 14:25:22.167797       6 status.go:86] new leader elected: ngix-ingress-private-nginx-ingress-controller-57f7bf5b8b-pms4w",

Any ideas/hints would be appreciated