Running filebeat version 6.2.4 with the autodiscover prospector for docker.
If the docker container restarts(crashes) creating a bad entry in the log file, filebeat gets stuck on the bad json entry
Example:
{"log":"2018-06-11 20:13:30 v2.8.2 [index] INFO: Kafka Subscriber\n","stream":"stdout","time":"2018-06-11T20:13:30.75194521Z"}
{"log":"2018-06-11 20:13:31 v2.8.2 [index] INFO: Kafka Subscriber: {"log":"2018-06-11 20:13:32 v2.8.2 [indexing] INFO: Kafka Subscriber: Caught up to latest: topic\n","stream":"stdout","time":"2018-06-11T20:13:31.52812067Z"}
As you can see, the first line is valid, the second line is invalid json. This will cause this error:
Read line error: decoding docker JSON: EOF; File: %!(EXTRA string=/var/lib/docker/containers/d58108f8c27171773c69995571430f3b90a33afe789b0464f33b48677e790522/d58108f8c27171773c69995571430f3b90a33afe789b0464f33b48677e790522-json.log)
I need filebeat to just ignore the bad entry and keep going. Instead it hangs and we will lose log data unless we go remove any bad line.