We have filebeat 6.7.1 running as a daemonset on our kubernetes cluster. Everything was working fine till suddenly logs stopped popping up in ES.
When we checked filebeat logs, there was no ERROR but as filebeat there was nothing to harvest:
2019-07-31T12:30:08.604Z INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":35080},"total":{"ticks":229250,"value":229250},"user":{"ticks":194170}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":5},"info":{"ephemeral_id":"91360383-6212-4667-90be-daef7d298991","uptime":{"ms":2579430011}},"memstats":{"gc_next":4194304,"memory_alloc":2155264,"memory_total":22969148448}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.51,"15":0.26,"5":0.34,"norm":{"1":0.1275,"15":0.065,"5":0.085}}}}}}
But no sooner we restarted the pod, logs started getting harvested and we could now see the logs in es.
Is this a known bug in the version we're using? How do we debug this further?