Filebeat publish metrics on start up to elastic and then stops publishing metrics

Filebeat publish metrics on start up to elastic and then stops publishing metrics.

We stopped filebeat by doing a kill -9 of the pid. Systemd restarts filebeat. Filebeat published data to the logs and then stops publishing

We are seeing these entries in the logs

2017-10-20T14:04:51-04:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=2 filebeat.harvester.running=2 filebeat.harvester.started=2 libbeat.es.call_count.PublishEvents=1 libbeat.es.publish.read_bytes=475 libbeat.es.publish.write_bytes=29946 libbeat.publisher.published_events=2046
2017-10-20T14:05:21-04:00 INFO No non-zero metrics in the last 30s
2017-10-20T14:05:21-04:00 INFO Error publishing events (retrying): temporary bulk send failure
2017-10-20T14:05:22-04:00 INFO Connected to Elasticsearch version 5.5.2
2017-10-20T14:05:22-04:00 INFO Trying to load template for client: http://monatee-loggy-master-cnj.dev.bnymellon.net:80
2017-10-20T14:05:22-04:00 INFO Template already exists and will not be overwritten.
2017-10-20T14:05:51-04:00 INFO Non-zero metrics in the last 30s: libbeat.es.call_count.PublishEvents=1 libbeat.es.publish.read_bytes=1064 libbeat.es.publish.write_bytes=6981 libbeat.es.published_and_acked_events=39 libbeat.es.published_but_not_acked_events=11

Which version of filebeat are you using?

We are using 5.5.2 - filebeat-5.5.2-linux-x86_64.tar.gz

I checked this morning and the issue resolved itself. I was seeing logs for the last 1 minute. I am surprised that it resolved itself. I was not expecting to see logs this morning

Kind of sounds like network issues or ES cluster not being stable.

How do we check the stability of the Elastic cliuster

X-pack basic (free) contains some monitoring support. Also check Elasticsearch logs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.