I'm trying to understand the following Filebeat INFO log:
INFO Non-zero metrics in the last 30s: filebeat.harvester.closed=2 filebeat.harvester.open_files=-1 filebeat.harvester.running=-2 libbeat.es.call_count.PublishEvents=138 libbeat.es.publish.read_bytes=2730237 libbeat.es.publish.write_bytes=555960458 libbeat.es.published_and_acked_events=635669 libbeat.publisher.published_events=545753 publish.events=917504 registrar.states.update=917504 registrar.writes=7
In particular, what do the following values mean? What are their differences? Which values are the important ones?
My guess is that:
libbeat.es.published_and_acked_events: The events actually published, which is equal to the number of documents indexed successfully in Elasticsearch.
libbeat.publisher.published_events: If everything goes smooth, this should be equal to
publish.events: The number of events in the original log files, including both the events being published, and the events excluded. Hence, if you drop/exclude some events/lines in Filebeat prospector, then
I am still trying to understand
libbeat.es.publish.write_bytes. The size of my log files is 42GB. The size of the relevant lines in my log files is 37,508,554,927B (37.5GB). But after running
expvar_rates.py, I get
libbeat.es.publish.read_bytes: total 243121926 and
libbeat.es.publish.write_bytes: total 56386818739, which do not match my data properties.
This topic was automatically closed after 21 days. New replies are no longer allowed.