Interpreting Filebeat INFO logs

[hding]

I'm trying to understand the following Filebeat INFO log:

INFO Non-zero metrics in the last 30s: filebeat.harvester.closed=2 filebeat.harvester.open_files=-1 filebeat.harvester.running=-2 libbeat.es.call_count.PublishEvents=138 libbeat.es.publish.read_bytes=2730237 libbeat.es.publish.write_bytes=555960458 libbeat.es.published_and_acked_events=635669 libbeat.publisher.published_events=545753 publish.events=917504 registrar.states.update=917504 registrar.writes=7

In particular, what do the following values mean? What are their differences? Which values are the important ones?

Events related:
libbeat.es.call_count.PublishEvents
libbeat.es.published_and_acked_events
libbeat.publisher.published_events
publish.events

Bytes related:
libbeat.es.publish.read_bytes
libbeat.es.publish.write_bytes

Thanks!

[hding]

My guess is that:

libbeat.es.published_and_acked_events: The events actually published, which is equal to the number of documents indexed successfully in Elasticsearch.

libbeat.publisher.published_events: If everything goes smooth, this should be equal to libbeat.es.published_and_acked_events. Otherwise, libbeat.publisher.published_events = libbeat.es.published_and_acked_events + #not_acked_events.

publish.events: The number of events in the original log files, including both the events being published, and the events excluded. Hence, if you drop/exclude some events/lines in Filebeat prospector, then publish.events = libbeat.publisher.published_events + #excluded_lines.

I am still trying to understand libbeat.es.publish.read_bytes and libbeat.es.publish.write_bytes. The size of my log files is 42GB. The size of the relevant lines in my log files is 37,508,554,927B (37.5GB). But after running expvar_rates.py, I get libbeat.es.publish.read_bytes: total 243121926 and libbeat.es.publish.write_bytes: total 56386818739, which do not match my data properties.