I understand that throughput of filebeat to elastic depends on many things, and before optimization I want to know it's really necessary.
I have a high load app that generates huge amount of logs, so I want to make sure that Filebeat send logs (directly to Elasticsearch) faster than logfile grows. Can I get information about amount of not processed lines in file or delta between processed and total number of lines? Does filebeat provides this kind of metrics out of the box? If not is it right way to compare offsets from registry file with total amount of lines in files for getting this kind of metric?