FileBeat not following logfile

[cawoodm (Marc)]

FileBeat is not noticing new entries in our log files. To take one example we have a logfile modified recently (20:59 CET) but the registry is showing 14:40 UTC (15:40 CET):

{"source":"D:\hybris\log\tomcat\access.20190227.log","offset":68114397,"timestamp":"2019-02-27T14:40:32.1251557+01:00"

We have since restarted FileBeat several times. What can be the reason?

We are seeing logstash complain that ES is not accepting data and in the ES Log we are seeing update_mapping [_doc] events. The ES/Logstash server is not running at high CPU usage.

Logstash Log:

[2019-02-27T21:21:48,734][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://xxxxxx:9200/_bulk"}

[kvch (Noémi Ványi)]

Could you please share the debug logs of your Filebeat instance? Also, please paste its configuration file here formatted using </>.
My hunch is that either Logstash or Elasticsearch is not able to process the requests quickly enough, so Filebeat stops reading new events until it can forward events in its queue. But ofc I need more info to say something useful.

[cawoodm (Marc)]

Yes, that was indeed the case. Elastic Search was waaay behind it's workload.