we are shipping logs using following route:
logfile <- filebeat -> redis <- logstash -> elasticsearch <- kibana
Using filebeat 6.5.2 on windows 2012 R1. Filebeat is installed as service.
Each day at about 23:00 local time we are facing a strange effect when we discover the data in kibana:
Then we have a block where all lines are looking that cryptic.
Lines before that block are fine.
Here you can see the garbage block. Much more lines than usual.
If i check the input logfile, all seems to be fine.
Other logfiles are not affected.
I checked if there are correctly parsed lines missing in kibana, e.g. to track down if some lines in the source logfile are causing this issue.
Looks like as if there is NO line missing. lasst message displayed ok before issue and first event without issue are next to each other in logfile.
Any Ideas? I will ask in datacenter if they are running a backup at this time, but it is strange that other logfiles are not effected.