We're using the Elastic Stack to centralize logs across multiple systems.
We're using Windows, and are currently on version 7.10. (Upgrading to 7.15 in the near future.)
Occasionally all of our log ingestion halts completely. After investigating, within the $logstash/data/queue/logbeat directory, there are two "page" files.
page.360 - 0 KB
page.361 - 65,536 KB
When we delete the 0KB file and restart logstash, processing resumes.
This happens once a month. Currently we have alerts set for when log ingestion fails, and we check for this condition. We're considering setting up a script to do it automatically. We're curious if anyone knows the cause and a good permanent resolution?
Any idea if this is a bug within logstash that may be fixed in 7.15? Or if this is related to Java/Windows?
Thanks.