Hello
In my company I've set up our ELK stack and this worked the way it should but since yesterday it isn't able to process all logs in the morning. Yesterday I could fix this with a simple restart but today that doesn't work. It's not like much has changed, I only added 5 filebeats but these don't really produce that many logs (in comparison to other servers).
There are 16 workers and 4GB op JVM memory but nor the CPU or Memory is fully used which it's why I don't understand the problem. The CPU or JVM of the elasticsearch nodes also isn't fully used.
The only thing I can think of is that our logstash.conf file is fairly big (40 different 'if else' statements to split our logs) but this wasn't a problem before.
Is there anybody here that has experienced this issue before or that has any idea what the problem might be.
If you need more information please just ask.
Thank you for your help