Hello, we're currently on Elasticsearch 6.3.2 with the log sources sending their data to Elasticsearch via Logstash.
I've recently noticed that Elasticsearch doesn't seem to be indexing anywhere near the number of documents it should be; for example our Winlogbeat index used to receive over 200,000 logs per 15 minutes and our syslog index used to receive about 700,000 logs per 15 minutes. However this has dropped drastically over time as we've added more log sources (Winlogbeat now about 2,000 logs per 15 mins, syslog about 60,000).
Over the months we've added more log sources and I can only assume that this is causing the slowdown / non-indexing because when I disable the other log sources and leave the Winlogbeat input enabled in Logstash, the logs per 15 minutes seems to return to normal (over 200,000 again).
I've looked at both the Logstash (logstash-plain.log) and Elasticseach (es-cluster.log) application logs and can't see anything about "throttling" or anything else that indicates Elasticsearch can't handle the log load I'm throwing at it - can anyone point me in the right direction?