I am using ELK stack 7.10.0. The cluster has 7 nodes and each node is assigned approx 30 GB JVM heap.
Over the past few weeks, the indexing speed is getting extremely slow. Before that System was able to ingest 700 GB of logs in less than 20 hours, but now it's taking 3-4 days to ingest the same size of logs.
I have increased the refresh_interval time to 30s(earlier 3s) and the number of replicas to zero(earlier 1). Even after these changes Indexing is happening at the same pace, with no improvement at all. Just to give you an idea here's the performance chart for the last 3-4 days.
I have recently taken ownership of this ELK stack and hold less knowledge about this. Any help would be really appreciated.
And here's the average indexing rate for the last 3 days.
