Hi Team,
We are running a cluster with 3 Masters and 12 data nodes which Network storage attached to them . All the instances are running on virtual machines . After running the cluster for 2 months where we keep only 30 days older indices and 20 indices per day are created which stores only 1 TB of data for 30 days . We know there are many factors which contribute to the heap usage such as file handles per index ,filter cache etc . We are not using field data cache . But with cluster running for 2 months having heap usage kept increasing to 78 GB which slows down the searches . When cluster is restarted heap is going down to 16 GB .
Default elastic configs are used with mlock true . Can you please suggest if we need to change any settings or modify the GC configs . Currently we are using CMS collector.
Hi team please suggest on the above scenario.