we using elastic search to process analytic data, we got daily basis analytic dump from our system, we need to do some analysis on analytic data, for this we push analytic data from log files to elastic search using log stash. log stash is working fine but on elastic search when doc's indexed heap is continually grow and after some hours nodes stopped due to memory issue, so we have to restart the cluster nodes, log stash using update groovy script to increase count += some values, so if docs with a id is already exist then it will update otherwise new doc is index,
right now we are not doing any kind of query from the ES cluster,
our cluster has 4 nodes, 3 data nodes with 4 cpu and 30 GB memory,
and 1 master node
and index configuration is :
60s refresh interval
please help me to resolve the heap issue......