Elasticsearch assumes all nodes in the cluster are equal, so having different types of hardware can cause a problem. Are you sure your cluster has formed properly and that you have set discovery.zen.minimum_master_nodes correctly according to these guidelines?
We are using logstash to append a new column to the data and push it to its corresponding node.
We tried solving the problem by dividing the data some modules will be stored in Node 2 and other modules will be stored in Node 3.
But still it doesn't solve the problem.
You seem to have a lot of shards being generated daily given the small data volumes. I would recommend reducing this significantly e.g. by changing to use only a single primary shard per index, use weekly or monthly indices or simply consolidate the data into fewer indices.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.