Currently I have a 3 node (DATA+MASTER). One node (CLIENT)
with 24 gig RAM. The besic info of cluster now is
JVM Heap
38.7 GB / 64.0 GB
Documents 1281635203
Data1.1 TB
Indices 636
Total shards 6032
retention period 25days
shards 5
im try now to up this cluster in K8s with operator with 3 nodes master-2 data-and 2 client
is that enough for my data. and what is the best sizing for it
The first thing to note is that you have far, far too many shards in your cluster. I would recommend at leasst changing the number of primary shards to 1 to reduce this over time, but it would most likely also make sense to look into consolidating some of the indices. Try to look for an average shard size in tens of GB.
The ideal size for the new cluster will depend on how the current one is working and what the expected growth and load is. It is impossible to give any recommendations based on the information you have provided.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.