Consult X-pack Elasticsearch cluster production model

Elastic Stack Elasticsearch
1

I intend to deploy a cluster with X-pack consisting of:

  • eligible master-data node 1: 32 Gb RAM, 16 core cpu, SSD 1Tb
  • eligible master-data node 2: 32 Gb RAM, 16 core cpu, SSD 1Tb
    The EPS is approximately 2500 for 250 log sources and the retention rate is 1 month. I calculated I would have 40 indices per month. I have some questions:
  1. Is this resource suitable with the dimension provided above?
  2. I also have a machine installed only Kibana with X-pack, so is this possible for Kibana connect back to Elasticsearch cluster ? And Is X-pack charged for 2 nodes or 3 three nodes (including this Kibana machine without Elasticsearch) ?
  3. Beside 2 nodes, I want to build a storage server HDD 10Tb which is responsible for backing up the snapshot of old indices. Is this possible with Curator transfering data like FTP? (No elasticsearch installed)
    Thanks so much
2

The machines seem to have a good spec, but I have no idea about how much data 250 log sources may generate over a month. One potential problem is that you only have 2 nodes, which is not sufficient for high availability (at least 3 master-eligible nodes required).

You often put a coordinating-only node together with Kibana, which then acts as a load balancer. This does not count towards the node count for licensing.

If you can expose this as an NFS file system to all nodes in the cluster, you can use it for snapshot and restore. No need to have Elasticsearch installed on that host.

1 Like
3
  1. Normally, How much the memory and hardware for coordinating node ? 2Gb RAM, 50GB SSD is enough ?
  2. Then it means, my logstash output Elasticsearch directly to this node instead of 2 data nodes ? Thanks.
  3. Because of the little number of our customers, so we don't need a big cluster at the moment. In the next month, the dimension may increase and our demand to extend the storage on 2 nodes. Are there any manners to mount additional disk directly on the nodes where Elasticsearch can store the data?
4

. anyone can help me to answer these questions plz ?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.