ES best practices?

Elastic Stack Elasticsearch
1

hello

I used recently ElasticSearch I have 4 servers
and wanted to know the good pratice
how many:
node-master
node-data
how minimum_master_nodes: 4/2 + 1 = 3?
number_of_shards
number_of_replicas

because I work with a large number of log server .. almost 3000
so I have a lot of data .. 20G per day at least
I use the RAID-0

thank you

2

how ur search pattern is going to look like? How much would u retain data and till when? How much documents per second u would expect coming to ur ES? the idle shard/replica, index, etc. depend a lot on ur scenario than general practice.

1 Like
3

thank you for your reply
I used to centralize logs with ELK for 3000 servers
it's 9 pm and I have only 1800 servers connects and i have 4,132,830 docs

just for testing i used 2 servers 2 master nodes 5 shared

thank you

4

Couple of general guidelines we follow -

  1. Minimum 3 master nodes to avoid split-brain
  2. We need HA (high-availability) and in our configuration, we leverage 2 replicas for the same
  3. Too less and too much shards - Both are bad. You have to select decent number. We have multiple indices and each index have 10 primaries. We cap the data going to each index using some parameters so that they are manageable.
  4. We have 3 query nodes basically to support HA, load balancing and fault-tolerance.