How to Best define the shards and replica in Elasticsearch

Elastic Stack Elasticsearch
1

Hello ELK Experts,

I would like to seek your expert opinion for my use case where i have 3 node ELK cluster setup and each not is also serving as a data node.

Below is my elasticsearch.yml file.

# cat /etc/elasticsearch/elasticsearch.yml
# Elasticsearch config
#########################
cluster.name: log-cohort
node.name: xprdc01
#node.master: true
path:
    data: /data/lib/elasticsearch
    logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["xprdc01", "xprdc02", "xprdc03"]
xpack.security.enabled: false
#bootstrap.mlockall: true
#######################################################
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 93%
cluster.routing.allocation.disk.watermark.high: 95%

Below is Just a screen shot of the details of indices, shards and documents and data size.

image
image.png1914×219 27.2 KB

2

You probably have too many shards per node.

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right

3

Thank you so much, appreciate your help, i'll go through these videos.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.