ES2.0 cluster node properties and config

Elastic Stack Elasticsearch
1

Hi ES community,

Trying to setup a cluster with about 10 heterogenous blades at the moment, more to be added in the form of VMs and other blades. Have had non-cluster experience with 1.5.4, but mostly to setup proof of concepts for small scale demos.

Now the requirement is to use logstash-forwarder from 10 remote sites to redis+LS+ES. 20 people need Grafana and Kibana based dashboards.

Some clarifications:

  1. What should be the ideal characteristics of a
  • Dedicated Master Node: (eg: high CPU, small disk size, high RAM)
  • Dedicated Data Node: (eg: high speed disk, high RAM, low CPU)
  • Dedicated Client Node: (eg: High CPU, RAM etc)

  1. discovery.zen.ping.unicast.hosts: ["IP0", "IP1"]
    Should this list contain the list of all IPs in the cluster? What happens when new hosts are added? Do all the configs need to be updated?

  2. ES2.0 yml config does not seem to have the config for shards and replicas. Is it not required?
    index.number_of_shards: 5
    index.number_of_replicas: 2

  3. Is a replica only for fault tolerance, or does it speed up querying when many people are using the front end dashboards for customer queries?

Thanks in advance!

2
  1. Depends on a lot of things! Like what sort of dashboards you have and what queries are run against them, your data volumes and retention requirements, and more :slight_smile:
  2. It seems that we have removed this from the config file, but default is still 5 primary and 1 replica. It makes sense to still set this yourself.
  3. It does also improve query performance :slight_smile:
3

Thanks @warkolm!

  1. The dashboard will be showing mostly time series. The only function they will probably end up doing is zoom out and zoom in for higher/lower granularity. I dont suspect any statistical operations taking place.

Renumbering:

4.discovery.zen.ping.unicast.hosts: ["IP0", "IP1"]
Should this list contain the list of all IPs in the cluster? What happens when new hosts are added? Do all the configs need to be updated?

4

You only need a few hosts in this list, it doesn't need to be all of them and you don't need to update it. As long as new nodes can contact any one of the IPs in the list, then it will still be able to join the cluster.

5

Fantastic. Thanks!