Data ingestion to cluster from multiple sources

Elastic Stack Elasticsearch
1

Hello,

I am starting to test my new 3 nodes ES cluster, but it is my first experience with it.

In production cluster will receive data in real time from 7 sources to the same index.

I expect 1M entries per data source so about 7M daily.

Which is the best approach to distribute load?

I was thinking of pointing data sources to different nodes like 3 sources to master, 2 to node 2, 2 to node3.

The alternative could be to set-up a load-balancer in front of ES.

Any comment or suggestion is very appreciated.

Thanks
P.

2

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

Specifically the slides I linked to. Look from slide 49.

3

If you have 3 nodes in the cluster and they are hosted on the same hardware, I would recommend setting them up with default configuration (master/data) and distribute load evenly across them.

4

Thank you, I will go this way.

5

I will read this docs.

Thank you.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.