# Data ingestion to cluster from multiple sources

**URL:** <https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937>\
**Category:** Elasticsearch\
**Created:** [March 21, 2018, 9:01am UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937 "2018-03-21T09:01:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![paspao](https://avatars.discourse-cdn.com/v4/letter/p/7bcc69/32.png) [@paspao](https://discuss.elastic.co/u/paspao)\
**Post date:** [March 21, 2018, 9:01am UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/1 "2018-03-21T09:01:19Z")

</div>

Hello,

I am starting to test my new 3 nodes ES cluster, but it is my first experience with it.

In production cluster will receive data in real time from 7 sources to the same index.

I expect 1M entries per data source so about 7M daily.

Which is the best approach to distribute load?

I was thinking of pointing data sources to different nodes like 3 sources to master, 2 to node 2, 2 to node3.

The alternative could be to set-up a load-balancer in front of ES.

Any comment or suggestion is very appreciated.

Thanks  
P.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 21, 2018, 9:21am UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/2 "2018-03-21T09:21:33Z")

</div>

May I suggest you look at the following resources about sizing:

[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)

> **[How many shards should I have in my Elasticsearch cluster?
	  	 | Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...

> **[NetSecureDay: Managing your Black Friday Logs](https://speakerdeck.com/elastic/netsecureday-managing-your-black-friday-logs)**
>
> Surveiller une application complexe n’est pas une tâche aisée, mais avec les bons outils, ce n’est pas si sorcier. Néanmoins, des périodes fortes telles que les opérations de type « Black Friday » (Vendredi noir) ou période de Noël peuvent pousser...

Specifically the slides I linked to. Look from slide 49.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 21, 2018, 9:29am UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/3 "2018-03-21T09:29:38Z")

</div>

If you have 3 nodes in the cluster and they are hosted on the same hardware, I would recommend setting them up with default configuration (master/data) and distribute load evenly across them.

---

<div class="post-metadata">

**Author:** ![paspao](https://avatars.discourse-cdn.com/v4/letter/p/7bcc69/32.png) [@paspao](https://discuss.elastic.co/u/paspao)\
**Post date:** [March 21, 2018, 5:14pm UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/4 "2018-03-21T17:14:42Z")

</div>

Thank you, I will go this way.

---

<div class="post-metadata">

**Author:** ![paspao](https://avatars.discourse-cdn.com/v4/letter/p/7bcc69/32.png) [@paspao](https://discuss.elastic.co/u/paspao)\
**Post date:** [March 21, 2018, 5:15pm UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/5 "2018-03-21T17:15:15Z")

</div>

I will read this docs.

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2018, 5:15pm UTC](https://discuss.elastic.co/t/data-ingestion-to-cluster-from-multiple-sources/124937/6 "2018-04-18T17:15:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
