# Best practice with Logstash output and ES

**URL:** https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189
**Category:** Logstash
**Created:** [July 31, 2017, 1:02pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189 "2017-07-31T13:02:30Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![lendoly](https://avatars.discourse-cdn.com/v4/letter/l/439d5e/32.png) [@lendoly](https://discuss.elastic.co/u/lendoly)
#### Post date: [July 31, 2017, 1:02pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189/1 "2017-07-31T13:02:31Z")

</div>

Hello,

My question is related to the correct way to configure the output, for Elasticsearch.

Right now my Elasticsearch cluster is one master node, 2 ingest nodes and 1 data node.

My Logstash output configuration is linked to the ingest nodes, the data is routed in base some conditions, so the data can be sent to the ingest\_node\_1 or to ingest\_node\_2 but never to both of them. This works quite fine until now.

But after check the stack, review forums and, in general, make some research, I'm starting to doubt if I must link the output to the master node instead of the ingest nodes.

Any recommendations?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 2, 2017, 12:48pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189/2 "2017-08-02T12:48:03Z")

</div>

No, do not make Logstash send data to dedicated master nodes. I think this is mentioned in the documentation.

With such a small cluster as yours I'd consider making all nodes master-eligible to avoid having one node that can take down the whole cluster.

---

<div class="post-metadata">

### Author: ![lendoly](https://avatars.discourse-cdn.com/v4/letter/l/439d5e/32.png) [@lendoly](https://discuss.elastic.co/u/lendoly)
#### Post date: [August 2, 2017, 1:23pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189/3 "2017-08-02T13:23:11Z")

</div>

The idea is growth it to 2 masters nodes, 4 ingest nodes and 4 data nodes, but for now with this cluster is enough to handle the actual volume of data (yeah, I know that the replication is not working good now because we only have 1 data node, the first thing that we will add will be other data node), that's why I was trying to make it works with this structure.

So basically I was right, Logstash needs to send the data directly to the ingest nodes, isn't it? and any way to balance the load? I was thinking to put all the ingest nodes behind a load balancer for the Logstash, but not sure if this will work well.

---

<div class="post-metadata">

### Author: ![lendoly](https://avatars.discourse-cdn.com/v4/letter/l/439d5e/32.png) [@lendoly](https://discuss.elastic.co/u/lendoly)
#### Post date: [August 2, 2017, 1:39pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189/4 "2017-08-02T13:39:39Z")

</div>

Ok re-reading the docs I found the answer:

> To configure your Logstash instance to write to multiple Elasticsearch nodes, edit the output section of the second-pipeline.conf file to read:

```auto
output {
    elasticsearch {
        hosts => ["IP Address 1:port1", "IP Address 2:port2", "IP Address 3"]
    }
}

```

> **Use the IP addresses of three non-master nodes in your Elasticsearch cluster in the host line. When the hosts parameter lists multiple IP addresses, Logstash load-balances requests across the list of addresses.** Also note that the default port for Elasticsearch is 9200 and can be omitted in the configuration above.

So I just need to add only all the ingest nodes on the output definition and do not base it on my conditions (because, basically, this was for balance the load).

Thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 30, 2017, 1:39pm UTC](https://discuss.elastic.co/t/best-practice-with-logstash-output-and-es/95189/5 "2017-08-30T13:39:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
