# Could logstash distribute data to different nodes?

**URL:** <https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085>\
**Category:** Logstash\
**Created:** [December 4, 2017, 3:32am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085 "2017-12-04T03:32:51Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\
**Post date:** [December 4, 2017, 3:32am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/1 "2017-12-04T03:32:51Z")

</div>

Hi, I want to distribute data into many nodes from logstash.  
is it feasible?  
because I want to reduce the risk. If one node crash others can share the loading.

thank you in advance !

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 4, 2017, 6:22am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/2 "2017-12-04T06:22:23Z")

</div>

Logstash instances are independent and don't cluster, but depending on what kind of data you're processing it can be more or less easy to add the kind of fault tolerance that you want. What kind of inputs do you have?

---

<div class="post-metadata">

**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\
**Post date:** [December 4, 2017, 8:25am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/3 "2017-12-04T08:25:01Z")

</div>

this is my logsatsh input:

```
input{
	tcp{
		
		port => 5510
		codec => json
		type =>"ntopng-*"
	}
	
}
	filter{
		if[type]=="ntopng-*"
		{
			if "" not in [IPV4_SRC_ADDR] and "" not in [IPV6_SRC_ADDR]
			{
				drop{}
			}
		}
	}
output{
elasticsearch {
				
                codec => "json" 
                hosts => ["localhost:9200"]
				user =>elastic
				password =>SNOOPY255262

        }
	if[type]=="ntopng-*"
	{
		stdout{codec=> rubydebug}
	}
}

```

could I distribute the log or data to different hosts?  
In order to reduce the risk that if one host crash, all system crash.

---

<div class="post-metadata">

**Author:** ![nab](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@nab](https://discuss.elastic.co/u/nab)\
**Post date:** [December 4, 2017, 8:43am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/4 "2017-12-04T08:43:01Z")

</div>

You can setup HA load balancer (Nginx or HAProxy) in front of your Elasticsearch nodes and use it for logstash output.

---

<div class="post-metadata">

**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\
**Post date:** [December 4, 2017, 8:58am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/5 "2017-12-04T08:58:45Z")

</div>

sorry, if my OS is windows 10, is it feasible?

---

<div class="post-metadata">

**Author:** ![nab](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@nab](https://discuss.elastic.co/u/nab)\
**Post date:** [December 4, 2017, 10:26am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/6 "2017-12-04T10:26:04Z")

</div>

Windows 10 doesn't look like recommended environment for production setup, but you can specify [multiple hosts for elasticsearh output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-hosts), i.e.:

```
hosts => ["hostA:9200","hostB:9200","hostC:9200"]
```

---

<div class="post-metadata">

**Author:** ![f26227279](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f26227279/32/21296_2.png) [@f26227279](https://discuss.elastic.co/u/f26227279)\
**Post date:** [December 8, 2017, 11:22am UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/7 "2017-12-08T11:22:48Z")

</div>

```
hosts => ["hostA:9200","hostB:9200","hostC:9200"]

```

if I have "A","B","C" data, then it would distribute to hostA,hostB,hostC?  
or only send to hostA,then if hostA crash, then it would send to B or C?

thank you in advance!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2017, 1:58pm UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/8 "2017-12-08T13:58:17Z")

</div>

> if I have "A","B","C" data, then it would distribute to hostA,hostB,hostC?  
> or only send to hostA,then if hostA crash, then it would send to B or C?

Quoting the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-hosts):

> If given an array it will load balance requests across the hosts specified in the `hosts` parameter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 5, 2018, 1:58pm UTC](https://discuss.elastic.co/t/could-logstash-distribute-data-to-different-nodes/110085/9 "2018-01-05T13:58:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.