# Transport protocol not working for multiple hosts

**URL:** <https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292>\
**Category:** Logstash\
**Created:** [September 28, 2015, 8:23pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292 "2015-09-28T20:23:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![bennett](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennett/32/4903_2.png) [@bennett](https://discuss.elastic.co/u/bennett)\
**Post date:** [September 28, 2015, 8:23pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/1 "2015-09-28T20:23:41Z")

</div>

can someone tell me why this isn't communicating with my elasticsearch cluster? if i put one host in the host field it communicates but not with multiple.

output {  
elasticsearch {  
index =\> "logstash-%{+YYYY.MM.dd}"  
host =\> ["xxx.xxx.xxx.10", "xxx.xxx.xxx.11", "xxx.xxx.xxx.12", "xxx.xxx.xxx.13", "xxx.xxx.xxx.14", "xxx.xxx.xxx.15", "xxx.xxx.xxx.16", "xxx.xxx.xxx.17", "xxx.xxx.xxx.18"]  
cluster =\> "cluster1"  
port =\> "9300"  
sniffing =\> true  
protocol =\> "transport"  
workers =\> 5  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 29, 2015, 5:53am UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/2 "2015-09-29T05:53:40Z")

</div>

What do the logs say? I'd be surprised if they were silent.

---

<div class="post-metadata">

**Author:** ![andrewvc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewvc/32/5064_2.png) [@andrewvc](https://discuss.elastic.co/u/andrewvc)\
**Post date:** [September 29, 2015, 8:30am UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/3 "2015-09-29T08:30:59Z")

</div>

Try switching the protocol to 'http', which will be the default in logstash2.0 . Debugging is MUCH easier, and performance is almost identical to transport.

---

<div class="post-metadata">

**Author:** ![andrewvc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewvc/32/5064_2.png) [@andrewvc](https://discuss.elastic.co/u/andrewvc)\
**Post date:** [September 29, 2015, 8:32am UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/4 "2015-09-29T08:32:12Z")

</div>

Oh, and change the port to 9200 if you do that as well. Working with transport / node is much trickier! I only recommend it for ES experts at this point given the number of ways it can be accidentally misconfigured.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [September 29, 2015, 12:11pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/5 "2015-09-29T12:11:19Z")

</div>

@bennett

The use of multiple hosts entries (the array, as you have it) should round-robin between different hosts, at least with `protocol => http` and it should also with `protocol => transport`. However, as Andrew pointed out, we recommend _not_ using transport protocol, or even node protocol. Testing has shown that the http output is as fast as node and transport, and can even be faster when using round-robining like this. Node protocol will not be the default behavior starting with Logstash 2.0, but rather it will use http. Best to get used to that now.

---

<div class="post-metadata">

**Author:** ![andrewvc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewvc/32/5064_2.png) [@andrewvc](https://discuss.elastic.co/u/andrewvc)\
**Post date:** [September 29, 2015, 12:43pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/6 "2015-09-29T12:43:55Z")

</div>

Often times, sadly, the logs will be silent unless a log4j.properties file is present when using the Transport/Node protocols. One of the reasons we're now recommending HTTP as the default, which does not have this problem 😄

---

<div class="post-metadata">

**Author:** ![bennett](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennett/32/4903_2.png) [@bennett](https://discuss.elastic.co/u/bennett)\
**Post date:** [September 29, 2015, 3:01pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/7 "2015-09-29T15:01:41Z")

</div>

So i used http protocol and that works just fine. But we were looking into something that could load balance and found the node an transport protocols.

But are you guys saying that http load balances just like the node and transport protocols? and they have the same performance? Are the node and transport protocols going to be deprecated?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [September 29, 2015, 3:11pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/8 "2015-09-29T15:11:38Z")

</div>

> [@bennett](#):
>
> But are you guys saying that http load balances just like the node and transport protocols?

Specifying multiple hosts in your config, e.g. `host => ['host1', 'host2', 'host3']` will round robin _bulk requests_ to each of these hosts. It will send 500 events to the first host, then 500 to the second, then the same with the third. As such, it is more efficient because it distributes the client load between 3 clients as opposed to one (each node is a single client). Using multiple hosts with `protocol => node` will result in multiple nodes being spun up. This is definitely suboptimal and a bad idea.

> [@bennett](#):
>
> and they have the same performance?

Distributing via round robin like this _will_ give a performance boost over node protocol. The node client cannot do this. We've measured internally with single node and http clients and found that the http client is at least as performant as node, if not more so in many situations.

> [@bennett](#):
>
> Are the node and transport protocols going to be deprecated?

The short answer is "probably," though not immediately. We are discussing this internally. It is desired to eliminate the transport protocol in favor of http as http is easier to secure. Node and transport both use the transport protocol (this is the Elasticsearch sense of the word, rather than the Logstash one). Node and transport will persist for now, but it would be wiser to switch to http sooner rather than later.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [September 29, 2015, 3:24pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/9 "2015-09-29T15:24:38Z")

</div>

It's also important to understand that "load balancing" at the node/transport level is a bit different from what you may understand when hearing that word. There's _distributing,_ and _load balancing._ By _distributed,_ I mean that the documents are hashed and assigned assigned a document id by Elasticsearch, and the shard number determined by the result of a simple mathematical operation on that hash. The log line, or "document" goes to its calculated shard. Distributed, not load balanced, though it may externally appear similar. Indeed it may be considered a _form_ of load balancing due to sharding, though this applies to all documents indexed irrespective of how the documents got there.

When using `protocol => node`, Logstash launches a local Elasticsearch client-only node. Logstash puts all requests there, and they are _distributed_ across the shards and nodes in your Elasticsearch cluster.

When using `protocol => transport`, Logstash sends the request to an Elasticsearch client node via the transport protocol. That client node _distributes_ across the shards and nodes in your Elasticsearch cluster.

When using `protocol => http`, Logstash sends the request to an Elasticsearch client node via the http protocol. That client node _distributes_ across the shards and nodes in your Elasticsearch cluster.

At no point do any of these options actually do load balancing, except when multiple `host` entries are present with either `protocol => http` or `protocol => transport`. Even then, it's strictly round-robin distribution of bulk queries around the clients.

---

<div class="post-metadata">

**Author:** ![bennett](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennett/32/4903_2.png) [@bennett](https://discuss.elastic.co/u/bennett)\
**Post date:** [September 29, 2015, 3:35pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/10 "2015-09-29T15:35:41Z")

</div>

Oh wow! Thanks for clarifying and the help! I really appreciate it. This is definitely great and useful information.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292/11 "2017-07-06T05:27:46Z")

</div>


