# Filebeats logs -\> aws elb of logstash

**URL:** https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367
**Category:** Beats
**Tags:** filebeat
**Created:** [December 12, 2017, 2:57pm UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367 "2017-12-12T14:57:18Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![blackOcean](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blackocean/32/19722_2.png) [@blackOcean](https://discuss.elastic.co/u/blackOcean)
#### Post date: [December 12, 2017, 2:57pm UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/1 "2017-12-12T14:57:18Z")

</div>

Hi,

Using AWS elb for logstash server (3 nodes of EC2 instances), but I am getting client is unable to connect.

```
filebeat.log
2017-12-12T14:55:06Z ERR Failed to publish events caused by: client is not connected
2017-12-12T14:55:07Z ERR Failed to publish events: client is not connected
2017-12-12T14:55:07Z DBG connect 
2017-12-12T14:55:07Z DBG 3 events out of 3 events sent to logstash. Continue sending
2017-12-12T14:55:07Z ERR Failed to publish events caused by: lumberjack protocol error
2017-12-12T14:55:07Z DBG closing
2017-12-12T14:55:07Z DBG 3 events out of 3 events sent to logstash. Continue sending
2017-12-12T14:55:07Z DBG close connection
2017-12-12T14:55:07Z DBG close connection

```

Filebeat is able to send logs when I write all logstash instance manually use logstash loadbalancing rather than AWS elastic loadbalancing.

```
   hosts: ["ec2-logstash0:5043","ec2-logstash1:5043","ec2-logstash3:5043"] -- this is working with multiple instance
   loadbalance : true
   worker: 2

```

I tested with `telnet logstash-elb-amazonaws.com 5043`, Able to conect.

```
 And this with aws-elb is not working. 
 hosts: ["logstash-elb-amazonaws.com:5043"]
 loadbalance: true --this property does not affect anything
 worker: 2 -- this property also does not affect anything whether we use it or not.
```

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [December 12, 2017, 7:33pm UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/2 "2017-12-12T19:33:58Z")

</div>

What protocol did you configure the load balancer for? It should be TCP as the protocol is not HTTP based.

---

<div class="post-metadata">

### Author: ![blackOcean](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blackocean/32/19722_2.png) [@blackOcean](https://discuss.elastic.co/u/blackOcean)
#### Post date: [December 13, 2017, 9:13am UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/3 "2017-12-13T09:13:02Z")

</div>

Thanks.

I used Network load balancer with TCP. And Boom Boom !!! 🙂

It worked but I believe filebeat has some sticky session with logstash. It always sends logs to one logstash instance always. If that dies then only it choose another one and works.

I believe AWS ELB Should distribute the load on multiple logstash rather than on single instance.

Any Idea on this ?

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [December 13, 2017, 11:21pm UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/4 "2017-12-13T23:21:15Z")

</div>

Filebeat will keep its connection to Logstash alive. But you can set a TTL on the connection to allow for rebalancing. [https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#\_literal\_ttl\_literal](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#_literal_ttl_literal)

---

<div class="post-metadata">

### Author: ![blackOcean](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blackocean/32/19722_2.png) [@blackOcean](https://discuss.elastic.co/u/blackOcean)
#### Post date: [December 14, 2017, 10:41am UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/5 "2017-12-14T10:41:38Z")

</div>

Thanks 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 11, 2018, 10:41am UTC](https://discuss.elastic.co/t/filebeats-logs-aws-elb-of-logstash/111367/6 "2018-01-11T10:41:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
