# Filebeat -\> Logstash "write: broken pipe"

**URL:** <https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077>\
**Category:** Beats\
**Created:** [March 22, 2016, 8:11am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077 "2016-03-22T08:11:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![gregoryo](https://avatars.discourse-cdn.com/v4/letter/g/34f0e0/32.png) [@gregoryo](https://discuss.elastic.co/u/gregoryo)\
**Post date:** [March 22, 2016, 8:11am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/1 "2016-03-22T08:11:16Z")

</div>

I'm having trouble getting TLS communication working between Filebeat and Logstash. I have followed @andrewkroh 's [advice](https://gist.github.com/andrewkroh/fdc7e5f3f0f0ed63a11c) and got this far:

```
$ curl -v --cacert ca.crt https://logs.andrewkroh.com:5044
$ curl: (52) Empty reply from server

```

Starting Filebeat (which works with TLS turned off for both Logstash and Filebeat), however, results in this error:

```
2016/03/22 07:48:32.846449 publish.go:88: INFO Start sending events to output
2016/03/22 07:48:32.846759 log.go:113: INFO Harvester started for file: /var/log/messages
2016/03/22 07:48:35.360607 single.go:76: INFO Error publishing events (retrying): write tcp 192.168.0.56:51704->192.168.0.56:5043: write: broken pipe
2016/03/22 07:48:35.360650 single.go:152: INFO send fail 
2016/03/22 07:48:35.360666 single.go:159: INFO backoff retry: 1s

```

Can anyone offer assistance with this? Tomorrow I'll dig out tcpdump again and see if I can understand what's going wrong, but some tips would be appreciated.

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [March 22, 2016, 8:35am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/2 "2016-03-22T08:35:19Z")

</div>

@gregoryo Generally the “write: broken pipe” problem is found when there is a connection lost between you and logstash

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 22, 2016, 11:32am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/3 "2016-03-22T11:32:37Z")

</div>

can you check logstash logs too?

---

<div class="post-metadata">

**Author:** ![gregoryo](https://avatars.discourse-cdn.com/v4/letter/g/34f0e0/32.png) [@gregoryo](https://discuss.elastic.co/u/gregoryo)\
**Post date:** [March 22, 2016, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/4 "2016-03-22T14:53:32Z")

</div>

Starting logstash with this...

```
$ logstash -f /usr/local/etc/logstash/logstash.conf --debug

```

... nothing seems to happen in the output in response to the startup and attempted connection by filebeat. This line is output roughly every one second, regardless of what filebeat is doing:

```
Flushing buffer at interval {:instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x2729ffc2 @operations_mutex=#<Mutex:0x29ee1557>, @max_size=500, @operations_lock=#<Java::JavaUtilConcurrentLocks::ReentrantLock:0x2dd194b2>, @submit_proc=#<Proc:0xec2a5b@/usr/local/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.1.4-java/lib/logstash/outputs/elasticsearch/common.rb:54>, @logger=#<Cabin::Channel:0x59f1d3bc @metrics=#<Cabin::Metrics:0x7824b834 @metrics_lock=#<Mutex:0x8fce023>, @metrics={}, @channel=#<Cabin::Channel:0x59f1d3bc ...>>, @subscriber_lock=#<Mutex:0x69a3bd95>, @level=:debug, @subscribers={12450=>#<Cabin::Outputs::IO:0x632015d1 @io=#<IO:fd 1>, @lock=#<Mutex:0xcb21c6e>>}, @data={}>, @last_flush=2016-03-22 22:35:55 +0800, @flush_interval=1, @stopping=#<Concurrent::AtomicBoolean:0x78894778>, @buffer=[], @flush_thread=#<Thread:0x19136d8 run>>", :interval=>1, :level=>:info, :file=>"logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 22, 2016, 6:09pm UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/5 "2016-03-22T18:09:28Z")

</div>

> (I have replaced @ with @' in the above, since the forum thought I was trying to mention more than 10 users).

If you format the log snippet as code this won't be a problem.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 22, 2016, 11:55pm UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/6 "2016-03-22T23:55:01Z")

</div>

can you shre you filebeat and logstash config?

---

<div class="post-metadata">

**Author:** ![gregoryo](https://avatars.discourse-cdn.com/v4/letter/g/34f0e0/32.png) [@gregoryo](https://discuss.elastic.co/u/gregoryo)\
**Post date:** [March 23, 2016, 2:12am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/7 "2016-03-23T02:12:56Z")

</div>

logstash.conf:

```
input {
  beats {
    port => '5043'
    ssl => true
    ssl_certificate => '/var/ck-logstash/ssl/certs/HOSTNAME.pem'
    ssl_key => '/var/ck-logstash/ssl/private/HOSTNAME.key'
  }
}

output {
  stdout {}
}

```

filebeat.yml:

```
filebeat:
  prospectors:
    -
      paths:
        - '/var/log/messages'
      fields:
        type: syslog

output:
  logstash:
    hosts: ['HOSTNAME:5043']
  tls:
    certificate_authorities: /var/ck-logstash/ssl/cacert.pem
    certificate: /var/ck-logstash/ssl/certs/HOSTNAME.pem
    certificate_key: /var/ck-logstash/ssl/private/HOSTNAME.key
    timeout: 15

```

It works fine (although clearly without security) when I comment out 'ssl =\> true' line and 'tls' section.

Also, here's snippets of the curl details again (from @andrewkroh's reference above):

```
$ curl -v --cacert /var/ck-logstash/ssl/cacert.pem https://HOSTNAME:5043
* SSL certificate verify ok.
curl: (52) Empty reply from server

```

And hey, it looks like I managed to make the 'Preformatted text' button work this time. I **thought** that was the right one when I was first trying.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [March 23, 2016, 2:30am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/8 "2016-03-23T02:30:50Z")

</div>

Try this:

```auto
output:
  logstash:
    hosts: ['HOSTNAME:5043']
    tls:
      certificate_authorities: [/var/ck-logstash/ssl/cacert.pem]

```

All your `tls` options need right shifted by two spaces. `certificate_authorities` is a list. And `certificate` and `certificate_key` are only needed if you are doing mutual auth (authenticating the Filebeat client).

---

<div class="post-metadata">

**Author:** ![gregoryo](https://avatars.discourse-cdn.com/v4/letter/g/34f0e0/32.png) [@gregoryo](https://discuss.elastic.co/u/gregoryo)\
**Post date:** [March 23, 2016, 2:43am UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/9 "2016-03-23T02:43:14Z")

</div>

Excellent, thank you. Problem fixed. Another read of [https://www.elastic.co/guide/en/beats/filebeat/1.1/configuration-output-tls.html](https://www.elastic.co/guide/en/beats/filebeat/1.1/configuration-output-tls.html) was what I needed!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:54pm UTC](https://discuss.elastic.co/t/filebeat-logstash-write-broken-pipe/45077/10 "2017-07-05T21:54:16Z")

</div>