# Filebeat hangs when logstash dies

**URL:** https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406
**Category:** Beats
**Created:** [October 30, 2015, 6:06pm UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406 "2015-10-30T18:06:39Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![erikstephens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikstephens/32/5430_2.png) [@erikstephens](https://discuss.elastic.co/u/erikstephens)
#### Post date: [October 30, 2015, 6:06pm UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406/1 "2015-10-30T18:06:39Z")

</div>

I'm seeing the connection made to the logstash server stuck in a state of CLOSE\_WAIT when logstash goes offline. My expectation is that filebeat would try to re-establish the connection periodically. Anyone else experiencing this? This is version 1.0.0-beta4.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 30, 2015, 11:00pm UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406/2 "2015-10-30T23:00:44Z")

</div>

can you run filebeat with -d '\*,publish' and check output? logstash output has configurable [timeout](https://www.elastic.co/guide/en/beats/libbeat/current/configuration.html#_timeout) option by default set to 30 seconds.

---

<div class="post-metadata">

### Author: ![erikstephens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikstephens/32/5430_2.png) [@erikstephens](https://discuss.elastic.co/u/erikstephens)
#### Post date: [October 31, 2015, 12:28am UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406/3 "2015-10-31T00:28:19Z")

</div>

This is what I'm seeing, repeated indefinitely:

> prospector.go:130: DBG Start next scan  
> prospector.go:151: DBG scan path test-\*.log  
> prospector.go:163: DBG Check file for harvesting: test-1.log  
> prospector.go:280: DBG Update existing file for harvesting: test-1.log  
> prospector.go:322: DBG Not harvesting, file didn't change: test-1.log  
> prospector.go:163: DBG Check file for harvesting: test-2.log  
> prospector.go:280: DBG Update existing file for harvesting: test-2.log  
> prospector.go:322: DBG Not harvesting, file didn't change: test-2.log  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish  
> output.go:37: DBG output worker: no events to publish

Should be simple to reproduce: stop & start logstash and you should see filebeat stuck based on output above and `lsof` for the filebeat process. Also maybe related to this, I noticed during my tests that filebeat was not detecting changes to my test files, even with a very small `harvester_buffer_size`. Based on the "file didn't change" debug output, perhaps this is the core issue? The CLOSE\_WAIT issue might be a separate.

Here is my test filebeat config:

```
shipper:
  name: test
  geoip:
    paths: []

output:
  logstash:
    enabled: true
    loadbalance: false
    hosts:
      - localhost:12345
    tls:
      disabled: true

filebeat:
  harvester_buffer_size: 128
  prospectors:
    -
      paths:
        - test-*.log
      encoding: utf-8

```

and logstash config:

```
input {
  beats {
    type => beats
    port => 12345
  }
}
output {
  stdout {
    codec => rubydebug { metadata => true }
  }
}

```

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [November 3, 2015, 7:44am UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406/4 "2015-11-03T07:44:48Z")

</div>

@erikstephens Have a look at @jhidalgo Currently defining utf-8 is breaking some reading. We are working on a fix for this. In the meantime the only solution is to remove the encoding config.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:58pm UTC](https://discuss.elastic.co/t/filebeat-hangs-when-logstash-dies/33406/5 "2017-07-05T21:58:29Z")

</div>


