# Filebeat seems stop send information to logstash

**URL:** <https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 18, 2016, 11:17am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133 "2016-02-18T11:17:56Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 18, 2016, 11:17am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/1 "2016-02-18T11:17:56Z")

</div>

I have Filebeat installed on three servers. It sends information to the logstash than send to the Elastic Search.

It should create a index for each day like the example bellow:

```auto
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open systemout_tmc- **2016.02.17** 5 1 146 0 350.6kb 350.6kb

```

However it is not happening. Today I've just looked to the index list and it is not created.

Here is my output file:

```auto
output {
      if [type] == "systemout_tmc" {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "systemout_tmc-%{+YYYY.MM.dd}"
        }
      }

}

```

If I restart the filebeat and Logstahs everything work fine.

Any idea what is going on?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 18, 2016, 4:26pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/2 "2016-02-18T16:26:24Z")

</div>

So after you restarted Logstash, the indexes are now created as expected? Or after restart it only creates one for the day it was restarted?

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 18, 2016, 5:04pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/3 "2016-02-18T17:04:31Z")

</div>

After I restarted the Logstash and the Filebeat the index is created as expected (one index daily).

I just did this rigth now and I got the index:

yellow open systemout\_tmc-2016.02.18 5 1 26 0 69.9kb 69.9kb

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 18, 2016, 5:27pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/4 "2016-02-18T17:27:37Z")

</div>

That means the issue is resolved?

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 18, 2016, 5:49pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/5 "2016-02-18T17:49:59Z")

</div>

No actually. Because tomorrow the index will not be create as expected as the previous days.  
I will need to restart the logstash and filebeat again. I need this proccess be automatically.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 19, 2016, 6:38am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/6 "2016-02-19T06:38:44Z")

</div>

Start by looking in the logs of Logstash and Filebeat. I'm sure there's some clue in either log.

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 19, 2016, 10:42am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/7 "2016-02-19T10:42:00Z")

</div>

I got this logs from the logstash:

{:timestamp=\>"2016-02-18T19:11:26.443000+0100", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn}

And this from filebeat:

2016-02-19T11:28:52+01:00 DBG Not harvesting, file didn't change: /opt/IBM/WebSphere/Profiles/base/logs/tmc/SystemOut.log  
2016-02-19T11:28:57+01:00 DBG Flushing spooler because of timemout. Events flushed: 0

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 19, 2016, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/8 "2016-02-19T12:14:44Z")

</div>

So, something appears to be sending Logstash a SIGTERM signal. Is that the problem then, that Logstash goes down every day?

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 21, 2016, 3:59pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/9 "2016-02-21T15:59:03Z")

</div>

I'm sorry, I sent the wrong line on the log.

The logstash is working fine when after few hours I receive this message:

{:timestamp=\>"2016-02-21T16:54:45.915000+0100", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :level=\>:warn}

And the logstash stop to receive the logs from filebeat.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 22, 2016, 2:11am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/10 "2016-02-22T02:11:37Z")

</div>

which filebeat version are you using?

This message is output by logstash if it finds the outputs/filters creating too much back-pressure and then logstash closes the connection to filebeat.

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 22, 2016, 10:48am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/11 "2016-02-22T10:48:36Z")

</div>

The filebeat version is 1.0.1 (amd64).

How can I avoid to logstash close the connections?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 22, 2016, 11:29am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/12 "2016-02-22T11:29:21Z")

</div>

Logstash closing the connections is just a symptom of the actual problem, namely that Logstash's pipeline is stalled. IIRC there was a bug in either Filebeat or Logstash that caused such situations to happen unnecessarily. Dig through the release notes of both tools and see if it might apply to you.

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 22, 2016, 4:27pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/13 "2016-02-22T16:27:20Z")

</div>

I just updated the FIlebeat to 1.1.1 and I haven't got the error again. It seems working fine now.

I will watch closely until tomorrow.

Thank you!

---

<div class="post-metadata">

**Author:** ![rafaellamuniz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafaellamuniz/32/8885_2.png) [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\
**Post date:** [February 23, 2016, 11:02am UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/14 "2016-02-23T11:02:09Z")

</div>

The update solved the problem.

Thank you very much for the help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:55pm UTC](https://discuss.elastic.co/t/filebeat-seems-stop-send-information-to-logstash/42133/15 "2017-07-05T21:55:28Z")

</div>


