# Logstash pipeline getting blocked temporary

**URL:** <https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829>\
**Category:** Logstash\
**Created:** [July 19, 2016, 6:53am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829 "2016-07-19T06:53:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [July 19, 2016, 6:53am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829/1 "2016-07-19T06:53:38Z")

</div>

I have a logstash deployment which looks like below:-

`Multiple Logstash-forwarder ----> Single Logstash Collector ----> Single Kafka Instance ----> Single Logstash Indexer ----> Elastic Search Cluster`

The logstash Collector is getting stuck with the below errors in log:-

```
{:timestamp=>"2016-07-19T02:37:45.004000-0400", :message=>"Exception in lumberjack input thread", :exception=>#<LogStash::CircuitBreaker::OpenBreaker: for Lumberjack input>, :level=>:error}
{:timestamp=>"2016-07-19T02:37:45.005000-0400", :message=>"CircuitBreaker::Open", :name=>"Lumberjack input", :level=>:warn}
{:timestamp=>"2016-07-19T02:37:45.006000-0400", :message=>"Exception in lumberjack input thread", :exception=>#<LogStash::CircuitBreaker::OpenBreaker: for Lumberjack input>, :level=>:error}
{:timestamp=>"2016-07-19T02:37:45.092000-0400", :message=>"Lumberjack input: the pipeline is blocked, temporary refusing new connection.", :level=>:warn}
{:timestamp=>"2016-07-19T02:37:45.139000-0400", :message=>"CircuitBreaker::Open", :name=>"Lumberjack input", :level=>:warn}
{:timestamp=>"2016-07-19T02:37:45.140000-0400", :message=>"Exception in lumberjack input thread", :exception=>#<LogStash::CircuitBreaker::OpenBreaker: for Lumb

```

I have to restart the `logstash-collector` then it again works for some hours and then again it blocks. I am using `logstash 1.5.3`.

Can someone let me know what is going wrong and how can I prevent this?

I am starting logstash collector as below:-

`/opt/logstash-1.5.3/bin/logstash agent -f /opt/logstash-1.5.3/conf/logstash_collector.conf -l /var/log/logstash/logstash.log -w 16`

My `logstash_collector.conf` looks like below:-

```
input {
 lumberjack {
  port => 6782
  codec => json {}
  ssl_certificate => "/opt/logstash-1.5.3/cert/logstash-forwarder.crt"
  ssl_key => "/opt/logstash-1.5.3/cert/logstash-forwarder.key"
  type => "lumberjack"
 }
 lumberjack {
  port => 6783
  ssl_certificate => "/opt/logstash-1.5.3/cert/logstash-forwarder.crt"
  ssl_key => "/opt/logstash-1.5.3/cert/logstash-forwarder.key"
  type => "lumberjack"
 }
}

filter {
  if [env] != "prod" and [env] != "common" {
    drop {}
  }
  if [message] =~ /^\s*$/ {
    drop { }
  }
}

output {
  if "_jsonparsefailure" in [tags] {
    file {
      path => "/var/log/shop/parse_error/%{env}/%{app}/%{app}_%{host}_%{+YYYY-MM-dd}.log"
    }
  } else {
    kafka {
      broker_list => ["kafka:9092"]
      topic_id => "logstash_logs2"
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 19, 2016, 10:26am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829/2 "2016-07-19T10:26:45Z")

</div>

> [@Deb](#):
>
> Logstash-forwarder

LSF is deprecated, please use beats. :)[quote="Deb, post:1, topic:55829"]  
logstash 1.5.3.  
[/quote]

Please upgrade, this is _super old_.

Have you looked at ES to make sure it is ok?

---

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [July 19, 2016, 3:23pm UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829/3 "2016-07-19T15:23:12Z")

</div>

We can not upgrade immediately. Can you let me know what could be the problem? Is the logstash overloaded? Can we tune any parameters in logstash that will help?

The flow is getting blocked in the logstash-collector itself. This is not even reaching the Kafka.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-blocked-temporary/55829/4 "2017-07-06T04:47:25Z")

</div>