# The Pipeline is blocked **URL:** \ **Category:** Beats\ **Created:** [July 11, 2016, 6:55pm UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221 "2016-07-11T18:55:34Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![mits](https://avatars.discourse-cdn.com/v4/letter/m/ac91a4/32.png) [@mits](https://discuss.elastic.co/u/mits)\ **Post date:** [July 11, 2016, 6:55pm UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/1 "2016-07-11T18:55:34Z")
Having some trouble resolving this issue on my own. Here is the error at the end of my logstash.log {:timestamp=\>"2016-07-11T14:20:06.631000-0400", :message=\>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=\>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:06.951000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:07.452000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:07.953000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:08.454000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:08.955000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:09.460000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:09.961000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:10.462000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:10.963000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:11.464000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:11.968000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} {:timestamp=\>"2016-07-11T14:20:12.469000-0400", :message=\>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect\_backoff\_sleep=\>0.5, :level=\>:warn} My logstash config files are: 02-beats-input.conf input { beats { type =\> beats port =\> 5044 congestion\_threshold =\> "40" } } 10-syslog-filter.conf: filter { if [type] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" } add\_field =\> ["received\_at", "%{@timestamp}"] add\_field =\> ["received\_from", "%{host}"] } syslog\_pri { } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } 30-elasticsearch-output.conf: output { elasticsearch { hosts =\> ["localhost:9200"] sniffing =\> true manage\_template =\> false index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } stdout { codec =\> rubydebug } } Please let me know if you need any more information. I saw that there was a similar post made to which adding the http protocol to the output plugin which actually caused my LS to stop working as far as I could tell. ---
**Author:** ![mits](https://avatars.discourse-cdn.com/v4/letter/m/ac91a4/32.png) [@mits](https://discuss.elastic.co/u/mits)\ **Post date:** [July 11, 2016, 6:59pm UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/2 "2016-07-11T18:59:00Z")
It redid my formatting so just assume everything is tabbed correctly. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 11, 2016, 8:04pm UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/3 "2016-07-11T20:04:21Z")
What kind of message rates are you seeing? Don't rule out the possibility that Elasticsearch isn't capable of accept the amount of messages you're trying to send to it. > I saw that there was a similar post made to which adding the http protocol to the output plugin which actually caused my LS to stop working as far as I could tell. That advice was applicable for Logstash pre-2.0. > It redid my formatting so just assume everything is tabbed correctly. If you format your configuration as code using the `` button you won't have that problem. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [July 12, 2016, 10:00am UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/4 "2016-07-12T10:00:35Z")
@mits This post should probably be moved into the logstash forum? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [July 12, 2016, 10:42am UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/5 "2016-07-12T10:42:12Z")
the beats input plugin in logstash has a circuit-breaker with default of 5-seconds. If logstash output can not deal with amount of data or filters are too slow (e.g. grok basically being a regex), the pipeline in logstash becomes congested and the circuit breaker will start closing connections. Connection timeout in logstash is configured via [congestion\_threshold](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-congestion_threshold). Setting this to a very very large value (e.g. a few months), basically disables the circuit breaker in logstash. There is still a request [timeout](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#_timeout_2) in beats reconnecting if logstash can not handle a batch of events in time. The default timeout is 30 seconds. Increase to whatever makes sense for you. I see you've set congestion\_threshold to 40 seconds already. How did you choose this value? Did you do any throughput testings with and without filters to get some baseline numbers? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 1, 2016, 6:55pm UTC](https://discuss.elastic.co/t/the-pipeline-is-blocked/55221/6 "2016-08-01T18:55:34Z")
This topic was automatically closed after 21 days. New replies are no longer allowed.