# Logstash stopped pushing logs to elastic(UPD) **URL:** \ **Category:** Logstash\ **Created:** [July 17, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397 "2017-07-17T12:59:10Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![maslenkov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maslenkov/32/20111_2.png) [@maslenkov](https://discuss.elastic.co/u/maslenkov)\ **Post date:** [July 17, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/1 "2017-07-17T12:59:10Z")
Hey friends! My question about ELK stack. I have remote rails server. It writes two logs - production.log and reports.log. Also I didn't remove ELK's server syslog from my logstash input. So my logstash has to parse local syslog and prod.log and reports.log from remote host. It works until 27th of june O\_O. It is not a priority issue. But I have to fix it for better tomorrow;). I've started this issue 3-4 times and don't have enough patience. What I've already found: Filebeat: ```auto /usr/bin/filebeat[24911]: transport.go:125: SSL client failed to connect with: dial tcp x.x.x.x:5044: getsockopt: connection refused ``` this forum says that it is problem with logstash. When logstash running: I can connect with tellnet to this port and ns works too. But I don't see logs in elastic(kibana)... Logstash logs says next: ```auto {:timestamp=>"2017-07-14T19:00:08.690000-0400", :message=>"CircuitBreaker::rescuing exceptions", :name=>"Beats input", :exception=>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=>:warn} {:timestamp=>"2017-07-14T19:00:08.693000-0400", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} {:timestamp=>"2017-07-14T19:00:08.752000-0400", :message=>"CircuitBreaker::rescuing exceptions", :name=>"Beats input", :exception=>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=>:warn} {:timestamp=>"2017-07-14T19:00:08.757000-0400", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} ``` and many many ```auto {:timestamp=>"2017-07-14T19:00:38.821000-0400", :message=>"retrying failed action with response code: 503", :level=>:warn} ``` I can see syslog data into kibana, but not logs from remote host. Do you need some extra info? Any configs? ---
**Author:** ![maslenkov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maslenkov/32/20111_2.png) [@maslenkov](https://discuss.elastic.co/u/maslenkov)\ **Post date:** [July 17, 2017, 1:24pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/2 "2017-07-17T13:24:27Z")
Ok. I've remove syslog filter and add output rubydebug. And output looks like logstas gets data from remote: ```auto { "message" => "some json data which I can't post", ... } ``` So output looks correct. My output file(`30-elasticsearch-output.conf`) looks: ```auto output { stdout { codec => rubydebug } elasticsearch { hosts => ["localhost:9200"] sniffing => true manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } ``` And logstash logs: ```auto {:timestamp=>"2017-07-17T09:19:33.094000-0400", :message=>"CircuitBreaker::rescuing exceptions", :name=>"Beats input", :exception=>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=>:warn} {:timestamp=>"2017-07-17T09:19:33.098000-0400", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} {:timestamp=>"2017-07-17T09:19:39.212000-0400", :message=>"CircuitBreaker::rescuing exceptions", :name=>"Beats input", :exception=>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=>:warn} {:timestamp=>"2017-07-17T09:19:39.221000-0400", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} {:timestamp=>"2017-07-17T09:19:46.429000-0400", :message=>"CircuitBreaker::rescuing exceptions", :name=>"Beats input", :exception=>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=>:warn} {:timestamp=>"2017-07-17T09:19:46.433000-0400", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} ``` in the same time tail -f /var/log/elasticsearch/elasticsearch.log doesn't show new lines. I say "nothing happened". ---
**Author:** ![maslenkov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maslenkov/32/20111_2.png) [@maslenkov](https://discuss.elastic.co/u/maslenkov)\ **Post date:** [July 17, 2017, 3:02pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/3 "2017-07-17T15:02:07Z")
Ok. I've found these answer - [Lots of Beats input](https://discuss.elastic.co/t/lots-of-beats-input-the-circuit-breaker-has-detected-a-slowdown-or-stall-in-the-pipeline/52310/2?u=maslenkov) It says add `congestion_threshold` And I've added. So now logs contains only ```auto retrying failed action with response code: 503 ``` I've found [Retrying failed action with response code: 503](https://discuss.elastic.co/t/retrying-failed-action-with-response-code-503/24635/4) ugh: ```auto curl -XGET http://localhost:9200/_cluster/health ``` returns ```auto {"cluster_name":"elasticsearch","status":"red","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1...} ``` status "red" 😕 Can I fix that without removing all my data and/or indixes ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 18, 2017, 2:15pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/4 "2017-07-18T14:15:47Z")
> Can I fix that without removing all my data and/or indixes ? Maybe, but that's a question for the Elasticsearch group. ---
**Author:** ![maslenkov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maslenkov/32/20111_2.png) [@maslenkov](https://discuss.elastic.co/u/maslenkov)\ **Post date:** [July 19, 2017, 12:46pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/5 "2017-07-19T12:46:20Z")
Thank you. I think it is not a big deal as I've already removed all indices. So my question can be closed. But does it make any sense? Maybe I should remove it(question/topic)? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 16, 2017, 12:46pm UTC](https://discuss.elastic.co/t/logstash-stopped-pushing-logs-to-elastic-upd/93397/6 "2017-08-16T12:46:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.