# File beat fails to send events to logstash **URL:** \ **Category:** Beats\ **Created:** [May 1, 2016, 12:08pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885 "2016-05-01T12:08:03Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 1, 2016, 12:08pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/1 "2016-05-01T12:08:03Z")
File beat is not sending events to logstash after a while. Initially when i setup the filebeat (three instances) it was successfully sending the events to logstash after a couple of hours it thrown an error message "send fails". my production structure as below Filebeat (1.2) -------------------------\> Logstash (2.0) -----------------------\> elasticserach (AWS service) If i enable only one filebeat client this works perfectly. When i enable multiple filebeats then i got the above error. I used multiline config in file beat and sending 3 different document type to logstash. i am not using ssl just sending message from filebeat to logstash via tcp 5043. Your help would be appreciated. Thanks in advance. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 1, 2016, 10:25pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/2 "2016-05-01T22:25:50Z")
> [@sarathymv](#): > > it thrown an error message "send fails". Providing the entire error would he helpful. ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 2, 2016, 1:33pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/3 "2016-05-02T13:33:14Z")
Hi Mark, Below are the error messages from filebeat and logstash Filebeat 2016-04-27T06:33:22.74-04:00 INFO backoff retry: 1m0s 2016-04-27T06:33:22.74-04:00 INFO Error publishing events (retrying): EOF 2016-04-27T06:33:22.74-04:00 INFO send fail Logstash {:timestamp=\>"2016-04-27T06:33:22.743000+0000", :message=\>"CircuitBreaker::rescuing exceptions", :name=\>"Beats input", :exception=\>LogStash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=\>:warn} {:timestamp=\>"2016-04-27T06:33:22.758000+0000", :message=\>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=\>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=\>:warn} ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 2, 2016, 5:00pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/4 "2016-05-02T17:00:37Z")
is elasticsearch even running? Is elasticsearch properly scaled? The logstash error indicates logstash output and or filtering being way to slow. Having reached a retry of 1m in filebeat indicates some bad condition being active for quite some time. ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 2, 2016, 6:34pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/5 "2016-05-02T18:34:20Z")
Hi steffen, Thanks for the reply. yes elastic search is running. Regarding scaling, please guide me as i have aws elastic search and i am new to aws as well elasticsearch. I would like to know more on your statement "Having reached a retry of 1m in filebeat indicates some bad condition being active for quite some time." what could be the bad conditions? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 2, 2016, 9:26pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/6 "2016-05-02T21:26:57Z")
which elasticsearch aws service are you using? [Elastic Cloud](https://www.elastic.co/cloud)? filebeat uses some exponential backoff on error. first 1 second, next 2 seconds, 4 seconds, 8 seconds, 16 seconds up to 1 minute. After successfully publishing events the wait timer is set back to 1 second. That is the condition of logstash not being able to send data to elasticsearch in time seems to hold for quite some time. It's most likely a problem with elasticsearch or logstash output configuration, but not beats. Check your logstash logs for any other errors. Have you checked with netstat if logstash has active connections with elasticsearch. Any documents indexed in elasticsearch? What's the indexing rate? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 2, 2016, 10:05pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/7 "2016-05-02T22:05:04Z")
Do you have Marvel installed? It'll give you insight into what your cluster is doing/ ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 3, 2016, 11:27am UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/8 "2016-05-03T11:27:45Z")
yes. AWS elastic cloud service. 1. No errors recorded in logstash error file. 2. Yes. Logstash has active connection. 3. Yes. total\_docs:570541144 4. index\_time\_in\_millis:15385738 ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 3, 2016, 11:28am UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/9 "2016-05-03T11:28:25Z")
Marvel is not installed. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 3, 2016, 12:14pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/10 "2016-05-03T12:14:16Z")
does problem persist or happens only from time to time. Monitoring/Marvel is part of [Elastic Cloud](https://www.elastic.co/cloud) I think. The main problem is logstash being slow or slowed down by output. Checking the logstash docs there is a config to change the [timeout within logsthash] ([https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-congestion\_threshold](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-congestion_threshold)). The default is 5 seconds. Try setting `congestion_threshold => 600` in your logstash config (10 minutes). ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 3, 2016, 1:21pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/11 "2016-05-03T13:21:10Z")
Thanks Steffen . I will change the logstash config and test it. Do i need to change any config in elastic search? eg,. no.of node or shards ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 3, 2016, 1:51pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/12 "2016-05-03T13:51:09Z")
No idea about your elasticsearch setup. What's exactly the name of your cluster? ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 7, 2016, 5:50am UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/13 "2016-05-07T05:50:52Z")
Hi steffen, i made the changes for congestion level and it was running fine last two days. suddenly logstash log file size increased and the service got stopped. When i investigate the log rotate is not happening when size increases. Also though logstash parse the log message successfully i could not see the messages in kibana. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 9, 2016, 10:14am UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/14 "2016-05-09T10:14:15Z")
kibana is reading data from elasticsearch. I guess elasticsearch has become unavailable, letting logstash logs go bonkers. What's your logs saying? ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 9, 2016, 12:00pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/15 "2016-05-09T12:00:37Z")
Hi stephen, As logstash log file size increases rapidly, i disable the logstash log under /etc/init.d/logstash file. I think logrotate is not happening based on size, it happens everyday i believe. As i disable the log, my logstash running smoothly. But i am not able to view the data in kibana. As you said i will check the elasticsearch and get back to you. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [May 9, 2016, 12:19pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/16 "2016-05-09T12:19:40Z")
why not configure `size` in you log-rotate config. See [manpage](http://www.linuxcommand.org/man_pages/logrotate8.html) ---
**Author:** ![sarathymv](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@sarathymv](https://discuss.elastic.co/u/sarathymv)\ **Post date:** [May 9, 2016, 1:40pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/17 "2016-05-09T13:40:42Z")
Hi stephen, I checked the elastic search, there is no disc space available and hence not able to view data in kibana. Thanks for your guidance. Actually logstash writes all the out put log file, as the elasticsearch disc space is not available that is the root cause. Now all things are working fine. I will monitor for couple days and share you the status. Thanks for your patience and timely support. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:52pm UTC](https://discuss.elastic.co/t/file-beat-fails-to-send-events-to-logstash/48885/18 "2017-07-05T21:52:13Z")