# Filebeat dies after trying to harvest 3 times (max\_retries) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [April 23, 2019, 5:39pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086 "2019-04-23T17:39:23Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Arthur19](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur19/32/41792_2.png) [@Arthur19](https://discuss.elastic.co/u/Arthur19)\ **Post date:** [April 23, 2019, 5:39pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/1 "2019-04-23T17:39:23Z")
Hi, I have some problems when getting logs after log file haven't changed after 3 retries of harvesting, filebeat dies after this retries and I have to start manually to start harvesting. This is the log in Filebeat before it dies (this fragment appears three times in total): ```auto 2019-04-18T17:09:54.324-0700 DEBUG [input] input/input.go:152 Run input 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:174 Start next scan 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:404 Check file for harvesting: /app/logs/A/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:494 Update existing file for harvesting: /app/logs/A/api.log, offset: 2922198 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:548 File didn't change: /app/logs/A/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:404 Check file for harvesting: /app/logs/B/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:494 Update existing file for harvesting: /app/logs/B/api.log, offset: 28149 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:548 File didn't change: /app/logs/B/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:404 Check file for harvesting: /app/logs/C/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:494 Update existing file for harvesting: /app/logs/C/api.log, offset: 37930655 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:546 Harvester for file is still running: /app/logs/C/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:404 Check file for harvesting: /app/logs/D/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:494 Update existing file for harvesting: /app/logs/D/api.log, offset: 173342 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:548 File didn't change: /app/logs/D/api.log 2019-04-18T17:09:54.324-0700 DEBUG [input] log/input.go:195 input states cleaned up. Before: 4, After: 4, Pending: 0 ``` In filebeat.yml this is the configuration for process in port 5044. I changed in filebeat.yml max\_retries:3 to -1 (infinite) as follows: ```auto #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["log1.cgi-dev.ca:5044"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: # Certificate for SSL client authentication #ssl.certificate: # Client Certificate Key #ssl.key: # The number of times to retry publishing an event after a publishing failure. # After the specified number of retries, the events are typically dropped. # Some Beats, such as Filebeat and Winlogbeat, ignore the max_retries setting # and retry until all events are published. Set max_retries to a value less # than 0 to retry until all events are published. The default is 3. max_retries: -1 ``` I am getting successfully the logs from filebeat, but when the log file does not change for a time, the filebeat process dies. What would be the culprit to cause Filebeat to die? What would be the configuration might cause this problem? Should I create a daemon to ensure the filebeat process in up and start it if the process goes down? Thanks ---
**Author:** ![TiNhO](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tinho/32/41986_2.png) [@TiNhO](https://discuss.elastic.co/u/TiNhO)\ **Post date:** [April 23, 2019, 9:56pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/2 "2019-04-23T21:56:04Z")
I'm using a machine builded by bitnami. When I access the dashboard Kibana by URL, show me the things are working good but if I try output test (filebeat test output) the conection was refused. I'm using default port filebeat (5044) and it is open on machine. I'm trying figure out how to stay the conection open full time. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [April 23, 2019, 9:57pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/3 "2019-04-23T21:57:14Z")
How did you start filebeat? It's supposed to be a daemon and shouldn't just stop. If the connection is lost, filebeat just tries to reconnect until it found a stable connection. But still it should not just die in this case. ---
**Author:** ![TiNhO](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tinho/32/41986_2.png) [@TiNhO](https://discuss.elastic.co/u/TiNhO)\ **Post date:** [April 23, 2019, 9:59pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/4 "2019-04-23T21:59:02Z")
I just run: > sudo filebeat ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [April 24, 2019, 12:18pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/5 "2019-04-24T12:18:00Z")
Now I'm confused. @TiNhO how is this related to the original issue by Arthur? The `filebeat test output` is a one time action. If the remote is not available you will get a connection refused (e.g. if Logstash is not running), still filebeat can start and tries to reconnect. Please consider to open a separate issue ---
**Author:** ![TiNhO](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tinho/32/41986_2.png) [@TiNhO](https://discuss.elastic.co/u/TiNhO)\ **Post date:** [April 24, 2019, 3:05pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/6 "2019-04-24T15:05:58Z")
@steffens NVM. I found out that my logstash is dying from some reason. For that reason filebeat show me connection refused. Tnks 4 ur help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 22, 2019, 3:06pm UTC](https://discuss.elastic.co/t/filebeat-dies-after-trying-to-harvest-3-times-max-retries/178086/7 "2019-05-22T15:06:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.