# Stopped Filebeat for 8 hours and old logs did not get pushed out **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 18, 2017, 4:55pm UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279 "2017-12-18T16:55:55Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![ramzey1981](https://avatars.discourse-cdn.com/v4/letter/r/d26b3c/32.png) [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Post date:** [December 18, 2017, 4:55pm UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279/1 "2017-12-18T16:55:55Z")
hello team, so i have filebeat 6.0 and our application is logging to a single file. Last night i put filebeat in debug mode to troubleshoot an issue and then I killed the debug session and did not resume the filebeat process. This morning when i resumed filebeat i was hoping that all the old logs from our application would get pumped into logstash and I would see them in kibana but filebeat seems to have lost its pointer in the file and is now is only pushing new log lines into kibana. I tried replication the issue by stopping filebeat and generating new logs and then starting filebeats to see if I can reproduce but filebeat is behaving the way it shoudld be. Is this an issue with the length of time filebeat was down and then started hence the reason it generated a new pointer? Please let me know if this is a setting i need to adjust? thanks ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [December 19, 2017, 4:26am UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279/2 "2017-12-19T04:26:19Z")
As you are writing all logs to 1 file, I wonder if you ever truncate the file because it's getting too big? Or is it ever growing? What do you have set for your `clean_*` and `close_*` options? Please share your config. ---
**Author:** ![ramzey1981](https://avatars.discourse-cdn.com/v4/letter/r/d26b3c/32.png) [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Post date:** [December 19, 2017, 4:04pm UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279/3 "2017-12-19T16:04:15Z")
hey Ruflin, my filebeat config is as follows which reads yml files from conf.d folder, filebeat: config\_dir: /etc/filebeat/conf.d config.modules: enabled: true path: /etc/filebeat/modules.d/\*.yml output.redis: # Array of hosts to connect to. hosts: ["logging.service.consul:6379"] ssl.enabled: false they yml file in question in conf.d is , [https://pastebin.com/iiMpUdQg](https://pastebin.com/iiMpUdQg) in particular the firs log path - /var/log/iam-service/rest-api/\* which has the following log files in its directory and they never get rotated since they grow in time. When the issue comes up there are only about 70 log lines in there or less -rw-r--r--. 1 iam iam 0 Dec 10 23:37 iam\_critical -rw-r--r--. 1 iam iam 24311 Dec 14 07:13 iam\_error -rw-r--r--. 1 iam iam 1980911 Dec 14 22:17 iam\_info to solve this issue of the logs not making it once i deleted the registry file and restarted filebeat and all the logs came in. The second time the issue occurred i did the same thing a few times but the logs did not make it to kibana so i manually added a log entry and then all the logs came in at once and many duplicates since i deleted the registry fiel a few times to debug. In terms of your questions for clean\_\* and close\_\* I have all the defaults that are set with filebeat 6.0 . I also checked to see if elasticsearch was dropping the logs or logstash and I dont see any errors. I also believe that they are not dropping the logs becasue the logs are eventually making it to kibana. thanks ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [December 27, 2017, 4:00am UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279/4 "2017-12-27T04:00:37Z")
Can you share some details on how the log files are written? Please also share the filebeat log file with debug enabled. There should be quite a bit of additional info inside which should help us solve this issue. Note: `filebeat.config_dir` is deprecated and `filbeat.config.prospector` should be used: [https://github.com/elastic/beats/pull/5321](https://github.com/elastic/beats/pull/5321) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 24, 2018, 4:00am UTC](https://discuss.elastic.co/t/stopped-filebeat-for-8-hours-and-old-logs-did-not-get-pushed-out/112279/5 "2018-01-24T04:00:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.