# Filebeat needs to be restarted in order to send logs to logstash dynamically **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 25, 2016, 6:54am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994 "2016-01-25T06:54:48Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 27, 2016, 9:46am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/21 "2016-01-27T09:46:28Z")
@ruflin So you mean I don't need to do tail\_files = true in my filebeat.yml file ???? What you mean is that it Filebeat will automatically send every line only once even if tail\_files = false ?? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 27, 2016, 9:47am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/22 "2016-01-27T09:47:03Z")
Correct ---
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 27, 2016, 9:47am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/23 "2016-01-27T09:47:30Z")
@ruflin And what if I do tail\_files = true ?? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 27, 2016, 9:48am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/24 "2016-01-27T09:48:06Z")
[https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-details.html#\_tail\_files](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-details.html#_tail_files) ---
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 27, 2016, 9:50am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/25 "2016-01-27T09:50:51Z")
@ruflin So if tail\_files = false my filebeat will read the file from the beginning but will send only the changes made (not the whole file )right ???? And with tail\_files = true Filebeat will not read the whole file but will read from the last offset of the file and then will send the logs , right?????? So overall in both the cases only the recent changes will be shipped by Filebeat , and not the whole file , right ?????? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 27, 2016, 10:28am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/26 "2016-01-27T10:28:49Z")
Correct ---
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 27, 2016, 10:34am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/27 "2016-01-27T10:34:06Z")
@ruflin thanx ruflin .............So why the use of tail\_files = true is deprecated ??? I mean what kind of loss can incurr . Please provide me a sound explanation (with example if possible). I have seen it on web but that does not sound convincing because even if log rotation is going how can logs be lost if the previous logs are not getting changed ????? I mean what ever is happening is happening with the logs that are getting added not with the previous logs so how can we have loss of logs or data ? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 27, 2016, 10:39am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/28 "2016-01-27T10:39:49Z")
tail\_files is not deprecated. Not sure where you saw that. I can't really add more details then what you already found in other issues / discuss posts. ---
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 27, 2016, 10:49am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/29 "2016-01-27T10:49:19Z")
@ruflin So i can continue with tail\_files = true without any fear of loss of data right ?? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 27, 2016, 10:53am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/30 "2016-01-27T10:53:15Z")
It is clearly stated in the docs that there is a risk of data loss. ---
**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\ **Post date:** [January 28, 2016, 5:25am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/31 "2016-01-28T05:25:15Z")
@ruflin @steffens thanx guys for being cooperative . Finally I have found the problem . The problem was in the way I was updating my file . I was doing it maunally meaning just copying and pasting the contents . Due to this the event for change was not getting fired and as a result of this Filebeat was unable to detect any changes in the file . Hence I wrote a bash script to inculcate changes in the file . And it worked !!!!!!!! ---
**Author:** ![Atul\_Patel](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@Atul\_Patel](https://discuss.elastic.co/u/Atul_Patel)\ **Post date:** [April 12, 2016, 10:21am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/32 "2016-04-12T10:21:10Z")
HI Pls find the real problem We have 2 sets of logs /folderA/app.log and /folderB/app1.log.filebeat is pushing the logs from these folders. we want to push them continuously as new log messages arrive but in sequential order which will be based on timestamp of logs getting updated in different files. Let say we have 2 files and both are getting within fraction of millisecond difference , So while receiving log line at logstash we want to maintain the logs to be received in same sequence. Which is not happening currently in filebeat. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 12, 2016, 10:24am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/33 "2016-04-12T10:24:01Z")
Please start a new thread for your question 🙂 ---
**Author:** ![Atul\_Patel](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@Atul\_Patel](https://discuss.elastic.co/u/Atul_Patel)\ **Post date:** [April 12, 2016, 10:25am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/34 "2016-04-12T10:25:08Z")
Sorry i will move this to different thread already i opened a thread for this. ---
**Author:** ![pat7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pat7/32/81768_2.png) [@pat7](https://discuss.elastic.co/u/pat7)\ **Post date:** [June 16, 2016, 10:19am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/35 "2016-06-16T10:19:11Z")
I got the same problem with filebeat. the first time I started the filebeat service it shipped all the available loginformation from the logfile to logstash (on the elastic stack server) - filebeat runs as it should be. But if the application (which produces entries in the logfile) add more lines to the logfile, filebeat dosen't ship the new data from the file. (filebeat service is running) But when i start filebeat with filebeat -c /etc/filebeat/filebeat.yml manually on the shipper, filebeat does ship the new entries. my config looks like this: (maybe i forget an entry to make filebeat ship for new values in the file?) filebeat: prospectors: - paths: ``` - /applications/IBM/WebSphere/Profiles/AppSrv01/logs/server1/info.log input_type: log document_type: waslog registry_file: /var/lib/filebeat/registry config_dir: /etc/filebeat/conf.d output: elasticsearch: enabled: false hosts: ["localhost:9200"] logstash: hosts: ["53.74.227.151:5044"] #tls: # certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"] ``` shipper: logging: files: rotateeverybytes: 10485760 # = 10MB Maybe you can help me 🙂 this is how the service looks like: patrick@was1:~\> ps -ef | grep filebeat root 3370 1 0 Jun09 ? 00:00:00 /usr/bin/filebeat-god -r / -n -p /var/run/filebeat.pid -- /usr/bin/filebeat -c /etc/filebeat/filebeat.yml root 3371 3370 0 Jun09 ? 00:11:32 /usr/bin/filebeat -c /etc/filebeat/filebeat.yml ---
**Author:** ![Dmitry\_Belyakov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitry_belyakov/32/11011_2.png) [@Dmitry\_Belyakov](https://discuss.elastic.co/u/Dmitry_Belyakov)\ **Post date:** [July 21, 2016, 3:20pm UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/36 "2016-07-21T15:20:11Z")
Same here, Having exactly the same problem as @pat7. Running as service needs a restart to pick up changes and runs fine for a couple of scan cycles. Then is starts failing to detect changes in logfile. After service restart it picks up updates from before and works for couple cycles more, then fails again. Stopping the service and running it manually with `sudo filebeat -c /etc/filebeat/filebeat.yml` works fine and does not fail after 2-3 scan cycles. P.S. I will create a separate topic for that as well. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [July 24, 2016, 9:08pm UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/37 "2016-07-24T21:08:46Z")
Which filebeat version are you using? We should also move this to a new thread as the initial problem was resolved. ---
**Author:** ![Dmitry\_Belyakov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmitry_belyakov/32/11011_2.png) [@Dmitry\_Belyakov](https://discuss.elastic.co/u/Dmitry_Belyakov)\ **Post date:** [July 24, 2016, 9:20pm UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/38 "2016-07-24T21:20:52Z")
Hey! I already did. Here's the link: [Filebeat service looses track of files (restart required)](https://discuss.elastic.co/t/filebeat-service-looses-track-of-files-restart-required/56162) ---
**Author:** ![Me\_Cloud](https://avatars.discourse-cdn.com/v4/letter/m/f4b2a3/32.png) [@Me\_Cloud](https://discuss.elastic.co/u/Me_Cloud)\ **Post date:** [September 8, 2016, 9:25am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/39 "2016-09-08T09:25:07Z")
> [@abinay](#): > > thanx guys for being cooperative . Finally I have found the problem . The problem was in the way I was updating my file . I was doing it maunally meaning just copying and pasting the contents . Due to this the event for change was not getting fired and as a result of this Filebeat was unable to detect any changes in the file . Hence I wrote a bash script to inculcate changes in the file . And it worked !!!!!!!! @abinay i have same problem with you, i need restart my filebeat to send logs to logstash. can you tell me to how fix that? thankyou ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [September 8, 2016, 10:59am UTC](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994/40 "2016-09-08T10:59:39Z")
@Me_Cloud As this problem was resolved, please open a new topic and share all your details there. Please also describe there which of the solutions here you tried. [Previous page](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994.md?page=1) [Next page](https://discuss.elastic.co/t/filebeat-needs-to-be-restarted-in-order-to-send-logs-to-logstash-dynamically/39994.md?page=3)