# How to configure prospectors when files are rsynced?

**URL:** <https://discuss.elastic.co/t/how-to-configure-prospectors-when-files-are-rsynced/63590>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 21, 2016, 9:13am UTC](https://discuss.elastic.co/t/how-to-configure-prospectors-when-files-are-rsynced/63590 "2016-10-21T09:13:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Torsten\_Reinhard](https://avatars.discourse-cdn.com/v4/letter/t/46a35a/32.png) [@Torsten\_Reinhard](https://discuss.elastic.co/u/Torsten_Reinhard)\
**Post date:** [October 21, 2016, 9:13am UTC](https://discuss.elastic.co/t/how-to-configure-prospectors-when-files-are-rsynced/63590/1 "2016-10-21T09:13:14Z")

</div>

Hi,

I´m new to ELK as well as to filebeat. Due to some restrictions we cannot install filebeat running on the source server(s), so we synchronize (rsync) all files regulary to the machine where filebeat is running.  
After starting the sync, filebeat starts reading the whole file again - it´s not continuing at the latest offset.

How do I have to setup filebeat, so it remembers the last read position - and continues in processing only the new entries ?

Thanx for any hint,

Torsten

---

<div class="post-metadata">

**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\
**Post date:** [October 21, 2016, 10:38am UTC](https://discuss.elastic.co/t/how-to-configure-prospectors-when-files-are-rsynced/63590/2 "2016-10-21T10:38:57Z")

</div>

Filebeat stores the last-read offset by default, but I suspect that the way rsync works (at least by default) breaks some of the assumptions it makes. In particular, Filebeat will think a file is new if:

- the device/inode changed, even if the file name is the same
- the file size is smaller than the current offset stored

I would recommend trying the `--append` or `--append-verify` option of rsync.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 11, 2016, 9:13am UTC](https://discuss.elastic.co/t/how-to-configure-prospectors-when-files-are-rsynced/63590/3 "2016-11-11T09:13:22Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.