# Remote file collection (rsync, scp, ...)

**URL:** <https://discuss.elastic.co/t/remote-file-collection-rsync-scp/54063>\
**Category:** Logstash\
**Created:** [June 27, 2016, 5:12pm UTC](https://discuss.elastic.co/t/remote-file-collection-rsync-scp/54063 "2016-06-27T17:12:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![echalmer](https://avatars.discourse-cdn.com/v4/letter/e/71c47a/32.png) [@echalmer](https://discuss.elastic.co/u/echalmer)\
**Post date:** [June 27, 2016, 5:12pm UTC](https://discuss.elastic.co/t/remote-file-collection-rsync-scp/54063/1 "2016-06-27T17:12:42Z")

</div>

I need to collect log files from a closed appliance. I cannot configure the appliance to send log events out with syslog. I cannot install software on the appliance. I can copy files from the appliance, e.g. with rsync or scp.

The log files rotate from a basename (logfile.log) to an archived name (logfile\_timestamp.log).

I was planning to frequently rsync the files to a linux box and use filebeat to ship them to elastic.

However, some reading suggests that logstash/filebeat + rsync does not work reliably (e.g. due to rsync changing inode and logstash/filebeat depending on that staying the same).

For scp, "When copying a source file to a target file which already exists, scp will replace the contents of the target file (keeping the inode)." So I think that might be the non-optimal solution.

Is there another solution that I'm missing?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 28, 2016, 6:27am UTC](https://discuss.elastic.co/t/remote-file-collection-rsync-scp/54063/2 "2016-06-28T06:27:33Z")

</div>

> For scp, "When copying a source file to a target file which already exists, scp will replace the contents of the target file (keeping the inode)." So I think that might be the non-optimal solution.

Does scp truncate the file first or does it overwrite the file in place? In the latter case you'll probably be fine, otherwise you'll probably need to write a script that runs after the rsync operation and either appends new lines to the files that Filebeat monitors or overwrites them in place.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:50am UTC](https://discuss.elastic.co/t/remote-file-collection-rsync-scp/54063/3 "2017-07-06T04:50:43Z")

</div>


