# Regarding Filebeat Ouput to a File in Remote Server or NFS

**URL:** https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038
**Category:** Beats
**Created:** [April 1, 2016, 9:54am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038 "2016-04-01T09:54:58Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 1, 2016, 9:54am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/1 "2016-04-01T09:54:58Z")

</div>

I want to output data from filebeat to a file in NFS on remote server. Please suggest how we can do that.

I have found the option to forward data using following configuration . But how we can we forward the data to file remote server.

```auto
# File as output
  # Options:
  # path: where to save the files
  # filename: name of the files
  # rotate_every_kb: maximum size of the files in path
  # number of files: maximum number of files in path
  file:
    path: "/tmp/filebeat"
    filename: filebeat
    rotate_every_kb: 1000
    number_of_files: 7

```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 1, 2016, 10:00am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/2 "2016-04-01T10:00:02Z")

</div>

Mount the NFS volume on the host that runs Filebeat and change /tmp/filebeat to an NFS-mounted path.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 1, 2016, 10:43am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/3 "2016-04-01T10:43:21Z")

</div>

Thanks , Is it possible to ouput file to a file in a remote server.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 1, 2016, 10:50am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/4 "2016-04-01T10:50:10Z")

</div>

Once Filebeat forwards logs to this file , I understand that logstash data from this file by configure the file path as input and also whether logstash can ouput to a file after applying the filters and then elastic read it from a particular fiel.

How I can configure elastic search to read the input from file. I hope my query is clear.

Filebeat --\> file in NFS volume --\> Logstash --\> File --\> Elastic Search . I'm trying this is due to some firewall issues which doesn't allow to ouput directly to logstash and elastic search .

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 1, 2016, 11:14am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/5 "2016-04-01T11:14:20Z")

</div>

Elasticsearch can't currently read from files in the way you describe. But why not run Logstash on the Elasticsearch machine?

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 4, 2016, 6:59am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/6 "2016-04-04T06:59:37Z")

</div>

Can filebeat write into the specified server where the nfs mount is available? If yes, what is the port used by filebeat?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 4, 2016, 7:13am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/7 "2016-04-04T07:13:38Z")

</div>

If you mount an NFS volume any program can access the files in the volume just like they were on a local disk (well, file permissions still come into play). In other words yes, Filebeat can write files there and there's no port number involved.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 4, 2016, 8:45am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/8 "2016-04-04T08:45:01Z")

</div>

Thanks for the information. To be more specific on my question , We are not able to mount filebeat to a server where logs are present . The NFS would be mounted on a different server and fiebeat should be able to write to that server where NFS mount is available . Would it be possible.

One more question , would it be possible for logstash to read the input from file where filebeat has written the data in beats format. If so what should be the input format.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 4, 2016, 12:17pm UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/9 "2016-04-04T12:17:58Z")

</div>

Please respond to me at the earliest on this.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 4, 2016, 5:16pm UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/10 "2016-04-04T17:16:23Z")

</div>

I, like many people here, volunteer with my time and therefore respond at my leisure. If that's not good enough for you there are paid services.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 5, 2016, 4:25am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/11 "2016-04-05T04:25:37Z")

</div>

Sorry , Your support is very beneficial you , As i got your initial response very fast i thought you will be responding soon. Please take you own time and respond to us. I was expected some one in the group to respond to the query and it is very helpful for us.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 5, 2016, 9:32am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/12 "2016-04-05T09:32:49Z")

</div>

NFS is something supported/provided by the OS and fully transparent to it's users/processes. You can not 'mount filebeat'. You have to mount the disk on remote via NFS into local directory tree. Filebeat will be configured like writing to local disk, but using the remote mount point (your config is already correct, just update the `path` to your mount point).

With NFS being so common just google for ['linux mount nfs'](https://www.google.de/search?rls=en&q=linux+mount+nfs&ie=UTF-8&oe=UTF-8). Better, add your distribution name to the search to find some administrator resources/handbook. Some results from [redhat/centos](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/System_Administration_Guide/s1-nfs-mount.html), [ubuntu wiki](https://help.ubuntu.com/community/SettingUpNFSHowTo), [archlinux](https://wiki.archlinux.org/index.php/NFS).

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 5, 2016, 10:00am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/13 "2016-04-05T10:00:26Z")

</div>

There is a typo error , what i meant is we are not able to mount NFS on the server where File beat is installed.

Thanks a lot for the information . So what i understand is Filebeat can write to NFS file mounted on a remote server by configuring the "path" to the mount point. I'll go through the links to understand more on the NFS.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 5, 2016, 10:55am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/14 "2016-04-05T10:55:01Z")

</div>

maybe your firewall is blocking NFS? Wrong credentials? Which NFS version are you using in your environment. Pre v4 does not encrypt network traffic.

Shared disk are kinda tricky. While it's possible you have to check in some test environment what happens if NFS server becomes unavailable (e.g. start dropping packets via firewall rule and restart much later). What happens if NFS becomes unavailable, machine running filebeat is restarted (where does buffering happen)? Any data loss in any of these scenarios? When writing to NFS, filebeat fully relies on NFS for not dropping any data as filebeat can not detect any network problems. Advantage of filebeat-\>logstash is support for encryption via TLS + detection of network failures with support for re-sending lost log lines.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 5, 2016, 12:35pm UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/15 "2016-04-05T12:35:56Z")

</div>

Thanks for the information. The filebeat is installed in a chassis from where logs has to be forwarded and there the NFS mount is not possible on chassis. I'm planning to come up the design for forwarding logs from chassis logs from chassis to Elastic search as there is firewall blocking direct file transfer from logstash to elastic search. So the plan was to write to NFS from there elasticsearch reads it.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 5, 2016, 3:17pm UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/16 "2016-04-05T15:17:14Z")

</div>

elasticsearch can not read from file. You need some process to push your logs (e.g. logstash)

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 7, 2016, 4:26am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/17 "2016-04-07T04:26:15Z")

</div>

Thanks for the response . Okay then please tell if the following design is fine.  
filebeat  
Chassis --\>logstash--\>NFS--\>logstash--\>ELK Server  
Firewall

Please confirm if logstash can read from NFS and forward to ELK.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 7, 2016, 5:38am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/18 "2016-04-07T05:38:37Z")

</div>

Again, Logstash doesn't care if an input file resides on an NFS-mounted volume. That said, networked file systems are notorious for having different edge case behavior and Logstash is usually not used for reading files from NFS. I'd try it out thoroughly before committing to anything.

---

<div class="post-metadata">

### Author: ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)
#### Post date: [April 7, 2016, 6:23am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/19 "2016-04-07T06:23:18Z")

</div>

Thanks for the information.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 7, 2016, 11:37am UTC](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/20 "2016-04-07T11:37:32Z")

</div>

File handling reading/writing are pretty similar in filebeat/logstash and any other product on market you will find. NFS is no 'communication channel' per se, but remote 'disk'. If communication is done via files one can use NFS (as service is provided transparently by OS and services running on OS).

Why have another logstash instance inbetween writing to disk?

[Next page](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038.md?page=2)
