Could filebeat support multipath for input and output separately?

Elastic Stack Beats
1

Hi anybody,
I would like to capture some logs with totally different fields using filebeat on windows 2008 server,and then send logs to different index in elasticsearch.
Could filebeat support multipath like this (individually):
input1->output1
input2->output2
input3->output3
...

just like routing mechanism.

1 Like
2

this is currently not supported by filebeat. For event-routing of any-kind we recommend logstash.

3

@steffens
but logstash is really big size,not lightweight, Is there a plan to improve the filebeat for event-routing function?

4

@tuankun One of the main goals of beats is to keep it lightweight. So we are very careful when adding new features. Concerning output there are several ongoing discussions on Github, for example this one here: https://github.com/elastic/beats/issues/1587 Routing is currently not on the closer roadmap.

1 Like
5

but logstash is really big size,not lightweight, Is there a plan to improve the filebeat for event-routing function?

Why not run multiple Filebeat instances? The RAM overhead of each process shouldn't be more than 20 MB or so, most of which should be shared between the processes.

6

@magnusbaeck
would you please describe the steps on how to run multiple filebeat instances? I run filebeat as a service, thanks.

7

The answer probably depends on whether you run Filebeat via systemd or as a classic SysV init script, but at least in the latter case you should be okay if you make a copy of the init script and adjust the NAME, DAEMON_ARGS, and PIDFILE variables. I haven't tried it myself.

8

@magnusbaeck
Got it, I will test.
thank you so much.

@steffens
@ruflin
Thank you two also! :slight_smile: