How to setup logstash-forwarder to read recursively on a path?

Elastic Stack Logstash
1

Please helo me understand how to setup logstash-forwarder to read recursively on a path.

For example if the file I want logstash forwarder is build.xml and it can be anywhere in the directory structure "/foo/bar/jobs/**/builds/*/build.xml where the 'foo', 'bar' and 'jobs' and 'builds' are the fixed directories names.
The valid paths can be foo/bar/jobs/xyz/abc/builds/11/build.xml , foo/bar/jobs/xyz/abc/def/ghe/builds/12/build.xml , foo/bar/jobs/lmn/opq/jbz/abc/builds/91/build.xml

2

You can use wildcards there, are you having problems if you use them?

3

Yes I tried using multiple combinations but nothing is working. For example I use the below pattern but it doesn't work. /foo/bar/jobs/**/builds/*/build.xml. Here ** and * can be any number of subdirectories in the path till build.xml file from Jenkins.

4

Well each * is considered a single directory, not one or more. If there are multiple directories there you need to add more */ entries.

5

Thank you for the reply.
Actually the number of subdirectories can differ from build to build. Sometime the it could be at 2 level deep, sometime at 5 levels deeps, etc.

I believe this is a limitation of logstash forwarder that it cannot monitor the subfolders recursively for logs files creations.

Can anybody please cofirm or is there a way?

6

Is there anyone knows how to setup logstash-forwarder to read recursively on a path? This is show-stopper for using ELK.

7

Is this limitation of recursive read on a path for log files is resolved with Logstash 2.0?