Searching subdirectories

Elastic Stack Beats
1

I am trying to setup filebeat to collect all the files within a folder. Here is the folder structure

Main
├───subdir1A
│ ├───subdir1B
| │ ├───subdir1C
| | | ├───file1
| | | ├───file2
| | | ├───file3
| | | ├───file4
| │ ├───subdir2C
| | | ├───file5
| | | ├───file6
| | | ├───file7
│ ├───subdir2B
| | ├───....
│ ├───subdir3B
| | ├───....
│ ├───subdir4B
| | ├───....
├───subdir2A
│ ├───subdir5B
| | ├───....
│ ├───subdir6B
| | ├───....
│ ├───subdir7B
| | ├───....

My filebeat.yml file is setup like this:

filebeat.prospectors:

  • input_type: log
    paths:
    • C:/Users/Person/Desktop/Main/*
      output.logstash:
      hosts: ["localhost:5044"]

I am unable to however read any file. I have 2 issues here that I think are causing the problem:

  1. Filebeat is not recursively going down the directories until it reaches the files
  2. The files in question are actually just un-extensioned files. In the window explorer their type is listed as just File. These are log files generated by another application and I can open them up with a text editor to read them just fine but they don't have any extension like .log or .txt or anything like that.

Any ideas how to fix this?

2

For 1, best have a look at https://github.com/elastic/beats/issues/2084
For 2: As long as the files have one event per line and are not in a binary format, this should not be an issue.

3

This topic was automatically closed after 21 days. New replies are no longer allowed.