Searching subdirectories

I am trying to setup filebeat to collect all the files within a folder. Here is the folder structure

Main
├───subdir1A
│ ├───subdir1B
| │ ├───subdir1C
| | | ├───file1
| | | ├───file2
| | | ├───file3
| | | ├───file4
| │ ├───subdir2C
| | | ├───file5
| | | ├───file6
| | | ├───file7
│ ├───subdir2B
| | ├───....
│ ├───subdir3B
| | ├───....
│ ├───subdir4B
| | ├───....
├───subdir2A
│ ├───subdir5B
| | ├───....
│ ├───subdir6B
| | ├───....
│ ├───subdir7B
| | ├───....

My filebeat.yml file is setup like this:

filebeat.prospectors:

  • input_type: log
    paths:
    • C:/Users/Person/Desktop/Main/*
      output.logstash:
      hosts: ["localhost:5044"]

I am unable to however read any file. I have 2 issues here that I think are causing the problem:

  1. Filebeat is not recursively going down the directories until it reaches the files
  2. The files in question are actually just un-extensioned files. In the window explorer their type is listed as just File. These are log files generated by another application and I can open them up with a text editor to read them just fine but they don't have any extension like .log or .txt or anything like that.

Any ideas how to fix this?

For 1, best have a look at https://github.com/elastic/beats/issues/2084
For 2: As long as the files have one event per line and are not in a binary format, this should not be an issue.

This topic was automatically closed after 21 days. New replies are no longer allowed.