How to let Logstash read all my Input texts Files automatically

Elastic Stack Logstash
1

Hello Folks,
i need some help to know if its Possible, to let Logstash read my 1000 Textfiles separately and automatically from the Input.
I have about 1000 textfiles , each textfile has his name and different text inside.
I already wrote my filter in Logstash to read this texts, but the Question is :
Can Logstash read all those textfiles separately and automatically from my Input???
Is there any Command for that ?
does anybody have an Idea about that?

Thx.

2

You can use a file input with a path regex (or regexes) that match those files.

3

hello Badger,
that is my Input Config , after i created a folder includes my all textfiels inside.
input
{ type => "log"
path => "C:\Users\samyo\Desktop\ELK\folder\log*"
#path => "C:\Users\samyo\Desktop\ELK\folder\((?:.(?!\))+.log)"
start_position => "beginning"
exclude => "*.gz"
}}

but it stills could not read the textfiels inside the "folder" wich i created?
can you please correct my Input config above?

thx.

4

Try adding sincedb_path => "NUL".

5

i tried it too , still Logstash could not go inside the "folder" and get the textfiles to read them,
it seems Logstash can not read a lot of Inputs automatically .
i tried to put just two paths for two differen textfiels and it works
path => "C:\Users\samyo\Desktop\ELK\inputtest1.log"
path => "C:\Users\samyo\Desktop\ELK\inputtest2.log"

, but after adding manually 3 paths separately for 3 different textfiels , logstash was not able more to read those 3 textfiles.

any Suggestions ?
thx.

6

check whether your log file contain new line at end of the file.

7

that is my Log config file all about , i even tried different ways of "path" as you can see,
what do you mean with "new line at the end of file" ? Can you please adjust my input?
input
{ type => "log"
path => "C:\Users\samyo\Desktop\ELK\folder*.log"
#path => "C:\Users\samyo\Desktop\ELK\folder((?:.(?!))+.log)"
#path => "C:\Users\samyo\Desktop\ELK\folder\log*"
start_position => "beginning"
sincedb_path => "NUL"
exclude => "*.gz"
}}

8

I'm not saying your configuration file and i'm talking about log file which your parsing into logstash

9

yes of course my textfiels have text after each text i pressed Enter for a new Line and saved it , still did not work , i know that logstash can not do that by it self , so i did it inorder to help logstash , still could not get the textfiels from the "folder" .

10

Use forward slash instead of backslash in those paths.

11

Badger you have right , it works now, it was all cause of the Backslash .
i could not believe that , thank you guys so much for supporting, you are the best.
thanx :slight_smile:

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.