One or more Inputs to One or more Outputs

Elastic Stack Logstash
1

Hello Folks,
i would like to ask, i have a lot of textfiels(log files) in one folder, each file has his name , and i matched them all in Logstash ,
is there any possibility after i matched all those textfiles(log files) to send them and save them in separately Output files, that mean each input Log file will be sended into separately Output file.
I will be thankful for any Idea.

2

each log file are same format or diff format

yes you could save each file into seperate index depend upon any one unique field from each file

3

my Log files are all unstructured text Format, they have also all same datatype ".log" but different structure inside, i could match them all , and i could read them all automatically.
until now i could send them all to one Output "file.log" .
but i need to send each Input Logfile i have to different Output file . no matter if logstash generate them by its self or not.
ist that possible ? any Idea?
my input , out put look like ,
input
{ path => "C:/Users/samyo/Desktop/ELK/folder/**.log"
tags => "log"
sincedb_path => "NUL"
exclude => ".gz"
}}
filter {...}
output {
file {
index => "%{log}-index"
path => "C:/Users/samyo/Desktop/ELK/.%{+YYYY-MM-dd}.log"
codec => line { format => "text: %{message}"}
}}

4

any Update or Suggestions will be thankful ?
how to save each input logfile to seperate output logfile?
i do not want to use the if Condetionals , cause it make no sense when i have a lot of input logfiels,and give each input logfile a type.
Any Other Ideas?
thanx..

5

i can read all my input logfiels automatically with this ,
input
{ path => "C:/Users/samyo/Desktop/ELK/folder/**.log"}

but why i can not use the same line for my output , something like this ,
output
{ path => "C:/Users/samyo/Desktop/ELK/outputfolder/**.log"}

Does anyone have an Idea, i will appreciate it ?

6

The file input adds a "path" field to the event. If you want to use the same name in a different directory as the output then you could

mutate { add_field => { "filename" => "%{path}" } }
mutate { gsub => [ "filename", "^.*/", "" ] }

and then reference that field in the output

output { file { path => "/some/path/%{filename}" } }
7

hello Badger , i could not put dynamic inputsfolder path like follow,
mutate { add_field => { "textNr1" => "%{C:/Users/samyo/Desktop/ELK/inputsfolder/**.log}" } }
i want from logstash to go inside my inputsfolder and read all the input logfiles (textNr1, textNr2, textNr3, etc) dynamicly and send each of those to seperat output .
until now i could read dynamic all my inputfiles writing "*.log" in my Input path as i wrote in my Input above but send them to one output.
Because in the output i could put something like "*.log" .
And the out put as follow,
output { file { path => "C:/Users/samyo/Desktop/ELK/%{textNr1}" } }
did not work.
but this ,
output { file { path => "C:/Users/samyo/Desktop/ELK/textNr1" } }
works and created me a file name "textNr1" and puted all my input logfiles from my inputsfolder
into this one file "textNr1".
i want logstash take dynamicly every input textfile like (textNr1) into output (textNr1), and input logfile(textNr2) into output file (textNr2) etc. how to write more than one output file path??

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.