One or more Inputs to One or more Outputs

(samyo) #1

Hello Folks,
i would like to ask, i have a lot of textfiels(log files) in one folder, each file has his name , and i matched them all in Logstash ,
is there any possibility after i matched all those textfiles(log files) to send them and save them in separately Output files, that mean each input Log file will be sended into separately Output file.
I will be thankful for any Idea.

(Ganesh) #2

each log file are same format or diff format

yes you could save each file into seperate index depend upon any one unique field from each file

(samyo) #3

my Log files are all unstructured text Format, they have also all same datatype ".log" but different structure inside, i could match them all , and i could read them all automatically.
until now i could send them all to one Output "file.log" .
but i need to send each Input Logfile i have to different Output file . no matter if logstash generate them by its self or not.
ist that possible ? any Idea?
my input , out put look like ,
input
{ path => "C:/Users/samyo/Desktop/ELK/folder/**.log"
tags => "log"
sincedb_path => "NUL"
exclude => ".gz"
}}
filter {...}
output {
file {
index => "%{log}-index"
path => "C:/Users/samyo/Desktop/ELK/
.%{+YYYY-MM-dd}.log"
codec => line { format => "text: %{message}"}
}}

(samyo) #4

any Update or Suggestions will be thankful ?
how to save each input logfile to seperate output logfile?
i do not want to use the if Condetionals , cause it make no sense when i have a lot of input logfiels,and give each input logfile a type.
Any Other Ideas?
thanx..

(samyo) #5

i can read all my input logfiels automatically with this ,
input
{ path => "C:/Users/samyo/Desktop/ELK/folder/**.log"}

but why i can not use the same line for my output , something like this ,
output
{ path => "C:/Users/samyo/Desktop/ELK/outputfolder/**.log"}

Does anyone have an Idea, i will appreciate it ?

#6

The file input adds a "path" field to the event. If you want to use the same name in a different directory as the output then you could

mutate { add_field => { "filename" => "%{path}" } }
mutate { gsub => [ "filename", "^.*/", "" ] }

and then reference that field in the output

output { file { path => "/some/path/%{filename}" } }
(samyo) #7

hello Badger , i could not put dynamic inputsfolder path like follow,
mutate { add_field => { "textNr1" => "%{C:/Users/samyo/Desktop/ELK/inputsfolder/**.log}" } }
i want from logstash to go inside my inputsfolder and read all the input logfiles (textNr1, textNr2, textNr3, etc) dynamicly and send each of those to seperat output .
until now i could read dynamic all my inputfiles writing "*.log" in my Input path as i wrote in my Input above but send them to one output.
Because in the output i could put something like "*.log" .
And the out put as follow,
output { file { path => "C:/Users/samyo/Desktop/ELK/%{textNr1}" } }
did not work.
but this ,
output { file { path => "C:/Users/samyo/Desktop/ELK/textNr1" } }
works and created me a file name "textNr1" and puted all my input logfiles from my inputsfolder
into this one file "textNr1".
i want logstash take dynamicly every input textfile like (textNr1) into output (textNr1), and input logfile(textNr2) into output file (textNr2) etc. how to write more than one output file path??

(system) closed #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.