Hello,
I'm new to ELK so please apologies if my question is trivial.
I have 3 CSV files with data that I would like to merge.
In each file, each row represent a data which can be father or child of a row in another file(s).
I would like to be able to create a "single" merged object to be send to elastic ( some fields can be null if there is no link between datas)
Note that each file might have field with same names and I should rename it in the result one. I also must do some field addition with date comparison.
MyData :
A.id | B.id | C.id | A. startDate | A.endDate | A.diff | B.startDate | B.endDate | B.diff| ......
I currently succeed to parse one file but I do not found any tips on internet for my need.
My current file content
input {
file {
type => "LevelA"
path => "D:/LevelA.txt"
start_position => "beginning"
sincedb_path => "NUL"
}
file {
type => "LevelB"
path => "D:/LevelB.txt"
start_position => "beginning"
sincedb_path => "NUL"
}
file {
type => "LevelC"
path => "D:/LevelC.txt"
start_position => "beginning"
sincedb_path => "NUL"
}
}
filter {
csv {
separator => ","
skip_header => "true"
columns => [ "id","startDate", "endDate","msg" ]
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "mydata"
}
stdout {codec => rubydebug}
}