I use Logsash and filebeat for ingestion in elastic
my config file beat is
filebeat.inputs:
- type: log
enabled: true
paths:
- C:\Users\h83710\Desktop\elastic\logstash-7.5.2\test\FICHIER.csv
paths:
- C:\Users\h83710\Desktop\elastic\logstash-7.5.2\test\poca.csv
and my config logsth is
input {
beats {
port => "5044"
}
}
filter {
csv {
separator => ";"
columns => ["chaine", "job", "date_plan", "statut", "date_debut", "date_fin", "serveur", "numero_passage", "application", "sous_application"]
}
csv {
separator => ";"
columns => ["date_fraicheur"]
}
date {
match => [ "date_fraicheur" , "dd/MM/YYYY" ]
}
date {
match => [ "date_plan" , "dd/MM/YYYY" ]
}
date {
match => [ "date_debut" , "dd/MM/YYYY HH:mm" ]
}
date {
match => [ "date_fin" , "dd/MM/YYYY HH:mm" ]
}
mutate {
convert => { "numero_passage" => "integer" }
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "hello"
}
stdout {codec => rubydebug}
}
I have a problem from mix ingestion
you can create separate input on filebeat for each path, then add different tag for each input. you can then apply correct csv mapping in logstash to correct tag.