I use logsth and filebeat for ingestion in elastic
my conf logstash is
input {
beats {
port => "5044"
}
}
filter {
if "fraicheur_devgroupe" in [tags] {
csv{
separator => ";"
columns => ["NM_APPLIC_DECIS", "NM_SOUS_APPLIC_DECIS", "NM_TRAIT_CHARG", "DT_FIN_PERIOD_TRAITE"]
}
date {
match => [ "DT_FIN_PERIOD_TRAITE" , "dd-MM-YYYY" ]
timezone => "Europe/Paris"
}
}
}
output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "datefraicheur"
action => "index"
}
stdout { codec => rubydebug }
}
and my conf filebeat is:
filebeat.inputs:
- type: log
enabled: true
paths:
- /data/volumes/monitoring/logstash/logCtrlM/export_fraicheur_devgroupe.csv
tags: ["fraicheurdevgroupe"]
exclude_lines: ["^NM_APPLIC_DECIS;"]
output.logstash:
hosts: ["logstash:5044"]
when i run logstash colonnes is not created in elastic