Hello. I have a problem with ingest pipeline best practicies.
I want to parse system logs by Filebeat pipeline, but do not use Filebeat, because we already collect these logs.
I do ./filebeat setup --pipelines for inserting pipeline in ELK.
Then i use such config, that support provide me. Is this a best way? I have some mapping errors, that's why i want to check config first. Pipeline name and all network adresses are correct, i check it. I have events, but have a mapping errors.
input {
udp {
port => 10002
codec => line
tags => ["linux_sys_secure"]
}
}
filter {
if "linux_secure" in [tags]{
mutate {
add_field => { "[@metadata][pipeline]" => "filebeat-7.2.0-system-syslog-pipeline" }
}
}
}
output {
if "linux_secure" in [tags]{
elasticsearch {
pipeline => "%{[@metadata][pipeline]}"
hosts => ["skynet-elk-1:9200"]
index => 'filebeat-linux_sys_secure-11-%{+YYYY.MM.dd}'
}
}
}