Hi,
is there a recommended way to set some default @metadata fields in Logstash for all documents? I'm using this logstash output config to "shard" different types of logs to their own indices as ESv6 no longer supports more than one type of documents per index
output {
elasticsearch {
hosts => ["10.0.0.3:9200", "10.0.0.18:9200", "10.0.0.26:9200"]
index => "%{[@metadata][log_prefix]}-%{[@metadata][index]}-%{+YYYY.MM.dd}"
}
}
For documents that to not have these metadata fields set I get these indices
$ curl -s http://localhost:9200/_cat/indices | grep meta
green open %{[@metadata][log_prefix]}-%{[@metadata][index]}-2018.06.18 -N6-8xHgTK6YxihD0euTJw 5 1 7002784 0 2.6gb 1.3gb
green open %{[@metadata][log_prefix]}-%{[@metadata][index]}-2018.05.23 fouzYG2ZQS2-OmGneendKg 5 1 12354 0 9.3mb 4.6mb
which is not quite what I had planned...
One way would be to use the alter
filter plugin at the top of the filter chain.
filter {
alter {
add_field => { "%{[@metadata][log_prefix]}" => "my_prefix" }
add_field => { "%{[@metadata][index]}" => "my_index" }
}
}
Any better way to do it?
Cheers,
AB