Hi,
Is it possible to add a field in elasticsearch output only if the document in the index does not contain that field?
I'm trying to fetch data from a API (using the input http_poller) and storing the data in a elasticsearch index (using output elasticsearch). If the index does not contains a specific field i want to add it (with a default value). I need to check its existence because I have action 'update' and doc_as_upsert true.
Currently I'm using the "add_field" in the http_poller but at every schedule it overwrite the current value of the field stored in the index.
Current:
input {
http_poller {
urls => {
test1 => "http://localhost:9000/test"
}
request_timeout => 60
schedule => { cron => "* * * * *"}
codec => "json"
metadata_target => "http_poller_metadata"
add_field => { "status" => "active"}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "test-2020.08.31"
document_id => "%{[uid]}"
action => 'update'
doc_as_upsert => true
}
}
When the value of "status" change after a schedule (eg. from "active" to "stopped") at the next schedule it will overwrite the current value (from "stopped" to "active")