Using Upsert across index

Elastic Stack Logstash
1

I use this config to insert document in ES :```
output {
if [type] == "usage" {
elasticsearch {
hosts => ["elastic4:9204"]
index => "usage-%{+YYYY-MM}"
document_id => "%{IDSU}"
action => "update"
doc_as_upsert => true
}

    }

This is fine but the problem i have the duplicates occur within the different month, when the month rolls over, and the document still appears, elastic/logstash thinks it's a new doc, and create new document bu i want to do is to update document wich dos'ent exist in current index but in the old index.

Is there a way to make the upsert work cross index? These would all be the same type of doc, they would simply apply change status field for exampl

2

I cannot think of any way to have an update to one index be redirected to a different index depending on the contents of the second index.

3

But the indexs have the same structure juste , i create every month new index , what i want when i want add doc i must search in old index if the same doc with doc_id already exist if this case i update in the document in old index, else i creat one new document in the last index index (of current month) , i use upsert it work fine if this is the same index but i if not i have duplicate document

4

Exactly. It works in the same index but not across indexes. That's working as expected.

5

Thank you so much , so there is no way to check document accos index befor insert , using elastic search output plugin ?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.