Logstash upsert partial document

I have an index that contains some documents where field that are null and eventually once another particular row comes that has data for these fields, I want to upsert all old documents containing a particular document id with the same data that this new row has for the null fields.

So basically in event A the field 'system' is null, in event B that comes later for this same id it has a 'system' value, so while inserting the row B into elasticsearch I also want it to update row A'system' field.

Is this possible or not within logstash using the output elasticsearch?

elasticsearch{
    hosts=> ["X:9200"]
    index => "search-%{record_date}"
    document_id => "%{id}"
    action => "update"
    doc_as_upsert => true
    workers => 4
}
1 Like

Yes, this should be possible. How are you setting the id field which is used as the document id?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.