Conditional update based on priority in Elasticsearch

Elastic Stack Elasticsearch
7

Thanks for the suggestion.

I've tried using the versioning. Unfortunately, it doesn't work with the update API using Logstash.

I have to use the update API since I have incremental data.
e.g.

{"id": 5, "status": "pending", "customerID": 7}
{"id": 5, "status": "processed", "processordID": 12}

doc in elasticsearch:
{"id": 5, "status": "processed", "customerID": 7, "processordID": 12}

If I use the index API, I'll have a missing field:
{"id": 5, "status": "processed", "processordID": 12}

Logstash Error:

[2021-01-25T02:57:53,886][ERROR][logstash.outputs.elasticsearch][main][375cd2bcc47962e4aa3a8e9f0373d50720bbe90b85cd9126eee85cd4adaa71ed] Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://127.0.0.1:9200/_bulk", :body=>"{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Update requests do not support versioning. Please use `if_seq_no` and `if_primary_term` instead\"}],\"type\":\"illegal_argument_exception\",\"reason\":\"Update requests do not support versioning. Please use `if_seq_no` and `if_primary_term` instead\"},\"status\":400}"}