I am trying to merge two document based on a common information in the document. Using the doc_as_upsert option for this. My logstash config is
if [uri_query] and ![Server_ElapsedMS] {
elasticsearch {
hosts => ["abcd:9200"]
index => 'beta-%{+YYYY-MM}'
action => "update"
doc_as_upsert => true
document_id => "beta-%{comp1}%{comp2}%{comp3}%{comp4}%{comp5}%{comp6}"
}
}
else if [uri_query] and [Server_ElapsedMS] {
elasticsearch {
hosts => ["abcd:9200"]
index => 'beta-%{+YYYY-MM}'
action => "update"
doc_as_upsert => true
document_id => "beta-%{comp1}%{comp2}%{comp3}%{comp4}%{comp5}%{comp6}"
}
}
Following is the error i get in logstash logs
[2019-02-13T09:17:59,563][WARN ][logstash.outputs.elasticsearch] Failed action. {:status=>409, :action=>["update", {:_id=>"beta-271162.16.840.1.114151.10181811386601901181.2.840.113681.2887585702.1549638557.4400.510574ComponentByNameCS=262144&OS=11272192&sid=c9414b41-3f19-4568-9f6a-b8b4e90b6f53", :_index=>"beta-2019-02", :_type=>"doc", :_routing=>nil, :_retry_on_conflict=>1}, #LogStash::Event:0x22987f34], :response=>{"update"=>{"_index"=>"beta-2019-02", "_type"=>"doc", "_id"=>"beta-271162.16.840.1.114151.10181811386601901181.2.840.113681.2887585702.1549638557.4400.510574ComponentByNameCS=262144&OS=11272192&sid=c9414b41-3f19-4568-9f6a-b8b4e90b6f53", "status"=>409, "error"=>{"type"=>"version_conflict_engine_exception", "reason"=>"[doc][beta-271162.16.840.1.114151.10181811386601901181.2.840.113681.2887585702.1549638557.4400.510574ComponentByNameCS=262144&OS=11272192&sid=c9414b41-3f19-4568-9f6a-b8b4e90b6f53]: version conflict, document already exists (current version [1])", "index_uuid"=>"yvM1Rv1qS0qXs-SB4YHnyw", "shard"=>"0", "index"=>"beta-2019-02"}}}}
I understand that the id exists, but the action i have given is update so it should go ahead and update to ES. Am i wrong here ?