# Updating documents with Elasticsearch output
**URL:** https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898
**Category:** Logstash
**Created:** [July 5, 2017, 12:20pm UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898 "2017-07-05T12:20:20Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@orweinberger](https://discuss.elastic.co/u/orweinberger)
#### Post date: [July 5, 2017, 12:20pm UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/1 "2017-07-05T12:20:21Z")
I'm trying to figure out how to update documents properly using the Elasticsearch output. Documentation on this issue is very confusing.
I have a document that **might** already be in Elasticsearch, and if so I would like to update it. Going over the documentation I see that there are multiple options for updating documents.
So for example this is my current configuration:
```
if [doc_id] {
elasticsearch {
hosts => ["http://elastic:9200"]
document_type => "log"
document_id => "%{doc_id}"
doc_as_upsert => true
action => "update"
index => "myindex"
}
}
```
This confiugration sometimes throws an error saying:
`WARN logstash.outputs.elasticsearch - Failed action. {:status=>409, :action=>["update", {:_id=>"6b5b8db751dcd3b8586badfd70dca5", :_index=>"main_solan", :_type=>"log", :_routing=>nil, :_retry_on_conflict=>1}, 2017-07-05T10:27:46.555Z %{host} %{message}], :response=>{"update"=>{"_index"=>"myindex", "_type"=>"log", "_id"=>"6b5b8db751dcd3b8586badfd70dca5", "status"=>409, "error"=>{"type"=>"version_conflict_engine_exception", "reason"=>"[log][6b5b8db751dcd3b8586badfd70dca5]: version conflict, current version [406] is different than the one provided [405]", "index_uuid"=>"fN3yXZ6xSdSb-hdbP3SnOw", "shard"=>"0", "index"=>"myindex"}}}}`
I'm not providing the version myself, so I'm not sure how Logstash knows what version I'm currently providing. I'm also not sure if I should be using `action=update`, or maybe just define `upsert=true`?
---
### Author:  [@orweinberger](https://discuss.elastic.co/u/orweinberger)
#### Post date: [July 9, 2017, 1:34pm UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/2 "2017-07-09T13:34:36Z")
Any ideas?
---
### Author:  [@Nico-DF](https://discuss.elastic.co/u/Nico-DF)
#### Post date: [July 10, 2017, 7:58am UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/3 "2017-07-10T07:58:10Z")
If I remember well, It happend to me too.
See: [https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index\_.html#index-versioning](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-versioning)
And if I'm correct, it means you try to update a document while its version is newer than you expect it to be. So probably due to multithreading or delay in I/O.
---
### Author:  [@orweinberger](https://discuss.elastic.co/u/orweinberger)
#### Post date: [July 10, 2017, 8:01am UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/4 "2017-07-10T08:01:12Z")
But logstash does not determine any versions. I mean, I insert documents and if I try and insert an older version due to latency or something else, it just means that the latest version will be incorrect, but from what I understand Logstash does not do its own versioning so why am I getting an error message?
---
### Author:  [@Nico-DF](https://discuss.elastic.co/u/Nico-DF)
#### Post date: [July 10, 2017, 8:07am UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/5 "2017-07-10T08:07:41Z")
You might maybe want to wait someone who know how it works behind scene (or google it maybe), but I imagine the following:
Suppose 2 document that will be in output (if your pipeline is not set to 1, it will happen).
- Doc A arrives in output
- For the update, I imagine (maybe) a read operation is perform. It reads data and its version: 100
- The same for doc B. Version: 100
- Doc A updates. Now the version of the doc in ES is 101 (automatic)
- Doc B updates, but now the document is in version 101 but B has made the update on version 100. It isn't compatible anymore. Hence, fail.
Logstash will also contain version info.
[https://www.elastic.co/guide/en/elasticsearch/guide/current/version-control.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/version-control.html)
[https://www.elastic.co/guide/en/elasticsearch/guide/current/optimistic-concurrency-control.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/optimistic-concurrency-control.html)
So I don't know, can you try to set pipeline workers to 1? (All of these are speculations based on update=get+put which can be reasonable for integrity purposes)
---
### Author:  [@paz](https://discuss.elastic.co/u/paz)
#### Post date: [July 10, 2017, 10:32am UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/6 "2017-07-10T10:32:48Z")
> [@orweinberger](#):
>
> But logstash does not determine any versions. I mean, I insert documents and if I try and insert an older version due to latency or something else, it just means that the latest version will be incorrect, but from what I understand Logstash does not do its own versioning so why am I getting an error message?
You are correct, Logstash does not know the document's version.
What you see there in the log is the error Elasticsearch reports back to Logstash.
As @Nico-DF said, it's most likely a [race condition](https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html#_updates_and_conflicts) where you try to update a document's version before the previous change has finished indexing (a version number of 406 kinda points to many back-to-back updates).
You can try the [retry\_on\_conflict](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-retry_on_conflict) setting and see if it solves the issue somewhat.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 7, 2017, 10:39am UTC](https://discuss.elastic.co/t/updating-documents-with-elasticsearch-output/91898/7 "2017-08-07T10:39:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.