Ignore doc versioning ES 6.5

abaullosa · January 9, 2019, 2:37pm

Hi guys.

I need to ignore the Doc version once creating a new document, I need to insert a document and wait for another transaction to update that document. since this happens some time extremely fast, Im receiving a lot of Versioning errors once creating a doc.

Is there a way I can ignore the doc version?.

Thanks a lot in advance

dadoonet · January 9, 2019, 4:30pm

How do you "update" the document?

abaullosa · January 9, 2019, 4:55pm

Hi David, thanks a lot for your reply!.

Im doing this on Nodejs, this is my scenario:
I need to log some network traffic requests and responses. I need to store some information from that request and whenever the response comes back I need to update that document where the request is with some information that comes on the response.

this is my line of code in nodejs.

   me.client.exists({
                index: index,
                type: type,
                id: id
            }, function(err, exists) {
                if (err) {
                    cb(err)
                } else {
                    if (exists === true) {
                        out('exists. doing update.');
                        // update existing document
                        me.client.update({
                            index: index,
                            type: type,
                            id: id,
                            waitForActiveShards: 'all',
                            retryOnConflict: '10',
                            // versionType: "external",
                            body: {
                                doc: body
                            }
                        }, function(err, resp) {
                            if (err) {
                                cb(err);
                            } else {
                                cb(null, resp);
                            }
                        });
                    } else {
                        out('adding new document');
                        // add new document
                        me.client.create({
                            index: index,
                            type: type,
                            id: id,
                            waitForActiveShards: 'all',
                            body: body
                        }, function(err, resp) {
                            if (err) {
                                cb(err);
                            } else {
                                cb(null, resp);
                            }
                        });
                    }
               }

Thanks a lot again

dadoonet · January 9, 2019, 5:13pm

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

Is there any reason you are using the update API instead of the index API?

abaullosa · January 9, 2019, 5:46pm

Thanks, David,

I forgot to format the code, sorry about that.

Well that's a good question, can I use the Index API to update the doc as well?. I'm using the Update because I don't know if I can update a Doc with the Index API.

dadoonet · January 9, 2019, 6:56pm

Yes. You can do:

DELETE index
PUT index/_doc/1
{
  "foo": "bar"
}
PUT index/_doc/1
{
  "foo": "baz"
}
GET index/_doc/1

abaullosa · January 9, 2019, 8:24pm

Thanks a lot David,

I changed to Index, but Im still having the same errors. If I use "Index" instead of "Create" api, then I will not have any issue but I will not get the original doc updated with the request. so I cant use the client.index instead of the client.create that Im using now.

response: '{"error":{"root_cause":[{"type":"version_conflict_engine_exception","reason":"[api][3e5b2f21-f8c3-4934-9e92-639b2af86080]: version conflict, document already exists (current version [1])","index_uuid":"OvcPuciUSLWB__4zv0dSyg","shard":"0","index":"api-20190109"}],"type":"version_conflict_engine_exception","reason":"[api][3e5b2f21-f8c3-4934-9e92-639b2af86080]: version conflict, document already exists (current version [1])","index_uuid":"OvcPuciUSLWB__4zv0dSyg","shard":"0","index":"api-20190109"},"status":409}

thanks a lot

dadoonet · January 9, 2019, 8:42pm

Don't use create but index as I did in my example.

abaullosa · January 9, 2019, 9:48pm

using Index for some reason will not update the same doc ID,looks like the Request and response are store with a different doc ID

i will receive a request and I need to save it, then I need to update that particular request with the response later.. using Index, will create a DOC ID on each one and will not match the Request and response right?..

abaullosa · January 10, 2019, 7:16pm

If I use Index instead of Create, it will not fail, but I will not get the expected data, my documents will not be updated as desired. If I use Create instead of Index, then I will get the correct data but I will have a lot of version issues like this one:

response: '{"error":{"root_cause":[{"type":"version_conflict_engine_exception","reason":"[api][eb4cd858-1097-4983-be61-aa8de2874c51]: version conflict, document already exists (current version [1])

Thanks a lot for the help!.

dadoonet · January 10, 2019, 9:19pm

I don't understand why you feel that using index will be a problem.

May be illustrate with an example?

abaullosa · January 10, 2019, 9:51pm

Thanks a lot for the reply.

This is my scenario:
Example: Im inserting Doc that has,

ID = 123, person_name: “mike.”

After inserting that Doc ID, I will need to do upsert to the same doc this time with

ID = 123, Person_name: “mike”, Last_name: “stuart”

So my final doc need to look like:

ID= 123, person_name:”mike”, Last_name: ’’Stuart’’.

This is my code now

    me.client.exists({
            index: index,
            type: type,
            id: id
            }, function(err, exists) {
            if (err) {
                cb(err)
            } else {
                if (exists === true) {
                    out('exists. doing update.');
                    // update existing document
                    me.client.update({
                        index: index,
                        type: type,
                        id: id,
                          body: {
                            doc: body

                        }
                    }, function(err, resp) {
                        if (err) {
                            cb(err);
                        } else {
                            cb(null, resp);
                        }
                    });
                } else {
                    out('adding new document');
                    // add new document
                    me.client.create({
                        index: index,
                        type: type,
                        id: id,
                        body: body
                    }, function(err, resp) {
                        if (err) {
                            cb(err);
                        } else {
                         cb(null, resp);
                        }
                    });
                }

Using this, Im able to get the correct results, but, I have a lot of "version conflict, document already exists" errors.

If I replace Update and Create with Index. then I dont have any errors, but Im saving im not doing an upsert.
what I receive in ES is
Doc ID: 123, person_name: "Mike"
Doc ID:124, Last_name: "Stuart".

it looks like its creating a new ID for each doc.

Thanks a lot in advance for your answer

dadoonet · January 11, 2019, 10:29am

You are probably doing something wrong. I don't see why you are using upsert.
Here is a simple script that works IMO:

DELETE test
PUT test/_doc/123
{
  "person_name": "mike"  
}
PUT test/_doc/123
{
  "person_name": "mike",
  "last_name": "stuart"
}
GET test/_doc/123

It gives:

{
  "_index" : "test",
  "_type" : "_doc",
  "_id" : "123",
  "_version" : 2,
  "found" : true,
  "_source" : {
    "person_name" : "mike",
    "last_name" : "stuart"
  }
}

system · February 8, 2019, 10:29am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Indexing new document and check version Elasticsearch	1	414	July 6, 2017
Update(upsert) with query support Elasticsearch	6	2325	July 6, 2018
External versioning for create Elasticsearch	7	4722	December 30, 2016
Create or Update a document Elasticsearch	3	21738	August 13, 2017
Version conflict, document already exists (current version [1]) Logstash	14	17285	September 25, 2018

Ignore doc versioning ES 6.5

Related Topics