Index Inconsistency

Elastic Stack Elasticsearch
1

We are seeing inconsistent results for a get on one of our documents. When
I tried a get using the doc id, sometimes the document would come back and
sometimes it would not. We are using routing for this index but this
happens even when specifying the right routing key. To try and fix the
problem, I reindexed the doc to try and fix the problem but I am still
seeing inconsistent results. A document is now consistently returned but
when the original doc is returned, it has version set to 2. When the new
doc that was indexed is returned, version is set to 1.

[dd@search ~]$ curl -XGET
'http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":2,"exists":true,
"_source" : { "roll_key" :
"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32", "dom" : "foo.com",
"received_date" : "20140303"}}

[dd@search ~]$ curl -XGET
'http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":1,"exists":true,
"_source" :
{ "roll_key" : "foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32",
"domain" : "foo.com", "received_date" : "20140303"}}

We are using elasticsearch 0.90.7.

-drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2fa15b98-f9a7-480f-a33d-29b5bb35819d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Do you index with replica level > 0?
How many nodes?
Do you execute GET against same node or against different nodes?
Does searching the doc _id work?
Is it only this document, and others are ok?
Are are messages in a server logfile?
Does preference "primary" make a difference?

And, can you upgrade ES?

Jörg

On Thu, Aug 7, 2014 at 7:59 PM, Drew Daugherty <
drew.daugherty@returnpath.com> wrote:

We are seeing inconsistent results for a get on one of our documents.
When I tried a get using the doc id, sometimes the document would come
back and sometimes it would not. We are using routing for this index but
this happens even when specifying the right routing key. To try and fix
the problem, I reindexed the doc to try and fix the problem but I am still
seeing inconsistent results. A document is now consistently returned but
when the original doc is returned, it has version set to 2. When the new
doc that was indexed is returned, version is set to 1.

[dd@search ~]$ curl -XGET '
http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com
'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":2,"exists":true,
"_source" : { "roll_key" :
"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32", "dom" : "foo.com",
"received_date" : "20140303"}}

[dd@search ~]$ curl -XGET '
http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com
'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":1,"exists":true,
"_source" :
{ "roll_key" : "foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32",
"domain" : "foo.com", "received_date" : "20140303"}}

We are using elasticsearch 0.90.7.

-drew

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fa15b98-f9a7-480f-a33d-29b5bb35819d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2fa15b98-f9a7-480f-a33d-29b5bb35819d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEvjQ5Kwmg6kFX10FyCwh-D%3Doc%3DSvR5C%3DnDPc3uANo4uQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Index is set to one replica. 4 data nodes, 3 masters, 1 client node.
Executing the get against a client node on the cluster but masters yield
the same result. Data nodes have http turned off currently. Other docs
seem to work fine but we suspect the problem is more widespread. Is there
a way to search by document version numbers? Searching without routing
returns both docs, with routing only returns 1 (but I can't tell which
one, guessing the version 1 doc). Using preference _primary always causes
the version 1 doc to be retrieved. There do not seem to be any useful
messages in the cluster logfiles. We are planning on upgrading to 1.2 soon.
Is there a possibility this problem will be fixed if we upgrade?

-drew

On Thursday, August 7, 2014 3:33:56 PM UTC-6, Jörg Prante wrote:

Do you index with replica level > 0?
How many nodes?
Do you execute GET against same node or against different nodes?
Does searching the doc _id work?
Is it only this document, and others are ok?
Are are messages in a server logfile?
Does preference "primary" make a difference?

Elasticsearch Platform — Find real-time answers at scale | Elastic

And, can you upgrade ES?

Jörg

On Thu, Aug 7, 2014 at 7:59 PM, Drew Daugherty <drew.da...@returnpath.com
<javascript:>> wrote:

We are seeing inconsistent results for a get on one of our documents.
When I tried a get using the doc id, sometimes the document would come
back and sometimes it would not. We are using routing for this index but
this happens even when specifying the right routing key. To try and fix
the problem, I reindexed the doc to try and fix the problem but I am still
seeing inconsistent results. A document is now consistently returned but
when the original doc is returned, it has version set to 2. When the new
doc that was indexed is returned, version is set to 1.

[dd@search ~]$ curl -XGET '
http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com
'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":2,"exists":true,
"_source" : { "roll_key" :
"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32", "dom" : "foo.com",
"received_date" : "20140303"}}

[dd@search ~]$ curl -XGET '
http://localhost:9200/blah_201403/segment/foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32?routing=foo.com
'
{"_index":"blah_201403","_type":"segment","_id":"foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32","_version":1,"exists":true,
"_source" :
{ "roll_key" : "foo.com_20140303_fbd28d761e38fe82465c94c1cc23fa32",
"domain" : "foo.com", "received_date" : "20140303"}}

We are using elasticsearch 0.90.7.

-drew

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2fa15b98-f9a7-480f-a33d-29b5bb35819d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2fa15b98-f9a7-480f-a33d-29b5bb35819d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/023a1810-9c30-4639-abc3-d79286f0972d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

If you see both docs, you probably encountered the hashing bug of 1.2.0

but if you are on 1.1 I have doubts. Strange.

I recommend to upgrade to 1.3.x und reindex all relevant docs...

Jörg

On Fri, Aug 8, 2014 at 4:46 PM, Drew Daugherty <
drew.daugherty@returnpath.com> wrote:

Searching without routing returns both docs, with routing only returns
1 (but I can't tell which one, guessing the version 1 doc). Using
preference _primary always causes the version 1 doc to be retrieved. There
do not seem to be any useful messages in the cluster logfiles. We are
planning on upgrading to 1.2 soon. Is there a possibility this problem will
be fixed if we upgrade?

-drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHLNJn8FKLwuL1o3iBJr0rxqtGbr9yx8t-S%3DvDp72-Qiw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

We are using elasticsearch 0.90.7.

-drew

On Friday, August 8, 2014 2:48:21 PM UTC-6, Jörg Prante wrote:

If you see both docs, you probably encountered the hashing bug of 1.2.0

Elasticsearch Platform — Find real-time answers at scale | Elastic

but if you are on 1.1 I have doubts. Strange.

I recommend to upgrade to 1.3.x und reindex all relevant docs...

Jörg

On Fri, Aug 8, 2014 at 4:46 PM, Drew Daugherty <drew.da...@returnpath.com
<javascript:>> wrote:

Searching without routing returns both docs, with routing only returns
1 (but I can't tell which one, guessing the version 1 doc). Using
preference _primary always causes the version 1 doc to be retrieved. There
do not seem to be any useful messages in the cluster logfiles. We are
planning on upgrading to 1.2 soon. Is there a possibility this problem will
be fixed if we upgrade?

-drew

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d019326a-fc12-4c23-9b21-d34d41442ad5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

I think you should upgrade anything above 1.1

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c719ce8-23fd-4a44-bb9c-4d620299c932%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.