Failed to load uid from the index in match_all query


(Lar Mader) #1

We are getting an error when executing the following query:

{"query":{"match_all":{}}}

Error in search:[shard [[n0KvEaZoQh2jEaB1xUoHHg][ssga-dev_ssga-dev]
[0]], reason [RemoteTransportException[[Cannonball][inet[/
10.205.24.63:9300]][search/phase/fetch/id]]; nested:
FetchPhaseExecutionException[[ssga-dev_ssga-dev][0]:
query[ConstantScore(:)],from[0],size[10]: Fetch Failed [Failed to
load uid from the index]]; ]]

We are also having trouble with some "ids" queries failing, some
working.

What could cause this?


(Shay Banon) #2

It seems like its missing the internal _uid field that is stored in each document from the document fetched. Is it something reproducible? (I improved the failure message to include the loaded fields).

On Friday, July 8, 2011 at 12:10 AM, lmader wrote:

Failed to
load uid from the index


(Matt Smalley) #3

Hi, I work with the OP and have been digging into this. I don't have a
hard example to send you just yet, but have uncovered some important
details. We use a parent/child mapping to index attachments separate
from our metadata and it appears to only occur when doing a
"match_all" against both the parent (content) and child type
(contentFiles), eg:

curl -XGET http://localhost:9200/temp/_search -d '{"query":
{"match_all":{}}}'
curl -XGET http://localhost:9200/temp/content,contentFiles/_search -d
'{"query":{"match_all":{}}}'

both fail, but:

curl -XGET http://localhost:9200/temp/content/_search -d '{"query":
{"match_all":{}}}'
curl -XGET http://localhost:9200/temp/contentFiles/_search -d
'{"query":{"match_all":{}}}'

do not. Here is log snippet, showing the index being dropped, created,
and several mapping updates before the exception trace:

[2011-07-12 18:57:14,692][INFO ][cluster.metadata ] [Nital,
Adri] [temp] deleting index
[2011-07-12 18:57:16,879][INFO ][cluster.metadata ] [Nital,
Adri] [temp] creating index, cause [auto(index api)], shards [5]/[1],
mappings [content, contentFiles]
[2011-07-12 18:57:16,927][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,122][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,244][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,372][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,700][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:18,122][DEBUG][action.search.type ] [Nital,
Adri] [257386] Failed to execute fetch phase
org.elasticsearch.search.fetch.FetchPhaseExecutionException: [temp]
[0]: query[ConstantScore(:)],from[0],size[10]: Fetch Failed [Failed
to load uid from the index]
at
org.elasticsearch.search.fetch.FetchPhase.extractUid(FetchPhase.java:
167)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:
80)
at
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:
352)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:
314)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:
140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.access$200(TransportSearchQueryThenFetchAction.java:59)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction$1.run(TransportSearchQueryThenFetchAction.java:113)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Our documents are rather complex so I'm not exactly sure what about
that particular document (if anything) triggered the issue. Other
child documents were inserted by my test script but did not cause any
shard failures for the match_all query. Unfortunately I can't post the
data so I will try to whittle it down to something simple you will be
able to reproduce on your end.

On Jul 7, 5:58 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

It seems like its missing the internal _uid field that is stored in each document from the document fetched. Is it something reproducible? (I improved the failure message to include the loaded fields).

On Friday, July 8, 2011 at 12:10 AM, lmader wrote:

Failed to
load uid from the index


(Lar Mader) #4

I've created a program that repros the problem:

This is essentially a problem with parent/child documents.

Please note that the _source for the child document must be disabled
for this problem to show up. The program puts the mapping and thus
sets this.

Keep me posted on whether this program works for you to repro the
problem.

Thanks,
Lar

On Jul 12, 7:10 pm, Matt Smalley matt.smal...@gmail.com wrote:

Hi, I work with the OP and have been digging into this. I don't have a
hard example to send you just yet, but have uncovered some important
details. We use a parent/child mapping to index attachments separate
from our metadata and it appears to only occur when doing a
"match_all" against both the parent (content) and child type
(contentFiles), eg:

curl -XGEThttp://localhost:9200/temp/_search-d '{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/content,contentFiles/_search-d
'{"query":{"match_all":{}}}'

both fail, but:

curl -XGEThttp://localhost:9200/temp/content/_search-d '{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/contentFiles/_search-d
'{"query":{"match_all":{}}}'

do not. Here is log snippet, showing the index being dropped, created,
and several mapping updates before the exception trace:

[2011-07-12 18:57:14,692][INFO ][cluster.metadata ] [Nital,
Adri] [temp] deleting index
[2011-07-12 18:57:16,879][INFO ][cluster.metadata ] [Nital,
Adri] [temp] creating index, cause [auto(index api)], shards [5]/[1],
mappings [content, contentFiles]
[2011-07-12 18:57:16,927][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,122][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,244][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,372][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,700][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:18,122][DEBUG][action.search.type ] [Nital,
Adri] [257386] Failed to execute fetch phase
org.elasticsearch.search.fetch.FetchPhaseExecutionException: [temp]
[0]: query[ConstantScore(:)],from[0],size[10]: Fetch Failed [Failed
to loaduidfrom the index]
at
org.elasticsearch.search.fetch.FetchPhase.extractUid(FetchPhase.java:
167)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:
80)
at
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java :
352)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFet ch(SearchServiceTransportAction.java:
314)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:
140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.access$200(TransportSearchQueryThenFetchAction.java:59)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction$1.run(TransportSearchQueryThenFetchAction.java:113)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Our documents are rather complex so I'm not exactly sure what about
that particular document (if anything) triggered the issue. Other
child documents were inserted by my test script but did not cause any
shard failures for thematch_allquery. Unfortunately I can't post the
data so I will try to whittle it down to something simple you will be
able to reproduce on your end.

On Jul 7, 5:58 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

It seems like its missing the internal _uid field that is stored in each document from the document fetched. Is it something reproducible? (I improved the failure message to include the loaded fields).

On Friday, July 8, 2011 at 12:10 AM, lmader wrote:

Failed to
loaduidfrom the index


(Shay Banon) #5

Heya, thanks for the repro!. Just so we won't forget it, can you open an
issue for it?

On Thu, Jul 21, 2011 at 2:14 AM, lmader lmaderintrepid@gmail.com wrote:

I've created a program that repros the problem:

https://gist.github.com/1096154

This is essentially a problem with parent/child documents.

Please note that the _source for the child document must be disabled
for this problem to show up. The program puts the mapping and thus
sets this.

Keep me posted on whether this program works for you to repro the
problem.

Thanks,
Lar

On Jul 12, 7:10 pm, Matt Smalley matt.smal...@gmail.com wrote:

Hi, I work with the OP and have been digging into this. I don't have a
hard example to send you just yet, but have uncovered some important
details. We use a parent/child mapping to index attachments separate
from our metadata and it appears to only occur when doing a
"match_all" against both the parent (content) and child type
(contentFiles), eg:

curl -XGEThttp://localhost:9200/temp/_search-d '{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/content,contentFiles/_search-d
'{"query":{"match_all":{}}}'

both fail, but:

curl -XGEThttp://localhost:9200/temp/content/_search-d '{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/contentFiles/_search-d
'{"query":{"match_all":{}}}'

do not. Here is log snippet, showing the index being dropped, created,
and several mapping updates before the exception trace:

[2011-07-12 18:57:14,692][INFO ][cluster.metadata ] [Nital,
Adri] [temp] deleting index
[2011-07-12 18:57:16,879][INFO ][cluster.metadata ] [Nital,
Adri] [temp] creating index, cause [auto(index api)], shards [5]/[1],
mappings [content, contentFiles]
[2011-07-12 18:57:16,927][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,122][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,244][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,372][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,700][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:18,122][DEBUG][action.search.type ] [Nital,
Adri] [257386] Failed to execute fetch phase
org.elasticsearch.search.fetch.FetchPhaseExecutionException: [temp]
[0]: query[ConstantScore(:)],from[0],size[10]: Fetch Failed [Failed
to loaduidfrom the index]
at
org.elasticsearch.search.fetch.FetchPhase.extractUid(FetchPhase.java:
167)
at
org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:
80)
at

org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java
:

  1. at

org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFet
ch(SearchServiceTransportAction.java:

  1. at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:
  2. at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction.access$200(TransportSearchQueryThenFetchAction.java:59)
    at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction$1.run(TransportSearchQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor
    $Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor
    $Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

Our documents are rather complex so I'm not exactly sure what about
that particular document (if anything) triggered the issue. Other
child documents were inserted by my test script but did not cause any
shard failures for thematch_allquery. Unfortunately I can't post the
data so I will try to whittle it down to something simple you will be
able to reproduce on your end.

On Jul 7, 5:58 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

It seems like its missing the internal _uid field that is stored in
each document from the document fetched. Is it something reproducible? (I
improved the failure message to include the loaded fields).

On Friday, July 8, 2011 at 12:10 AM, lmader wrote:

Failed to
loaduidfrom the index


(Lar Mader) #6

Issue #1149, Failed to load uid from the index in match_all query
with parent/child and _source disabled

Has been created.

Thanks!
Lar

On Jul 21, 5:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Heya, thanks for the repro!. Just so we won't forget it, can you open an
issue for it?

On Thu, Jul 21, 2011 at 2:14 AM, lmader lmaderintre...@gmail.com wrote:

I've created a program that repros the problem:

https://gist.github.com/1096154

This is essentially a problem with parent/child documents.

Please note that the _source for the child document must be disabled
for this problem to show up. The program puts the mapping and thus
sets this.

Keep me posted on whether this program works for you to repro the
problem.

Thanks,
Lar

On Jul 12, 7:10 pm, Matt Smalley matt.smal...@gmail.com wrote:

Hi, I work with the OP and have been digging into this. I don't have a
hard example to send you just yet, but have uncovered some important
details. We use a parent/child mapping to index attachments separate
from our metadata and it appears to only occur when doing a
"match_all" against both the parent (content) and child type
(contentFiles), eg:

curl -XGEThttp://localhost:9200/temp/_search-d'{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/content,contentFiles/_search-d
'{"query":{"match_all":{}}}'

both fail, but:

curl -XGEThttp://localhost:9200/temp/content/_search-d'{"query":
{"match_all":{}}}'
curl -XGEThttp://localhost:9200/temp/contentFiles/_search-d
'{"query":{"match_all":{}}}'

do not. Here is log snippet, showing the index being dropped, created,
and several mapping updates before the exception trace:

[2011-07-12 18:57:14,692][INFO ][cluster.metadata ] [Nital,
Adri] [temp] deleting index
[2011-07-12 18:57:16,879][INFO ][cluster.metadata ] [Nital,
Adri] [temp] creating index, cause [auto(index api)], shards [5]/[1],
mappings [content, contentFiles]
[2011-07-12 18:57:16,927][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,122][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,244][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,372][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:17,700][INFO ][cluster.metadata ] [Nital,
Adri] [temp] update_mapping [content] (dynamic)
[2011-07-12 18:57:18,122][DEBUG][action.search.type ] [Nital,
Adri] [257386] Failed to execute fetch phase
org.elasticsearch.search.fetch.FetchPhaseExecutionException: [temp]
[0]: query[ConstantScore(:)],from[0],size[10]: Fetch Failed [Failed
to loaduidfrom the index]
at
org.elasticsearch.search.fetch.FetchPhase.extractUid(FetchPhase.java:
167)
at
org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:
80)
at

org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java
:

  1. at

org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFet
ch(SearchServiceTransportAction.java:

  1. at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:
  2. at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction.access$200(TransportSearchQueryThenFetchAction.java:59)
    at
    org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
    $AsyncAction$1.run(TransportSearchQueryThenFetchAction.java:113)
    at java.util.concurrent.ThreadPoolExecutor
    $Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor
    $Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

Our documents are rather complex so I'm not exactly sure what about
that particular document (if anything) triggered the issue. Other
child documents were inserted by my test script but did not cause any
shard failures for thematch_allquery. Unfortunately I can't post the
data so I will try to whittle it down to something simple you will be
able to reproduce on your end.

On Jul 7, 5:58 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

It seems like its missing the internal _uid field that is stored in
each document from the document fetched. Is it something reproducible? (I
improved the failure message to include the loaded fields).

On Friday, July 8, 2011 at 12:10 AM, lmader wrote:

Failed to
loaduidfrom the index


(system) #7