How to avoid "failed to find nested object under path" when searching over many indexes?

Hi,

is it possible to search on indexes that have no nested objects in the
mapping when searching with a nested query over many indexes?

I get the following error message "failed to find nested object under
path". The situation is, some indexes have a nested object
"dc:subject.xbib:subject", some not. I would expect a successful search. It
would be nice to fail silently on those indexes where the mapping is
different, even if another mapping such as an un-nested field is present.

To work around, I must either constrain the search to the indexes that have
the nested object in the mapping, or I need to extend the nested object
mapping to all indexes, even if they do not make use of it.

Best,

Jörg

[2012-05-22 10:54:18,241][DEBUG][action.search.type ] [Blitzkrieger]
[oai][4], node[3ByGVhulSmeXXxO3jtEUxA], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@1aed904a]
org.elasticsearch.search.SearchParseException: [oai][4]: from[0],size[10]:
Parse Failure [Failed to parse source [{"from":0,"size":10, "query" :
{"nested":{"path":"dc:subject.xbib:subject","score_mode":"avg","query":{"bool":{"must":[{"term":{"dc:subject.xbib:subject.xbib:subjectAuthority":"RSWK"}},{"text":{"dc:subject.xbib:subject.xbib:subjectValue":"köln"}}]}}}}}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:557)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:470)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:228)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.elasticsearch.index.query.QueryParsingException: [oai]
[nested] failed to find nested object under path [dc:subject.xbib:subject]
at
org.elasticsearch.index.query.NestedQueryParser.parse(NestedQueryParser.java:117)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:192)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:243)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:545)

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Or you can use the indices query to send two different versions:
http://www.elasticsearch.org/guide/reference/query-dsl/indices-query.html

clint

Since I use a front-end query translator for translating from another query
language to Elasticsearch DSL, I have only one version.

Jörg

On Tuesday, May 22, 2012 11:39:24 AM UTC+2, Clinton Gormley wrote:

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Or you can use the indices query to send two different versions:
http://www.elasticsearch.org/guide/reference/query-dsl/indices-query.html

clint

You will still get a response back, just the shards belonging to the index
with no nested mapping will be marked as failed.

On Tue, May 22, 2012 at 11:19 AM, Jörg Prante joergprante@gmail.com wrote:

Hi,

is it possible to search on indexes that have no nested objects in the
mapping when searching with a nested query over many indexes?

I get the following error message "failed to find nested object under
path". The situation is, some indexes have a nested object
"dc:subject.xbib:subject", some not. I would expect a successful search. It
would be nice to fail silently on those indexes where the mapping is
different, even if another mapping such as an un-nested field is present.

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Best,

Jörg

[2012-05-22 10:54:18,241][DEBUG][action.search.type ] [Blitzkrieger]
[oai][4], node[3ByGVhulSmeXXxO3jtEUxA], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@1aed904a]
org.elasticsearch.search.SearchParseException: [oai][4]: from[0],size[10]:
Parse Failure [Failed to parse source [{"from":0,"size":10, "query" :
{"nested":{"path":"dc:subject.xbib:subject","score_mode":"avg","query":{"bool":{"must":[{"term":{"dc:subject.xbib:subject.xbib:subjectAuthority":"RSWK"}},{"text":{"dc:subject.xbib:subject.xbib:subjectValue":"köln"}}]}}}}}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:557)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:470)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:228)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.elasticsearch.index.query.QueryParsingException: [oai]
[nested] failed to find nested object under path [dc:subject.xbib:subject]
at
org.elasticsearch.index.query.NestedQueryParser.parse(NestedQueryParser.java:117)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:192)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:243)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:545)

Hi Clint,

I'm not sure this is working as expected. I would assume an Index query or Index filter would only be applied to the indicated index. In the query below, we are filtering on a nested field that exists in one index ("ideas") and does not exist in the other index ("documents").

Yet, we still get the following error for all document results

QueryParsingException[[documents] [nested] failed to find nested object under path [authors]];

Our query JSON is as follows:

{
"query": {
"filtered": {
"query": {
"custom_score": {
"query": {
"dis_max": {
"queries": [
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"name^1.0",
"network.name^1.0"
],
"operator": "AND"
}
},
"filter": {
"term": {
"doc_type": "deck"
}
}
}
},
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"name^1.0",
"body_search^1.0",
"network.name^1.0"
],
"operator": "AND"
}
},
"filter": {
"term": {
"doc_type": "file"
}
}
}
},
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"term^1.0"
],
"operator": "AND"
}
},
"filter": {
"type": {
"value": "idea"
}
}
}
}
]
}
},
"script": "(_score / 3) * (1.0 + (doc['views'].value == 0 ? 0 : log10(doc['views'].value))) * (1.0 + (doc['_type'].value == 'idea' ? ((doc['term_media'].empty ? 0 : 0.2) + (doc['definition_media'].empty ? 0 : 0.2)) : (0.0) ))* (1.0 + (0.2 * (doc['time_updated'].date.millis - 1167631200000) / (time() - 1167631200000)))",
"lang": "mvel"
}
},
"filter": {
"indices": {
"filter": {
"and": {
"filters": [
{
"type": {
"value": "idea"
}
},
{
"and": {
"filters": [
{
"nested": {
"filter": {
"and": {
"filters": [
{
"term": {
"authors.network_id": [
1793
]
}
}
]
}
},
"path": "authors"
}
}
]
}
}
]
}
},
"indices": [
"ideas"
],
"no_match_filter": {
"or": {
"filters": [
{
"and": {
"filters": [
{
"term": {
"doc_type": "file"
}
},
{
"and": {
"filters": [
{
"term": {
"network.network_id": [
1793
]
}
}
]
}
}
]
}
},
{
"and": {
"filters": [
{
"term": {
"doc_type": "deck"
}
},
{
"and": {
"filters": [
{
"term": {
"network.network_id": [
1793
]
}
}
]
}
}
]
}
}
]
}
}
}
}
}
}
}

On Thu, 2012-11-15 at 16:22 -0800, rae1creve wrote:

Hi Clint,

I'm not sure this is working as expected. I would assume an Index query or
Index filter would only be applied to the indicated index. In the query
below, we are filtering on a nested field that exists in one index ("ideas")
and does not exist in the other index ("documents").

Looks like a bug to me. I've opened this issue:

clint

--