How to avoid "failed to find nested object under path" when searching over many indexes?

jprante · May 22, 2012, 9:19am

Hi,

is it possible to search on indexes that have no nested objects in the
mapping when searching with a nested query over many indexes?

I get the following error message "failed to find nested object under
path". The situation is, some indexes have a nested object
"dc:subject.xbib:subject", some not. I would expect a successful search. It
would be nice to fail silently on those indexes where the mapping is
different, even if another mapping such as an un-nested field is present.

To work around, I must either constrain the search to the indexes that have
the nested object in the mapping, or I need to extend the nested object
mapping to all indexes, even if they do not make use of it.

Best,

Jörg

[2012-05-22 10:54:18,241][DEBUG][action.search.type ] [Blitzkrieger]
[oai][4], node[3ByGVhulSmeXXxO3jtEUxA], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@1aed904a]
org.elasticsearch.search.SearchParseException: [oai][4]: from[0],size[10]:
Parse Failure [Failed to parse source [{"from":0,"size":10, "query" :
{"nested":{"path":"dc:subject.xbib:subject","score_mode":"avg","query":{"bool":{"must":[{"term":{"dc:subject.xbib:subject.xbib:subjectAuthority":"RSWK"}},{"text":{"dc:subject.xbib:subject.xbib:subjectValue":"köln"}}]}}}}}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:557)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:470)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:228)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.elasticsearch.index.query.QueryParsingException: [oai]
[nested] failed to find nested object under path [dc:subject.xbib:subject]
at
org.elasticsearch.index.query.NestedQueryParser.parse(NestedQueryParser.java:117)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:192)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:243)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:545)

Clinton_Gormley · May 22, 2012, 9:39am

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Or you can use the indices query to send two different versions:

clint

jprante · May 22, 2012, 9:25pm

Since I use a front-end query translator for translating from another query
language to Elasticsearch DSL, I have only one version.

Jörg

On Tuesday, May 22, 2012 11:39:24 AM UTC+2, Clinton Gormley wrote:

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Or you can use the indices query to send two different versions:
Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

kimchy · May 25, 2012, 8:30pm

You will still get a response back, just the shards belonging to the index
with no nested mapping will be marked as failed.

On Tue, May 22, 2012 at 11:19 AM, Jörg Prante joergprante@gmail.com wrote:

Hi,

is it possible to search on indexes that have no nested objects in the
mapping when searching with a nested query over many indexes?

I get the following error message "failed to find nested object under
path". The situation is, some indexes have a nested object
"dc:subject.xbib:subject", some not. I would expect a successful search. It
would be nice to fail silently on those indexes where the mapping is
different, even if another mapping such as an un-nested field is present.

To work around, I must either constrain the search to the indexes that
have the nested object in the mapping, or I need to extend the nested
object mapping to all indexes, even if they do not make use of it.

Best,

Jörg

[2012-05-22 10:54:18,241][DEBUG][action.search.type ] [Blitzkrieger]
[oai][4], node[3ByGVhulSmeXXxO3jtEUxA], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@1aed904a]
org.elasticsearch.search.SearchParseException: [oai][4]: from[0],size[10]:
Parse Failure [Failed to parse source [{"from":0,"size":10, "query" :
{"nested":{"path":"dc:subject.xbib:subject","score_mode":"avg","query":{"bool":{"must":[{"term":{"dc:subject.xbib:subject.xbib:subjectAuthority":"RSWK"}},{"text":{"dc:subject.xbib:subject.xbib:subjectValue":"köln"}}]}}}}}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:557)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:470)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:228)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:140)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.elasticsearch.index.query.QueryParsingException: [oai]
[nested] failed to find nested object under path [dc:subject.xbib:subject]
at
org.elasticsearch.index.query.NestedQueryParser.parse(NestedQueryParser.java:117)
at
org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:192)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:243)
at
org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223)
at
org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:545)

rae1creve · November 16, 2012, 12:22am

Hi Clint,

I'm not sure this is working as expected. I would assume an Index query or Index filter would only be applied to the indicated index. In the query below, we are filtering on a nested field that exists in one index ("ideas") and does not exist in the other index ("documents").

Yet, we still get the following error for all document results

QueryParsingException[[documents] [nested] failed to find nested object under path [authors]];

Our query JSON is as follows:

{
"query": {
"filtered": {
"query": {
"custom_score": {
"query": {
"dis_max": {
"queries": [
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"name^1.0",
"network.name^1.0"
],
"operator": "AND"
}
},
"filter": {
"term": {
"doc_type": "deck"
}
}
}
},
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"name^1.0",
"body_search^1.0",
"network.name^1.0"
],
"operator": "AND"
}
},
"filter": {
"term": {
"doc_type": "file"
}
}
}
},
{
"filtered": {
"query": {
"multi_match": {
"query": "biology",
"fields": [
"term^1.0"
],
"operator": "AND"
}
},
"filter": {
"type": {
"value": "idea"
}
}
}
}
]
}
},
"script": "(_score / 3) * (1.0 + (doc['views'].value == 0 ? 0 : log10(doc['views'].value))) * (1.0 + (doc['_type'].value == 'idea' ? ((doc['term_media'].empty ? 0 : 0.2) + (doc['definition_media'].empty ? 0 : 0.2)) : (0.0) ))* (1.0 + (0.2 * (doc['time_updated'].date.millis - 1167631200000) / (time() - 1167631200000)))",
"lang": "mvel"
}
},
"filter": {
"indices": {
"filter": {
"and": {
"filters": [
{
"type": {
"value": "idea"
}
},
{
"and": {
"filters": [
{
"nested": {
"filter": {
"and": {
"filters": [
{
"term": {
"authors.network_id": [
1793
]
}
}
]
}
},
"path": "authors"
}
}
]
}
}
]
}
},
"indices": [
"ideas"
],
"no_match_filter": {
"or": {
"filters": [
{
"and": {
"filters": [
{
"term": {
"doc_type": "file"
}
},
{
"and": {
"filters": [
{
"term": {
"network.network_id": [
1793
]
}
}
]
}
}
]
}
},
{
"and": {
"filters": [
{
"term": {
"doc_type": "deck"
}
},
{
"and": {
"filters": [
{
"term": {
"network.network_id": [
1793
]
}
}
]
}
}
]
}
}
]
}
}
}
}
}
}
}

Clinton_Gormley · November 16, 2012, 8:41am

On Thu, 2012-11-15 at 16:22 -0800, rae1creve wrote:

Hi Clint,

I'm not sure this is working as expected. I would assume an Index query or
Index filter would only be applied to the indicated index. In the query
below, we are filtering on a nested field that exists in one index ("ideas")
and does not exist in the other index ("documents").

Looks like a bug to me. I've opened this issue:

github.com/elastic/elasticsearch

Indices filter parsed for indices to which it should not apply

opened 08:40AM - 16 Nov 12 UTC

closed 05:30PM - 14 Nov 13 UTC

clintongormley

>bug v1.0.0.Beta2 v0.90.8

A used in the forum is trying to apply a nested filter to just one index, while …querying more than one index, some of which don't have the nested mapping. The query is complaining about a missing nested mapping, but on the indices to which the query should not apply. Create two indices: `test_1` with a nested mapping, and `test_2` without a nested mapping: ``` curl -XPUT 'http://127.0.0.1:9200/test_1/?pretty=1' -d ' { "mappings" : { "foo" : { "properties" : { "authors" : { "type" : "nested", "properties" : { "name" : { "type" : "string" } } }, "title" : { "type" : "string" } } } } } ' curl -XPUT 'http://127.0.0.1:9200/test_2/?pretty=1' -d ' { "mappings" : { "bar" : { "properties" : { "title" : { "type" : "string" } } } } } ' ``` Index some data: ``` curl -XPOST 'http://127.0.0.1:9200/test_1/foo?pretty=1' -d ' { "author" : { "name" : "john smith" }, "title" : "test 1 doc" } ' curl -XPOST 'http://127.0.0.1:9200/test_2/bar?pretty=1' -d ' { "title" : "test 2 doc" } ' ``` Query `test_1` and `test_2` but limit the nested filter to just `test_1`: ``` curl -XGET 'http://127.0.0.1:9200/test_1%2Ctest_2/_search?pretty=1' -d ' { "query" : { "filtered" : { "filter" : { "indices" : { "no_match_filter" : "none", "filter" : { "nested" : { "filter" : { "term" : { "author.name" : "john" } }, "path" : "author" } }, "indices" : [ "test_1" ] } }, "query" : { "match" : { "title" : "test" } } } } } ' ``` Throws this error: ``` SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[Yit05d94RgiUwMg9vzMOgw][test_1][1]: SearchParseException[[test_1][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{ "query" : { "filtered" : { "filter" : { "indices" : { "no_match_filter" : "none", "filter" : { "nested" : { "filter" : { "term" : { "author.name" : "john" } }, "path" : "author" } }, "indices" : [ "test_1" ] } }, "query" : { "match" : { "title" : "test" } } } } } ```

clint

--

Topic		Replies	Views
Best way to query nested objects in indexes where they don't exist Elasticsearch	5	5485	December 31, 2018
Nested filter query giving error Elasticsearch	3	595	July 6, 2017
Multiple Indices Search: Failed to find nested object under path Elasticsearch	2	3827	December 26, 2019
[nested] failed to find nested object under path [orders] Elasticsearch	2	3120	August 7, 2020
Search on a nested field on multi-indexes, multi-document types throw QueryParsingException Elasticsearch	4	2228	July 5, 2017

How to avoid "failed to find nested object under path" when searching over many indexes?

Related topics