Can ES return only the part of an array that match the query

Elastic Stack Elasticsearch
1

I have documents like this:
{
... ...,
"id":"abc001",
"nominations":[
{
"comment":"test",
"events":[
{
"name":"aaa",
"year":"2012"
},{
"name":"bbb",
"year":"2011"
}
]
},
... ...
]
}

As you can see, there are two nested array, 'nominations' and 'events'.

When I search for docs that match {"term":{"events.name":"aaa"}} and
{"term":{"events.year":"2012"}}, the document above will be returned.

But in practice, array 'events' may contains many elements like
"{"name":"bbb","year":"2011"}" that don't match the query and also are
useless to users. So my question is, can Elasticsearch return a document
with array 'events' that contains element "{"name":"aaa","year":"2012"}"
only? If not, is there any other possible ways to solve this problem? Or
should I change the structure of the document?

Thanks a lot!

--

2

You have to look at nested docs:

HTH
David

Le 13 décembre 2012 à 16:42, Margaret xiaowen.huang.bjtu@gmail.com a écrit :

I have documents like this:
{
... ...,
"id":"abc001",
"nominations":[
{
"comment":"test",
"events":[
{
"name":"aaa",
"year":"2012"
},{
"name":"bbb",
"year":"2011"
}
]
},
... ...
]
}

As you can see, there are two nested array, 'nominations' and 'events'.

When I search for docs that match {"term":{"events.name":"aaa"}} and
{"term":{"events.year":"2012"}}, the document above will be returned.

But in practice, array 'events' may contains many elements like
"{"name":"bbb","year":"2011"}" that don't match the query and also are useless
to users. So my question is, can Elasticsearch return a document with array
'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is
there any other possible ways to solve this problem? Or should I change the
structure of the document?

Thanks a lot!

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

3

Thanks for reply, David.

Let me describe my problem more explicitly.
I tried the example given in Nested Type(
Elasticsearch Platform — Find real-time answers at scale | Elastic) and
Nested Query:
$ curl -XGET 'http://localhost:9200/test/es/_search' -d '
{
"query":{
"nested":{
"path":"obj1",
"query":{
"bool":{
"must":[
{
"text":{"obj1.name":"blue"}
},
{
"range":{"obj1.count":{"lt":5}}
}
]
}
}
}
}
}'.

The query result is:
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915,
"_source" :
{
"obj1":[
{
"name":"blue",
"count":4
}
,{
"name":"green",
"count":6
}
]
}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can
ES return result like this:
{
"obj1":[
{
"name":"blue",
"count":4
}
]
}
?

Thanks a lot.

2012/12/14 David Pilato david@pilato.fr

**
You have to look at nested docs:
Elasticsearch Platform — Find real-time answers at scale | Elastic

HTH
David

Le 13 décembre 2012 à 16:42, Margaret xiaowen.huang.bjtu@gmail.com a
écrit :

I have documents like this:
{
... ...,
"id":"abc001",
"nominations":[
{
"comment":"test",
"events":[
{
"name":"aaa",
"year":"2012"
},{
"name":"bbb",
"year":"2011"
}
]
},
... ...
]
}

As you can see, there are two nested array, 'nominations' and 'events'.

When I search for docs that match {"term":{"events.name":"aaa"}} and
{"term":{"events.year":"2012"}}, the document above will be returned.

But in practice, array 'events' may contains many elements like
"{"name":"bbb","year":"2011"}" that don't match the query and also are
useless to users. So my question is, can Elasticsearch return a document
with array 'events' that contains element "{"name":"aaa","year":"2012"}"
only? If not, is there any other possible ways to solve this problem? Or
should I change the structure of the document?

Thanks a lot!

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--
Thanks,
*
*
*Xiaowen Huang *| 黄晓雯
Student of School of Software Engineering
Beijing Jiaotong University
Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China
Tel.: +86-152-1058-2730
Email: Xiaowen.Huang.bjtu@gmail.com

--

4

Can you gist a curl recreation?
It will help to see what is your mapping and how you create your doc.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 déc. 2012 à 04:17, Xiaowen Huang xiaowen.huang.bjtu@gmail.com a écrit :

Thanks for reply, David.

Let me describe my problem more explicitly.
I tried the example given in Nested Type(Elasticsearch Platform — Find real-time answers at scale | Elastic) and Nested Query:
$ curl -XGET 'http://localhost:9200/test/es/_search' -d '
{
"query":{
"nested":{
"path":"obj1",
"query":{
"bool":{
"must":[
{
"text":{"obj1.name":"blue"}
},
{
"range":{"obj1.count":{"lt":5}}
}
]
}
}
}
}
}'.

The query result is:
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915, "_source" :
{
"obj1":[
{
"name":"blue",
"count":4
}
,{
"name":"green",
"count":6
}
]
}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can ES return result like this:
{
"obj1":[
{
"name":"blue",
"count":4
}
]
}
?

Thanks a lot.

2012/12/14 David Pilato david@pilato.fr
You have to look at nested docs: Elasticsearch Platform — Find real-time answers at scale | Elastic

HTH
David

Le 13 décembre 2012 à 16:42, Margaret xiaowen.huang.bjtu@gmail.com a écrit :

I have documents like this:
{
... ...,
"id":"abc001",
"nominations":[
{
"comment":"test",
"events":[
{
"name":"aaa",
"year":"2012"
},{
"name":"bbb",
"year":"2011"
}
]
},
... ...
]
}

As you can see, there are two nested array, 'nominations' and 'events'.

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned.

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document?

Thanks a lot!

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
Thanks,

Xiaowen Huang | 黄晓雯
Student of School of Software Engineering
Beijing Jiaotong University
Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China
Tel.: +86-152-1058-2730
Email: Xiaowen.Huang.bjtu@gmail.com

--

--

5

You want less than all the member of the elements in the array. Oh how
much, I'd like that too.

My understanding is that nested objects can prevent "cross object"
matches (searching for +name:"blue" +count:6 and incorrectly getting a
match), but when you match the parent document with a nested query you
can only get back either
(1) the whole parent (outer) document, or
(2) certain stored fields of the parent
that is all.

I believe at this time, the only thing you can do is ask for documents
that have at least 1 nested or 1 child document that matches.
If you switch to children docs you could:
(1) search for parent documents
(2) search for children documents that match your query of children, but
have the right parents.

Then combine the results.

If the criteria for matching children is not for scoring the relevancy
of the parent, you can put the matching of children in a filter and make
sure that result is cached, so even though you are going back again and
repeating the same possibly complex filter, the filtered children is
already done and it need only be further filtered by the requested
parent IDs.

Because of filter caching two requests can be very fast.

-Paul

On 12/13/2012 7:17 PM, Xiaowen Huang wrote:

The query result is:
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915,
"_source" :
{
"obj1":[
{
"name":"blue",
"count":4
}
,{
"name":"green",
"count":6
}
]
}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the
query. Can ES return result like this:
{
"obj1":[
{
"name":"blue",
"count":4
}
]
}
?

Thanks a lot.

--