Multiple AND on nested objects not working

joa · September 17, 2013, 11:51am

Hi, my I am using the following mapping for a flexible metadata structure
(k = key/name, i = integer values, s = string values):

{
"my_index" : {
"my_type" : {
"properties" : {
"metadata" : {
"type" : "nested",
"properties" : {
"i" : {
"type" : "integer"
},
"k" : {
"type" : "string"
},
"s" : {
"type" : "string"
}
}
}
}
}
}
}

Here is how sample data might look like:
{
"metadata": [
{
"k": "autor",
"s": "Arlene Franklin"
},
{
"k": "year1",
"i": 2001
},
{
"k": "year2",
"i": 2013
}
]
}

I am now trying to do queries like: All docs where year1 greater 1990 AND
where autor is "Arlene Franklin".
I've tried the following without success (query works, but I got no hits):
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "metadata",
"query": {
"filtered": {
"filter": {
"and": [
{
"and": [
{
"term": {
"metadata.k": "year1"
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]
},
{
"and": [
{
"term": {
"metadata.k": "autor"
}
},
{
"term": {
"metadata.s": "Arlene Franklin"
}
}
]
}
]
}
}
}
}
}
}
}
}

Is my mapping working at all for queries like this? Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

mvg · September 17, 2013, 1:43pm

The use of the nested seems correct. Can you try to change the term
filter into a match query that is wrapped a query filter?

The term filter (and query) doesn't take query time text analysis into
account, the match query does. ("Arlene Franklin" is handled at a single
token while it should be handled as ["arlene", "franklin"], so is also how
a string field is indexed as default).

On 17 September 2013 13:51, joa joafeldmann@gmail.com wrote:

Hi, my I am using the following mapping for a flexible metadata structure
(k = key/name, i = integer values, s = string values):

{

"my_index" : {
"my_type" : {
  "properties" : {
    "metadata" : {

      "type" : "nested",

      "properties" : {
        "i" : {

          "type" : "integer"
        },

        "k" : {
          "type" : "string"

        },
        "s" : {

          "type" : "string"
        }

      }
    }
  }
}
}
}

Here is how sample data might look like:
{
"metadata": [
{
"k": "autor",
"s": "Arlene Franklin"
},
{
"k": "year1",
"i": 2001
},
{
"k": "year2",
"i": 2013
}
]
}

I am now trying to do queries like: All docs where year1 greater 1990 AND
where autor is "Arlene Franklin".
I've tried the following without success (query works, but I got no hits):
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "metadata",
"query": {
"filtered": {
"filter": {
"and": [
{
"and": [
{
"term": {
"metadata.k": "year1"
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]
},
{
"and": [
{
"term": {
"metadata.k": "autor"
}
},
{
"term": {
"metadata.s": "Arlene Franklin"
}
}
]
}
]
}
}
}
}
}
}
}
}

Is my mapping working at all for queries like this? Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

joa · September 17, 2013, 1:48pm

Hi Martijn, thanks. I'll try that. But even working with two range queries
returns no hits:

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "metadata",
"query": {
"filtered": {
"filter": {
"and": [
{
"and": [
{
"term": {
"metadata.k": "year1"
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]
},
{
"and": [
{
"term": {
"metadata.k": "year2"
}
},
{
"range": {
"metadata.i": {
"lt": 2020
}
}
}
]
}
]
}
}
}
}
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

joa · September 17, 2013, 1:59pm

Should I replace all term filters or only the one used by the autor ("metadata.s":
"Arlene Franklin") ? If you mean all term filters, it the following as you
suggested?

...
"and": [
{
"query": {
"match": {
"metadata.k": "year1"
}
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

mvg · September 17, 2013, 2:10pm

Only change the term filter with: ("metadata.s": "Arlene Franklin")

Also I think you need to change to top level and filter in the second
filtered query in a or filter, because there is no inner nested object
with year1 AND year2.

On 17 September 2013 15:59, joa joafeldmann@gmail.com wrote:

Should I replace all term filters or only the one used by the autor ("metadata.s":
"Arlene Franklin") ? If you mean all term filters, it the following as
you suggested?

...
"and": [
{
"query": {
"match": {
"metadata.k": "year1"
}
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]...

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

joa · September 17, 2013, 2:36pm

I am not sure if I get you right, but changing any of the "and"s to an "or"
will lead to different results: I really want to get only documents, where
e.g. year1=2004 AND year2=2006. I think using an or filter would change my
results?
As this is not a productive environment I can also change my mapping, if
you think the current is not optimal for this. Thanks again.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

johtani · September 17, 2013, 4:02pm

Hi

This query is right?
I try to send sevral pattern query.
Sorry, I don't know different between this query and your query...

{
"explain": true,
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"nested": {
"path": "metadata",
"query": {
"filtered": {
"filter": {
"and": [
{
"term": {
"metadata.k": "year1"
}
},
{
"range": {
"metadata.i": {
"gt": 1990
}
}
}
]
}
}
}
}
},
{
"nested": {
"path": "metadata",
"query": {
"filtered": {
"filter": {
"and": [
{
"term": {
"metadata.k": "autor"
}
},
{
"query": {
"match": {
"metadata.s": "Arlene Franklin"
}
}
}
]
}
}
}
}
}
]
}
}
}
}

2013/9/17 joa joafeldmann@gmail.com

I am not sure if I get you right, but changing any of the "and"s to an
"or" will lead to different results: I really want to get only documents,
where e.g. year1=2004 AND year2=2006. I think using an or filter would
change my results?
As this is not a productive environment I can also change my mapping, if
you think the current is not optimal for this. Thanks again.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Jun Ohtani
blog : http://blog.johtani.info

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

joa · September 17, 2013, 4:22pm

Hi Jun,

your query works, thanks! Including nested": { "path": "metadata" } mutiple
times like you did solved it. I wonder if there a are more elegant ways to
write this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

johtani · September 18, 2013, 3:11am

Hi

I wonder if there a are more elegant ways to write this?

Sorry, I don't know elegant way.
Maybe "path" field is need to each filter, because "nested" filter requires "path" field.

Jun Ohtani
twitter : http://twitter.com/johtani

On 2013/09/18, at 1:22, joa joafeldmann@gmail.com wrote:

Hi Jun,

your query works, thanks! Including nested": { "path": "metadata" } mutiple times like you did solved it. I wonder if there a are more elegant ways to write this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.