Can't find anything with nested queries with 0.17.1

Elastic Stack Elasticsearch
1

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUT http://localhost:9200/echo -d '{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUT http://localhost:9200/echo/dataset/_mapping -d '
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}
}'

Then index a document:

curl -XPOST http://localhost:9200/_bulk -d '
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGET http://localhost:9200/echo/dataset/_search?pretty=true -d '
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}
}
'

I even tried a simpler one:

curl -XGET http://localhost:9200/echo/dataset/_search?pretty=true -d '
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}
}
'

If I drop the conditions, this works:

curl -XGET http://localhost:9200/echo/dataset/_search?pretty=true -d '
{
"query" : { "match_all" : {} }
}
'

I'm stuck.... what am I missing? Thanks -- Dan

2

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGET http://localhost:9200/echo/dataset/_search?pretty=true -d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}
}

'

But still doesn't get this one working:
curl -XGET http://localhost:9200/echo/dataset/_search?pretty=true -d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}
}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d '{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d '
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d '
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d '
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d '
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d '
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

3

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

4

Thank you Dan for your monologue, it helped me understand what nested documents and nested query actually are :slight_smile:

If I can suggest improvement to the documentation, it would be great if it can state which "document" is actually returned by a nested query.

It doesn't really state that querying nested doc returned the "root" doc. Maybe obvious, but not that clear to me.

And in that page the links which references the nested mapping is broken.

Nicolas

Le 24 juil. 2011 à 17:14, Dan Pilone a écrit :

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

5

2011/7/25 Nicolas Lalevée nicolas.lalevee@hibnet.org

Thank you Dan for your monologue, it helped me understand what nested
documents and nested query actually are :slight_smile:

If I can suggest improvement to the documentation, it would be great if it
can state which "document" is actually returned by a nested query.
Elasticsearch Platform — Find real-time answers at scale | Elastic
It doesn't really state that querying nested doc returned the "root" doc.
Maybe obvious, but not that clear to me.

Its stated there: "... resulting in the root parent doc (or parent nested
mapping) ... ", right in the first para

And in that page the links which references the nested mapping is broken.

Fixed.

Nicolas

Le 24 juil. 2011 à 17:14, Dan Pilone a écrit :

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

6

Le 25 juil. 2011 à 14:22, Shay Banon a écrit :

2011/7/25 Nicolas Lalevée nicolas.lalevee@hibnet.org
Thank you Dan for your monologue, it helped me understand what nested documents and nested query actually are :slight_smile:

If I can suggest improvement to the documentation, it would be great if it can state which "document" is actually returned by a nested query.
Elasticsearch Platform — Find real-time answers at scale | Elastic
It doesn't really state that querying nested doc returned the "root" doc. Maybe obvious, but not that clear to me.

Its stated there: "... resulting in the root parent doc (or parent nested mapping) ... ", right in the first para

........ shame on me......

And in that page the links which references the nested mapping is broken.

Fixed.

thanks !

Nicolas

Nicolas

Le 24 juil. 2011 à 17:14, Dan Pilone a écrit :

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

7

x^x

-----Original Message-----
From: Nicolas Lalevée nicolas.lalevee@hibnet.org
Date: Mon, 25 Jul 2011 11:13:03
To: users@elasticsearch.com
Reply-To: users@elasticsearch.com
Subject: Re: Can't find anything with nested queries with 0.17.1

Thank you Dan for your monologue, it helped me understand what nested documents and nested query actually are :slight_smile:

If I can suggest improvement to the documentation, it would be great if it can state which "document" is actually returned by a nested query.

It doesn't really state that querying nested doc returned the "root" doc. Maybe obvious, but not that clear to me.

And in that page the links which references the nested mapping is broken.

Nicolas

Le 24 juil. 2011 à 17:14, Dan Pilone a écrit :

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOST http://localhost:9200/_bulk -d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan

8

I'm glad someone got value out of my rambling. Anyway, to close the
loop, with 0.17.1 and the nested document queries we're seeing
dramatically improved performance versus parent/child querying. If
you're on the fence about transition, it really made a huge (order of
magnitude+) difference to our searches. Now I am a little concerned
about how "experimental" this feature is, but we're going to
transition to it for now due to the dramatic difference it made in
performance. Thanks again for the great work. -- Dan

On Jul 25, 9:14 am, "Anthony Nystrom" nystrom.anth...@gmail.com
wrote:

x^x

-----Original Message-----
From: Nicolas Lalevée nicolas.lale...@hibnet.org
Date: Mon, 25 Jul 2011 11:13:03
To: us...@elasticsearch.com

Reply-To: us...@elasticsearch.com
Subject: Re: Can't find anything with nested queries with 0.17.1

Thank you Dan for your monologue, it helped me understand what nested documents and nested query actually are :slight_smile:

If I can suggest improvement to the documentation, it would be great if it can state which "document" is actually returned by a nested query.Elasticsearch Platform — Find real-time answers at scale | Elastic
It doesn't really state that querying nested doc returned the "root" doc. Maybe obvious, but not that clear to me.

And in that page the links which references the nested mapping is broken.

Nicolas

Le 24 juil. 2011 à 17:14, Dan Pilone a écrit :

Ok, apparently the secret to solving any problem is to post it
somewhere public where you can prove you don't know what you're
doing. Another typo in my indexing. Now both queries work as
expected:

curl -XPOSThttp://localhost:9200/_bulk-d '{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"last_name" : "Pilone"}}
'

Now both queries work and find that document. On to testing multiple
"inner_thing"s. Thanks for listening. -- Dan

On Jul 24, 11:12 am, Dan Pilone d...@element84.com wrote:

Sigh - ok, not two seconds after posting that I realized where I
messed up in creating the inner object. You should index the document
like this:

curl -XPOSThttp://localhost:9200/_bulk-d'{"index":
{"_index":"echo","_type":"dataset","_id":"C1000000000-PROV2"}}
{"provider_id":"PROV2", "inner_thing" :{ "first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}}
'

That makes this search work as expected:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}

}

'

But still doesn't get this one working:
curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d
'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}

}

'

Still digging... -- Dan

On Jul 24, 10:56 am, Dan Pilone d...@element84.com wrote:

All,
I'm stuck. We have an index of approx 100M documents with lots of
parent/child relationships. Some of our parent/child queries are
extremely slow so we wanted to look at the new nested object/query
capabilities in 0.17.1. We tried converting one of our parent/child
relationships over but couldn't ever find results. Figuring we just
had something wrong with our query generation I started putting
together a really simple example but I can't get that working either.
Can someone please help?

First, I started with a completely new elastic 0.17.1 install.
Next, create an index as follows:

curl -XPUThttp://localhost:9200/echo-d'{ "settings" : { "index" :
{ "number_of_shards" : 3 } } }'

Then I loaded a simple mapping:

curl -XPUThttp://localhost:9200/echo/dataset/_mapping-d'
{
"dataset": {
"properties": {
"provider_id":{"type":"string","index":"not_analyzed"},
"inner_thing" : {
"type" : "nested",
"properties" : {
"first_name" : { "type" : "string", "index" :
"not_analyzed"},
"last_name" : { "type" : "string", "index" :
"not_analyzed"}
}
}
}
}

}'

Then index a document:

curl -XPOSThttp://localhost:9200/_bulk-d'
{"index":{"_index":"echo","_type":"dataset","_id":"C1000000000-
PROV2"}}
{"provider_id":"PROV2", "inner_thing.first_name" : "Dan",
"inner_thing.last_name" : "Pilone"}
'

Finally, try to find it:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : [
{"text" : { "inner_thing.first_name" :
"Dan" }},
{"text" : { "inner_thing.last_name" :
"Pilone"}}
]
}
}
}
}
}
}}

'

I even tried a simpler one:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"nested" : {
"path" : "inner_thing",
"query" : {
"bool" : {
"must" : {
"text" : { "first_name" : "Dan" }
}
}
}
}
}
}
}}

'

If I drop the conditions, this works:

curl -XGEThttp://localhost:9200/echo/dataset/_search?pretty=true-d'
{
"query" : { "match_all" : {} }}

'

I'm stuck.... what am I missing? Thanks -- Dan