New to ES. Querying a Document using a path


(David Harrigan) #1

Hi,

I put this into IRC, but got no response :slight_smile: Perhaps to early for people
:slight_smile:

Hi. I'm puzzled about querying ES - just learning about it. I have this
document as part of an index called "merchants". You can seen an example
here : http://pastebin.com/3grwDA2J
I thought I could do a query like this:

.../_search?q=employees.credentials.uid:yyyy

Since in my mind, clearly uid is a property of credentials which is a
property of employees. So I thought I would "walk the path" to narrow the
search and get the result.

This does not work, i.e., I get no results (based upon the document I have
shown as an example above).

However when I installed the "elasticsearch-head" plugin, I noticed that ES
has "flattened" my document, so this search "_search?q=uid:yyyy" works.

I need help to understand why this is and why doing a what I expected to
do, i.e., employees.credentials.uid failed.

Does ES index every property of a document, no matter if it's a nested
array, object, etc.?

Thank you.

-=david=-

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Josh Canfield) #2

Hi David.

The UI for head flattens things, but the mapping should have your structure.
What you've described should work fine. Here is a working example.

curl -XPOST 'http://localhost:9200/test/employee' -d '{
"name": "Garden Kitchen",
"employees": [
{
"firstName": "Joe",
"lastName": "Bloggs",
"state": 1,
"role": 0,
"credentials": [
{
"token": "XXXXX",
"_id": "kOqTUpL",
"uid": "yyyy@yyyy.co.uk"
}
],
"_id": "gl8gnoY",
"email": "yyyy@yyyy.co.uk"
}
]
}'

curl
http://localhost:9200/test/employee/_search?q=employees.credentials.uid:yyyy

Returns the data.

curl -s http://localhost:9200/test/employee/_mapping | python -m json.tool
{
"employee": {
"properties": {
"employees": {
"properties": {
"_id": {
"type": "string"
},
"credentials": {
"properties": {
"_id": {
"type": "string"
},
"token": {
"type": "string"
},
"uid": {
"type": "string"
}
}
},
"email": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"role": {
"type": "long"
},
"state": {
"type": "long"
}
}
},
"name": {
"type": "string"
}
}
}
}

Josh

On Thu, Oct 24, 2013 at 1:06 AM, David Harrigan dharrigan@gmail.com wrote:

Hi,

I put this into IRC, but got no response :slight_smile: Perhaps to early for people
:slight_smile:

Hi. I'm puzzled about querying ES - just learning about it. I have this
document as part of an index called "merchants". You can seen an example
here : http://pastebin.com/3grwDA2J
I thought I could do a query like this:

.../_search?q=employees.credentials.uid:yyyy

Since in my mind, clearly uid is a property of credentials which is a
property of employees. So I thought I would "walk the path" to narrow the
search and get the result.

This does not work, i.e., I get no results (based upon the document I have
shown as an example above).

However when I installed the "elasticsearch-head" plugin, I noticed that
ES has "flattened" my document, so this search "_search?q=uid:yyyy" works.

I need help to understand why this is and why doing a what I expected to
do, i.e., employees.credentials.uid failed.

Does ES index every property of a document, no matter if it's a nested
array, object, etc.?

Thank you.

-=david=-

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Harrigan) #3

Hi,

Thank you for your reply. I don't seem to get results, here is what I get:

curl
http://localhost:9200/merchants/_search?q=employees.credentials.token:xxxx
{"took":76,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

I've put the mapping file here:

http://pastebin.com/W8qzgC79

since it's quite big (8K).

I'm quite lost about why I'm not getting results back!

Thank you.

-=david=-

On 24 October 2013 22:00, Josh Canfield joshcanfield@gmail.com wrote:

Hi David.

The UI for head flattens things, but the mapping should have your
structure.
What you've described should work fine. Here is a working example.

curl -XPOST 'http://localhost:9200/test/employee' -d '{
"name": "Garden Kitchen",
"employees": [
{
"firstName": "Joe",
"lastName": "Bloggs",
"state": 1,
"role": 0,
"credentials": [
{
"token": "XXXXX",
"_id": "kOqTUpL",
"uid": "yyyy@yyyy.co.uk"
}
],
"_id": "gl8gnoY",
"email": "yyyy@yyyy.co.uk"
}
]
}'

curl
http://localhost:9200/test/employee/_search?q=employees.credentials.uid:yyyy

Returns the data.

curl -s http://localhost:9200/test/employee/_mapping | python -m json.tool
{
"employee": {
"properties": {
"employees": {
"properties": {
"_id": {
"type": "string"
},
"credentials": {
"properties": {
"_id": {
"type": "string"
},
"token": {
"type": "string"
},
"uid": {
"type": "string"
}
}
},
"email": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"role": {
"type": "long"
},
"state": {
"type": "long"
}
}
},
"name": {
"type": "string"
}
}
}
}

Josh

On Thu, Oct 24, 2013 at 1:06 AM, David Harrigan dharrigan@gmail.comwrote:

Hi,

I put this into IRC, but got no response :slight_smile: Perhaps to early for people
:slight_smile:

Hi. I'm puzzled about querying ES - just learning about it. I have this
document as part of an index called "merchants". You can seen an example
here : http://pastebin.com/3grwDA2J
I thought I could do a query like this:

.../_search?q=employees.credentials.uid:yyyy

Since in my mind, clearly uid is a property of credentials which is a
property of employees. So I thought I would "walk the path" to narrow the
search and get the result.

This does not work, i.e., I get no results (based upon the document I
have shown as an example above).

However when I installed the "elasticsearch-head" plugin, I noticed that
ES has "flattened" my document, so this search "_search?q=uid:yyyy" works.

I need help to understand why this is and why doing a what I expected to
do, i.e., employees.credentials.uid failed.

Does ES index every property of a document, no matter if it's a nested
array, object, etc.?

Thank you.

-=david=-

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Gmj57LU_zR8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
I prefer encrypted and signed messages. KeyID: B20A22F9
Fingerprint: 110A F423 3647 54E2 880F ADAD 1C52 85BF B20A 22F9

"It is not usually until you've built and used a version of the program
that you understand the issues well enough to get the design right." - Rob
Pike, Brian Kernighan.

No trees were harmed in the sending of this message, however, a number of
electrons were inconvenienced.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #4

From your mapping, it looks like employees.credentials.uid is not the
full path name. It is in fact: doc.employees.credentials.uid or, if you
include the type name: couchbaseDocument.doc.employees.credentials.uid

This is a peculiarity of how couchbase maps its documents to elasticsearch.
It shifts the document itself under doc because it adds a meta element
as well.

On 25 October 2013 10:12, David Harrigan dharrigan@gmail.com wrote:

Hi,

Thank you for your reply. I don't seem to get results, here is what I get:

curl
http://localhost:9200/merchants/_search?q=employees.credentials.token:xxxx

{"took":76,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

I've put the mapping file here:

http://pastebin.com/W8qzgC79

since it's quite big (8K).

I'm quite lost about why I'm not getting results back!

Thank you.

-=david=-

On 24 October 2013 22:00, Josh Canfield joshcanfield@gmail.com wrote:

Hi David.

The UI for head flattens things, but the mapping should have your
structure.
What you've described should work fine. Here is a working example.

curl -XPOST 'http://localhost:9200/test/employee' -d '{
"name": "Garden Kitchen",
"employees": [
{
"firstName": "Joe",
"lastName": "Bloggs",
"state": 1,
"role": 0,
"credentials": [
{
"token": "XXXXX",
"_id": "kOqTUpL",
"uid": "yyyy@yyyy.co.uk"
}
],
"_id": "gl8gnoY",
"email": "yyyy@yyyy.co.uk"
}
]
}'

curl
http://localhost:9200/test/employee/_search?q=employees.credentials.uid:yyyy

Returns the data.

curl -s http://localhost:9200/test/employee/_mapping | python -m
json.tool
{
"employee": {
"properties": {
"employees": {
"properties": {
"_id": {
"type": "string"
},
"credentials": {
"properties": {
"_id": {
"type": "string"
},
"token": {
"type": "string"
},
"uid": {
"type": "string"
}
}
},
"email": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"role": {
"type": "long"
},
"state": {
"type": "long"
}
}
},
"name": {
"type": "string"
}
}
}
}

Josh

On Thu, Oct 24, 2013 at 1:06 AM, David Harrigan dharrigan@gmail.comwrote:

Hi,

I put this into IRC, but got no response :slight_smile: Perhaps to early for people
:slight_smile:

Hi. I'm puzzled about querying ES - just learning about it. I have this
document as part of an index called "merchants". You can seen an example
here : http://pastebin.com/3grwDA2J
I thought I could do a query like this:

.../_search?q=employees.credentials.uid:yyyy

Since in my mind, clearly uid is a property of credentials which is a
property of employees. So I thought I would "walk the path" to narrow the
search and get the result.

This does not work, i.e., I get no results (based upon the document I
have shown as an example above).

However when I installed the "elasticsearch-head" plugin, I noticed that
ES has "flattened" my document, so this search "_search?q=uid:yyyy" works.

I need help to understand why this is and why doing a what I expected to
do, i.e., employees.credentials.uid failed.

Does ES index every property of a document, no matter if it's a nested
array, object, etc.?

Thank you.

-=david=-

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Gmj57LU_zR8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
I prefer encrypted and signed messages. KeyID: B20A22F9
Fingerprint: 110A F423 3647 54E2 880F ADAD 1C52 85BF B20A 22F9

"It is not usually until you've built and used a version of the program
that you understand the issues well enough to get the design right." - Rob
Pike, Brian Kernighan.

No trees were harmed in the sending of this message, however, a number of
electrons were inconvenienced.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Harrigan) #5

Hi,

Thank you for that. When I put doc. in front of the query it worked, so I
understand now why it failed originally. Thank you very much Clinton and
Josh for your replies :slight_smile:

-=david=-

On 25 October 2013 13:00, Clinton Gormley clint@traveljury.com wrote:

From your mapping, it looks like employees.credentials.uid is not the
full path name. It is in fact: doc.employees.credentials.uid or, if you
include the type name: couchbaseDocument.doc.employees.credentials.uid

This is a peculiarity of how couchbase maps its documents to
elasticsearch. It shifts the document itself under doc because it adds a
meta element as well.

On 25 October 2013 10:12, David Harrigan dharrigan@gmail.com wrote:

Hi,

Thank you for your reply. I don't seem to get results, here is what I get:

curl
http://localhost:9200/merchants/_search?q=employees.credentials.token:xxxx

{"took":76,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

I've put the mapping file here:

http://pastebin.com/W8qzgC79

since it's quite big (8K).

I'm quite lost about why I'm not getting results back!

Thank you.

-=david=-

On 24 October 2013 22:00, Josh Canfield joshcanfield@gmail.com wrote:

Hi David.

The UI for head flattens things, but the mapping should have your
structure.
What you've described should work fine. Here is a working example.

curl -XPOST 'http://localhost:9200/test/employee' -d '{
"name": "Garden Kitchen",
"employees": [
{
"firstName": "Joe",
"lastName": "Bloggs",
"state": 1,
"role": 0,
"credentials": [
{
"token": "XXXXX",
"_id": "kOqTUpL",
"uid": "yyyy@yyyy.co.uk"
}
],
"_id": "gl8gnoY",
"email": "yyyy@yyyy.co.uk"
}
]
}'

curl
http://localhost:9200/test/employee/_search?q=employees.credentials.uid:yyyy

Returns the data.

curl -s http://localhost:9200/test/employee/_mapping | python -m
json.tool
{
"employee": {
"properties": {
"employees": {
"properties": {
"_id": {
"type": "string"
},
"credentials": {
"properties": {
"_id": {
"type": "string"
},
"token": {
"type": "string"
},
"uid": {
"type": "string"
}
}
},
"email": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"role": {
"type": "long"
},
"state": {
"type": "long"
}
}
},
"name": {
"type": "string"
}
}
}
}

Josh

On Thu, Oct 24, 2013 at 1:06 AM, David Harrigan dharrigan@gmail.comwrote:

Hi,

I put this into IRC, but got no response :slight_smile: Perhaps to early for
people :slight_smile:

Hi. I'm puzzled about querying ES - just learning about it. I have this
document as part of an index called "merchants". You can seen an example
here : http://pastebin.com/3grwDA2J
I thought I could do a query like this:

.../_search?q=employees.credentials.uid:yyyy

Since in my mind, clearly uid is a property of credentials which is a
property of employees. So I thought I would "walk the path" to narrow the
search and get the result.

This does not work, i.e., I get no results (based upon the document I
have shown as an example above).

However when I installed the "elasticsearch-head" plugin, I noticed
that ES has "flattened" my document, so this search "_search?q=uid:yyyy"
works.

I need help to understand why this is and why doing a what I expected
to do, i.e., employees.credentials.uid failed.

Does ES index every property of a document, no matter if it's a nested
array, object, etc.?

Thank you.

-=david=-

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Gmj57LU_zR8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
I prefer encrypted and signed messages. KeyID: B20A22F9
Fingerprint: 110A F423 3647 54E2 880F ADAD 1C52 85BF B20A 22F9

"It is not usually until you've built and used a version of the program
that you understand the issues well enough to get the design right." - Rob
Pike, Brian Kernighan.

No trees were harmed in the sending of this message, however, a number of
electrons were inconvenienced.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Gmj57LU_zR8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
I prefer encrypted and signed messages. KeyID: B20A22F9
Fingerprint: 110A F423 3647 54E2 880F ADAD 1C52 85BF B20A 22F9

"It is not usually until you've built and used a version of the program
that you understand the issues well enough to get the design right." - Rob
Pike, Brian Kernighan.

No trees were harmed in the sending of this message, however, a number of
electrons were inconvenienced.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6