Embedded fields not searchable when addressed by name and type of same name exists?

jpotisch · March 13, 2015, 8:18pm

I ran into the following today in Elasticsearch 1.4.4 and am trying to
determine if this is a bug in Elasticsearch or a bug in my understanding of
Elasticsearch. I'm more than willing to believe it is the latter. It should
be very reproducible with the commands I've pasted.

Let's say I have two types - books and authors - and I add one of each to
my test index:

POST /tests/authors
{ "name": "mytest12345" }

POST /tests/books
{
"title": "My big book",
"authors": [{ "name": "mytest12345" }]
}

I can perform a simple query and I will get both records back - one artist,
one book:
GET /tests/_search?q=mytest12345

This is what I would expect.

If I then query against the authors.name field within the books type, I get
my book, also just as I expect:
GET /tests/books/_search?q=authors.name:mytest12345

However, if I perform the exact same query against the root of the index
instead of against the books type
GET /tests/_search?q=authors.name:mytest12345

I instead get back the author record. The book no longer comes back at
all even though to my understanding I'm performing the same query against
all types instead of just books.

If I delete the authors type (and I confirmed deleting just the authors records
won't work)
DELETE /tests/authors

Then the query against the index root behaves as expected
GET /tests/_search?q=authors.name:mytest12345

It basically appears that the query
GET /{index}/_search?q={type}.{field}:{query}

appears to run internally as:
GET /{index}/{type}/_search?q={field}:{query}

when the {type} type exists in the index. So my question is, is this
correct behavior that I don't understand? Or is it a bug? It feels like a
bug to me, but I'll defer to the experts here. I'm happy to open a ticket
if someone more experienced than me can verify it's an issue.

Thanks,

-joel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c8757abd-a99f-4c78-805b-17101ace7982%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

vineeth_mohan_2 · March 15, 2015, 9:39am

Hello Joel ,

This is a known issue.
Internally each field is stored as typeName.fieldName format as type name
is just an abstraction.
That is one of the reason for this issue.

You can find more information here -

github.com/elastic/elasticsearch

Type name and field name conflict

opened 06:15PM - 22 Aug 14 UTC

closed 07:04PM - 22 Aug 14 UTC

Vineeth-Mohan

Steps to reproduce ``` curl -XPOST 'http://localhost:9200/test/sublocality/1' …-d '{ "name" : "xxx", "country" : { "name" : "yyy" } }' curl -XPOST 'http://localhost:9200/test/country/1' -d '{ "name" : "zzz", "country" : { "name" : "yyy" } }' curl -XPOST 'http://localhost:9200/test/_search' -d '{ "query": { "match_all": {} }, "facets": { "term": { "terms": { "field": "country.name", "size": 10 } } } }' {"took":3,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":2,"max_score":1.0,"hits":[{"_index":"test","_type":"sublocality","_id":"1","_score":1.0,"_source":{ "name" : "xxx", "country" : { "name" : "yyy" } }},{"_index":"test","_type":"country","_id":"1","_score":1.0,"_source":{ "name" : "zzz", "country" : { "name" : "yyy" } }}]},"facets":{"term":{"_type":"terms","missing":0,"total":2,"other":0,"terms":[{"term":"zzz","count":1},{"term":"xxx","count":1}]}}} ``` This gives 1 ZZZ and 1 XXX instead of 2 YYY. But then ``` { "query": { "match_all": {} }, "facets": { "term": { "terms": { "field": "country.country.name", "size": 10 } } } } ``` Gives the expected result.

Thanks
Vineeth Mohan,
Elasticsearch consultant,
qbox.io ( Elasticsearch service provider http://qbox.io/)

On Sat, Mar 14, 2015 at 1:48 AM, Joel Potischman <
joel.potischman@beatport.com> wrote:

I ran into the following today in Elasticsearch 1.4.4 and am trying to
determine if this is a bug in Elasticsearch or a bug in my understanding of
Elasticsearch. I'm more than willing to believe it is the latter. It should
be very reproducible with the commands I've pasted.

Let's say I have two types - books and authors - and I add one of each to
my test index:

POST /tests/authors
{ "name": "mytest12345" }

POST /tests/books
{
"title": "My big book",
"authors": [{ "name": "mytest12345" }]
}

I can perform a simple query and I will get both records back - one
artist, one book:
GET /tests/_search?q=mytest12345

This is what I would expect.

If I then query against the authors.name field within the books type, I
get my book, also just as I expect:
GET /tests/books/_search?q=authors.name:mytest12345

However, if I perform the exact same query against the root of the index
instead of against the books type
GET /tests/_search?q=authors.name:mytest12345

I instead get back the author record. The book no longer comes back at
all even though to my understanding I'm performing the same query against
all types instead of just books.

If I delete the authors type (and I confirmed deleting just the authors records
won't work)
DELETE /tests/authors

Then the query against the index root behaves as expected
GET /tests/_search?q=authors.name:mytest12345

It basically appears that the query
GET /{index}/_search?q={type}.{field}:{query}

appears to run internally as:
GET /{index}/{type}/_search?q={field}:{query}

when the {type} type exists in the index. So my question is, is this
correct behavior that I don't understand? Or is it a bug? It feels like a
bug to me, but I'll defer to the experts here. I'm happy to open a ticket
if someone more experienced than me can verify it's an issue.

Thanks,

-joel

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8757abd-a99f-4c78-805b-17101ace7982%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8757abd-a99f-4c78-805b-17101ace7982%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5%3DUy9JkYxBYUX4RpOFSkn_L41NsVqLrJz05PbzKbcGHNA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jpotisch · March 15, 2015, 6:26pm

Thanks, Vineeth. Much appreciated! I read your ticket and the associated
issue and that definitely seems to be it. I think I found a workaround for
my specific needs though:

It seems that if instead of querying against the root of the index, I
supply the list of types, it works as expected. Using the example data from
your issue:
curl -XPOST "http://dev-searchapi.beatport.com:9200/tests/
country,sublocality/_search" -d'
{
"query": {
"term": {
"country.name": "yyy"
}
}
}'

finds both records (one sublocality, one country) as I'd expect, whereas
the same query payload without the types returns nothing.

That seems to remove my stumbling block but if you're aware of any other
unexpected behavior I should watch out for in this regard please let me
know so I can avoid additional frustrations!

Thanks again!

-joel

On Sunday, March 15, 2015 at 5:39:47 AM UTC-4, vineeth mohan wrote:

Hello Joel ,

This is a known issue.
Internally each field is stored as typeName.fieldName format as type name
is just an abstraction.
That is one of the reason for this issue.

You can find more information here -
Type name and field name conflict · Issue #7411 · elastic/elasticsearch · GitHub

Thanks
Vineeth Mohan,
Elasticsearch consultant,
qbox.io ( Elasticsearch service provider http://qbox.io/)

On Sat, Mar 14, 2015 at 1:48 AM, Joel Potischman <joel.po...@beatport.com
<javascript:>> wrote:

I ran into the following today in Elasticsearch 1.4.4 and am trying to
determine if this is a bug in Elasticsearch or a bug in my understanding of
Elasticsearch. I'm more than willing to believe it is the latter. It should
be very reproducible with the commands I've pasted.

Let's say I have two types - books and authors - and I add one of each to
my test index:

POST /tests/authors
{ "name": "mytest12345" }

POST /tests/books
{
"title": "My big book",
"authors": [{ "name": "mytest12345" }]
}

I can perform a simple query and I will get both records back - one
artist, one book:
GET /tests/_search?q=mytest12345

This is what I would expect.

If I then query against the authors.name field within the books type, I
get my book, also just as I expect:
GET /tests/books/_search?q=authors.name:mytest12345

However, if I perform the exact same query against the root of the index
instead of against the books type
GET /tests/_search?q=authors.name:mytest12345

I instead get back the author record. The book no longer comes back at
all even though to my understanding I'm performing the same query against
all types instead of just books.

If I delete the authors type (and I confirmed deleting just the authors records
won't work)
DELETE /tests/authors

Then the query against the index root behaves as expected
GET /tests/_search?q=authors.name:mytest12345

It basically appears that the query
GET /{index}/_search?q={type}.{field}:{query}

appears to run internally as:
GET /{index}/{type}/_search?q={field}:{query}

when the {type} type exists in the index. So my question is, is this
correct behavior that I don't understand? Or is it a bug? It feels like a
bug to me, but I'll defer to the experts here. I'm happy to open a ticket
if someone more experienced than me can verify it's an issue.

Thanks,

-joel

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c8757abd-a99f-4c78-805b-17101ace7982%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c8757abd-a99f-4c78-805b-17101ace7982%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d05dd849-505b-41f5-a8f2-8cde8a523bc3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.