Conflict between fields of same name under different types

Elastic Stack Elasticsearch
1

Hi,

I have a field 'customValue' typed as integer in type 'type1' and as long
in type 'type2'. Terms facets on the field is failing. Following is the
gist:

curl -XDELETE 'http://localhost:9200/try_test/'

curl -XPOST http://localhost:9200/try_test -d '{
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
}'

curl -XPUT 'http://localhost:9200/try_test/type1/_mapping' -d '{
"type1": {
"_source": {
"enabled": true
},
"properties": {
"customId": {
"type": "integer",
"index": "analyzed"
},
"customValue": {
"type": "integer",
"index": "analyzed"
}
}
}
}'

curl -XPUT 'http://localhost:9200/try_test/type2/_mapping' -d '{
"type2": {
"_source": {
"enabled": true
},
"properties": {
"customId": {
"type": "integer",
"index": "analyzed"
},
"customValue": {
"type": "long",
"index": "analyzed"
}
}
}
}'

curl -XPOST 'http://localhost:9200/try_test/type1/' -d '{
"customId": 2301,
"customValue": 67
}'

curl -XPOST 'http://localhost:9200/try_test/type2/' -d '{
"customId": 2302,
"customValue": 678917349187249
}'

curl -XGET 'http://localhost:9200/try_test/_search?pretty=true' -d '{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Failing, which is expected. Trying to facet simultaneously on two

different types can be illegal.

curl -XGET 'http://localhost:9200/try_test/type1/_search?pretty=true' -d '{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Failing, why? It should give result, as does the next one.

curl -XGET 'http://localhost:9200/try_test/type2/_search?pretty=true' -d '{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Working.

The exception is the following :

{
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query_fetch], total failure; shardFailures
{[h5W2XmgPTKS6pDylfkqBZQ][try_test][0]:
QueryPhaseExecutionException[[try_test][0]:
query[ConstantScore(NotDeleted(cache(_type:type1)))],from[0],size[0]: Query
Failed [Failed to execute main query]]; nested:
NumberFormatException[Invalid shift value in prefixCoded string (is encoded
value really an INT?)]; }]",
"status" : 500
}

Any hint on how to resolve this? Making everything as long is one solution,
I know, but other than that? This is failing on both 0.19 and 0.20 version.

Thanks in advance,
-- Sujoy.

--

2

I'm not sure this is relevant, but from

Field names with the same name across types are highly recommended to have
the same type and same mapping characteristics (analysis settings for
example). There is an effort to allow to explicitly “choose” which field to
use by using type prefix (my_type.my_field), but it’s not complete, and
there are places where it will never work (like faceting on the field).

On Friday, 18 January 2013 05:08:41 UTC-5, Sujoy Sett wrote:

Hi,

I have a field 'customValue' typed as integer in type 'type1' and as long
in type 'type2'. Terms facets on the field is failing. Following is the
gist:

curl -XDELETE 'http://localhost:9200/try_test/'

curl -XPOST http://localhost:9200/try_test -d '{
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
}'

curl -XPUT 'http://localhost:9200/try_test/type1/_mapping' -d '{
"type1": {
"_source": {
"enabled": true
},
"properties": {
"customId": {
"type": "integer",
"index": "analyzed"
},
"customValue": {
"type": "integer",
"index": "analyzed"
}
}
}
}'

curl -XPUT 'http://localhost:9200/try_test/type2/_mapping' -d '{
"type2": {
"_source": {
"enabled": true
},
"properties": {
"customId": {
"type": "integer",
"index": "analyzed"
},
"customValue": {
"type": "long",
"index": "analyzed"
}
}
}
}'

curl -XPOST 'http://localhost:9200/try_test/type1/' -d '{
"customId": 2301,
"customValue": 67
}'

curl -XPOST 'http://localhost:9200/try_test/type2/' -d '{
"customId": 2302,
"customValue": 678917349187249
}'

curl -XGET 'http://localhost:9200/try_test/_search?pretty=true' -d '{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Failing, which is expected. Trying to facet simultaneously on two

different types can be illegal.

curl -XGET 'http://localhost:9200/try_test/type1/_search?pretty=true' -d
'{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Failing, why? It should give result, as does the next one.

curl -XGET 'http://localhost:9200/try_test/type2/_search?pretty=true' -d
'{
"query": {
"match_all": {}
},
"from": 0,
"size": 0,
"facets": {
"facet1": {
"terms": {
"field": "customValue",
"size": 10
}
}
}
}'

Working.

The exception is the following :

{
"error" : "SearchPhaseExecutionException[Failed to execute phase
[query_fetch], total failure; shardFailures
{[h5W2XmgPTKS6pDylfkqBZQ][try_test][0]:
QueryPhaseExecutionException[[try_test][0]:
query[ConstantScore(NotDeleted(cache(_type:type1)))],from[0],size[0]: Query
Failed [Failed to execute main query]]; nested:
NumberFormatException[Invalid shift value in prefixCoded string (is encoded
value really an INT?)]; }]",
"status" : 500
}

Any hint on how to resolve this? Making everything as long is one
solution, I know, but other than that? This is failing on both 0.19 and
0.20 version.

Thanks in advance,
-- Sujoy.

--

3

On Fri, 2013-01-18 at 04:38 -0800, btiernay wrote:

I'm not sure this is relevant, but from
Elasticsearch Platform — Find real-time answers at scale | Elastic

    Field names with the same name across types are highly
    recommended to have the same type and same mapping
    characteristics (analysis settings for example). There is an
    effort to allow to explicitly “choose” which field to use by
    using type prefix (my_type.my_field), but it’s not complete,
    and there are places where it will never work (like faceting
    on the field).

Hmm, that is interesting... I hadn't seen that comment before. I wonder
if it is still true.

It certainly seems to work:

curl -XPOST 'http://127.0.0.1:9200/test/test?pretty=1' -d '
{
"bar" : {
"name" : 3
},
"name" : "foo"
}
'
curl -XGET
'http://127.0.0.1:9200/test/test/_search?pretty=1&search_type=count' -d
'
{
"facets" : {
"test.name" : {
"terms" : {
"field" : "test.name"
}
},
"test.bar.name" : {
"terms" : {
"field" : "test.bar.name"
}
},
"name" : {
"terms" : {
"field" : "name"
}
},
"bar.name" : {
"terms" : {
"field" : "bar.name"
}
}
}
}
'

{
"facets" : {
"test.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : "foo"
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"test.bar.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : 3
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"bar.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : 3
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : "foo"
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
}
}
}

--

4

Many thanks to both .....

Yes, I haven't noticed that comment too for mapping, but the problem can be
too annoying sometimes.

We have similar documents under different types, with explicit mapping (say
Integer). Somehow mapping for a type is not registered, then in absence of
explicit mapping default mapping (and that is usually long) comes in, and
that is where the problem starts.

Looks like more care has to be needed on mapping conflicts and failures.
Thanks again.

-- Sujoy.

On Friday, January 18, 2013 6:19:59 PM UTC+5:30, Clinton Gormley wrote:

On Fri, 2013-01-18 at 04:38 -0800, btiernay wrote:

I'm not sure this is relevant, but from
Elasticsearch Platform — Find real-time answers at scale | Elastic

    Field names with the same name across types are highly 
    recommended to have the same type and same mapping 
    characteristics (analysis settings for example). There is an 
    effort to allow to explicitly “choose” which field to use by 
    using type prefix (my_type.my_field), but it’s not complete, 
    and there are places where it will never work (like faceting 
    on the field).

Hmm, that is interesting... I hadn't seen that comment before. I wonder
if it is still true.

It certainly seems to work:

curl -XPOST 'http://127.0.0.1:9200/test/test?pretty=1' -d '
{
"bar" : {
"name" : 3
},
"name" : "foo"
}
'
curl -XGET
'http://127.0.0.1:9200/test/test/_search?pretty=1&search_type=count' -d
'
{
"facets" : {
"test.name" : {
"terms" : {
"field" : "test.name"
}
},
"test.bar.name" : {
"terms" : {
"field" : "test.bar.name"
}
},
"name" : {
"terms" : {
"field" : "name"
}
},
"bar.name" : {
"terms" : {
"field" : "bar.name"
}
}
}
}
'

{
"facets" : {
"test.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : "foo"
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"test.bar.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : 3
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"bar.name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : 3
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
},
"name" : {
"other" : 0,
"terms" : [
{
"count" : 1,
"term" : "foo"
}
],
"missing" : 0,
"_type" : "terms",
"total" : 1
}
}
}

--