Mapping namespace problem

Elastic Stack Elasticsearch
1

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

    Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping' -d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping' -d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1' -d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2' -d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true' -d
'{
"fields": "",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone

2

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Check Elasticsearch Platform — Find real-time answers at scale | Elastic
On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping' -d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping' -d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1' -d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2' -d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest/*/_search?pretty=true' -d
'{
"fields": "*",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone

3

Sorry I didn't read the help page.

I will do as you suggest.

Thanks

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest/*/_search?pretty=true'-d
'{
"fields": "*",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone

4

This is the gist with code for my test

https://gist.github.com/829239

Is this behaviour correct? If so, what's the correct way to use
mappings? Aren't they supposed to create different namespaces for
fields of documents into a index?

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest/*/_search?pretty=true'-d
'{
"fields": "*",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone

5

I see the problem now, basically, the types map1 and map2 have different mappings for field1, so unless explicitly specified in the query (map1.field1 or map2.field1 compared to just field1) will cause problems. Usually, it is better to have same name fields cross mapping types to be of hte same type.
On Wednesday, February 16, 2011 at 1:49 PM, Simone Zaninello wrote:

This is the gist with code for my test

https://gist.github.com/829239

Is this behaviour correct? If so, what's the correct way to use
mappings? Aren't they supposed to create different namespaces for
fields of documents into a index?

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest/*/_search?pretty=true'-d
'{
"fields": "*",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone