Mapping namespace problem


(Simone Zaninello) #1

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

    Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping' -d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping' -d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1' -d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2' -d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true' -d
'{
"fields": "
",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone


(Shay Banon) #2

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Check http://www.elasticsearch.org/help
On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping' -d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping' -d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1' -d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2' -d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true' -d
'{
"fields": "
",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone


(Simone Zaninello) #3

Sorry I didn't read the help page.

I will do as you suggest.

Thanks

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true'-d
'{
"fields": "
",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone


(Simone Zaninello) #4

This is the gist with code for my test

https://gist.github.com/829239

Is this behaviour correct? If so, what's the correct way to use
mappings? Aren't they supposed to create different namespaces for
fields of documents into a index?

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true'-d
'{
"fields": "
",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone


(Shay Banon) #5

I see the problem now, basically, the types map1 and map2 have different mappings for field1, so unless explicitly specified in the query (map1.field1 or map2.field1 compared to just field1) will cause problems. Usually, it is better to have same name fields cross mapping types to be of hte same type.
On Wednesday, February 16, 2011 at 1:49 PM, Simone Zaninello wrote:

This is the gist with code for my test

https://gist.github.com/829239

Is this behaviour correct? If so, what's the correct way to use
mappings? Aren't they supposed to create different namespaces for
fields of documents into a index?

On 16 Feb, 02:57, Shay Banon shay.ba...@elasticsearch.com wrote:

Hard to tell since you embed all in the mail... . Can you gist it (or better yet, create a curl sample of it)? Checkhttp://www.elasticsearch.org/help

On Monday, February 14, 2011 at 7:39 PM, Simone Zaninello wrote:

Hi,

I'm having the following problem:

  • Initial state
    I put two different mappings with a field named the same way into a
    single index.
    After that I indexed two documents one for each mapping.

Consider this as an example

Mappings

curl -XPUT 'http://localhost:9200/localtest/map1/_mapping'-d '{
"map1": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "date",
"store": "yes",
"format": "yyyyMMddHHmmssSSS",
"include_in_all": false
}
}
}
}'

curl -XPUT 'http://localhost:9200/localtest/map2/_mapping'-d '{
"map2": {
"_source" : {
"enabled" : false
},
"date_formats": ["date_time", "yyyyMMddHHmmss",
"yyyyMMddHHmmssSSS"],
"_all": {
"enabled": true,
"store": "yes"
},
"properties": {
"field1": {
"type": "string",
"store": "yes",
"term_vector": "yes",
"include_in_all": false
}
}
}
}'

documents

curl -XPUT 'http://localhost:9200/localtest/map1/1'-d '{
"field1" : "20110214172449000"
}'

curl -XPUT 'http://localhost:9200/localtest/map2/2'-d '{
"field1" : "Test map2 with string type field"
}'

If I execute a search over this index using the field1 seen as string,
the query breaks. But if I execute a search over this index using the
field1 seen as date, the query succeeds.

Query example:
curl -XGET 'http://localhost:9200/localtest//_search?pretty=true'-d
'{
"fields": "
",
"from" : 0, "size" : 100,
"query":{
"bool":{
"must":{"field":{"field1":""Test""}}
}
}
}'

Returned error:

MapperParsingException[failed to parse date field, tried both date
format [yyyyMMddHHmmssSSS], and timestamp number]; nested:
IllegalArgumentException[Invalid format: "Test"];

It seems that the first mapping is the one that determines the field
type, disregarding any following mappings.

Is it correct? If so, what's the correct way to use mappings?

Best regards
Simone


(system) #6