Elasticsearch 2 mapping


(mramaprasad) #1

Here is the mapping for acronym field and am running elsaticsearch 1.1.1.
"isbnfield": {
"type":"multi_field",
"fields":{
"isbn":{"type":"string","index":"analyzed", "store" : true},
"isbnuntouched":{"type":"string","index":"not_analyzed", "store" : true}
}
},
"acronymfield": {
"type":"multi_field",
"fields":{
"acronym":{"type":"string","index":"analyzed", "store" : true},
"acronymuntouched":{"type":"string","index":"not_analyzed", "store" : true}
}
},
"abbrev": {
"type": "string"
}

I am trying to update o elaseticsearch 2.0 and I am getting the following error message for acronym field. What is the problem?

java.lang.IllegalStateException: unable to upgrade the mappings for the index [csv1], reason: [Field name [acronymfield.acronym] cannot contain '.']
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:335)
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:112)
at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:226)
at org.elasticsearch.gateway.GatewayMetaState.(GatewayMetaState.java:85)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)


(David Pilato) #2

I believe it's because "type: multifield" has been deprecated for a long time and is not supported anymore.

Are you upgrading from a very old version?

Did you try the migration plugin before upgrading? https://github.com/elastic/elasticsearch-migration

May you could open an issue so we can at the very least fix the documentation here: https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_mapping_changes.html or try to find a way to read your mapping if doable.

For now, I'm afraid you have to create a new mapping and reindex.


(Mark Walkom) #3

Yes, because of https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_mapping_changes.html#_field_names_may_not_contain_dots


(David Pilato) #4

The problem here is that he is not using dot explicitly but the multifield type which is generating this dot.


(Mark Walkom) #5

Sorry, missed your reply somehow! :smiley:


(mramaprasad) #6

Thanks for the responses.

I am upgrading from version 1.1.1 to 2.0.

When I created a mapping in 2.0 with the multi field it did not complain.

But only has problems if I had data and trying to upgrade the data.


(mramaprasad) #7

How would the multi field be indexed in 2.0 version?

"acronymfield": {
"type":"multi_field",
"fields":{
"acronym":{"type":"string","index":"analyzed", "store" : true},
"acronymuntouched":{"type":"string","index":"not_analyzed", "store" : true}
}


(David Pilato) #8

Read https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html


(mramaprasad) #9

I have updated the mapping to have in elasticsearch 1.1.1 and indexed the data.

           	"acronymfield" : {
            	"type" : "string",
            	"fields": {
            		"raw" : {
            			"type": "string",
            	 		"index": "not_analyzed"
            	 	 }
             	}	          
		    },	            

I was trying to upgrade to version 2.0 by starting elasticsearch2.0.

I still get the following error:

Exception in thread "main" java.lang.IllegalStateException: unable to upgrade the mappings for the index [csv3], reason: [Field name [acronymfield.raw] cannot contain '.']
Likely root cause: MapperParsingException[Field name [acronymfield.raw] cannot contain '.']
at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:283)
at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:228)
at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:137)
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:211)
at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:192)
at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:368)
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:242)
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:329)
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:112)
at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:226)
at org.elasticsearch.gateway.GatewayMetaState.(GatewayMetaState.java:85)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at <<>>
at org.elasticsearch.node.Node.(Node.java:198)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

Any ideas please? Is it complaining about the "." in the data that is being indexed?


(Ivan Brusic) #10

This appears to be an expectation bug in Elasticsearch. Since 2.0, dots are
no longer valid inside field names. However, the use of "fields" in the
mapping will generate a field name with a dot. Elasticsearch needs to
remove multi fields since they no longer allow dots in name. Even the
example of multi fields still has a no longer relevant field name:

https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html

Try switching to copy_to

https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html

Ivan


(Ivan Brusic) #11

I do not have an Elasticsearch instance in front of me, so I cannot test,
but I would be surprised if this truly is the bug. Judging by the test
code, field names with dots are being generated.

Strange.

Ivan


(mramaprasad) #12

Objects fields in elasticsearch will have to be accessed by dot right? Are they valid in 2.0? We would have to use source.name to access the name field right?

"source" : {
"name" : "book",
"edition" : "1",
"numberOfPages" : "240"
}


(David Pilato) #13

I agree with @Ivan. It looks like a bug.

So @mramaprasad if you could provide a full simple script with only foo,bar,baz things to reproduce the issue that would be super useful.

And yes, accessing to source.name should work. source.name.raw also...


(Ivan Brusic) #14

It appears that the check for invalid names in the mapper service occurs
only on the field name, so the old unit tests for multi field still work. I
still do not have the code in front, looking at it only via Github, but I
cannot find where the new name gets generated. Interesting problem.

Ivan


(system) #15