Cannot open data from 5.x in 6.x : Mappings

Elastic Stack Elasticsearch
#1

I've created a routine that updates ES clients from 5.x to 6.x and finally 7.x
Somehow some clients cannot be updated.
Loading existing data in 6.8 fails.
Appearently some mappings are causing this.
But there are not templates applied and I cannot see any difference to the other clients, were everything works just fine.
Do I have to apply some templates to the 5.x version and retry or what?

Caused by: org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [datapoint]: No handler for type [string] declared on field [batchId]

java.lang.IllegalStateException: unable to upgrade the mappings for the index [[eve/xGvLV_oqSXeT3BwXNQEeyQ]]
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:205) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:103) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.gateway.GatewayMetaState.upgradeMetaData(GatewayMetaState.java:246) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:89) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.node.Node.<init>(Node.java:499) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.node.Node.<init>(Node.java:266) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.8.jar:6.8.8]
	at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.8.jar:6.8.8]
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [datapoint]: No handler for type [string] declared on field [batchId]
	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:399) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:355) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:334) ~[elasticsearch-6.8.8.jar:6.8.8]
	at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:201) ~[elasticsearch-6.8.8.jar:6.8.8] ```
#2

Sounds like you are trying to define a field datapoint as a string which is not supported anymore and must be a text or a keyword.

#3

hm actually /_mappings states:
"batchId":{"type":"keyword"},
the term 'string' does not exist in my mapping response...

#4

The field name is datapoint not batchId.

I'm not sure about what you are doing but may be you are trying to call the update mapping API?

unable to upgrade the mappings for the index eve

#5

Really? That cannot be true: datapoint is the entity not a field. Also the error states:

No handler for type [string] declared on field [batchId]

I am just starting the elasticsearch.bat file in 6.8.8 in order to reindex data

grafik

#6

Ha right. Sorry for the confusion.

So everything is correct in term of existing mapping.

But something is calling the Update Mapping API with a wrong schema.

#7

It seems so but the only thing I do is executing elasticsearch.bat

#8

It seems so but the only thing I do is executing elasticsearch.bat

Probably you need to explain what you exactly did:

When upgrading to a major version, you must go through all the steps which are documented at

I'd recommend also trying

Once you'll have finish to 6.8 upgrade, I'll recommend you using Kibana and the upgrade assistant to prepare your cluster to 7.9.

May be an easier path to this is to reindex the data in the 7.9 cluster "directly".

#9

Well the procedure is like suggested:

6.x should be able to load 5.x data. And I can tellt that this seems to be correct in general because that's what I always do.
The thing is that in this particular case, 6.x simply cannot load the data.

#10

Could you share what is the exact output of

GET /

When you are running the cluster in version 5?

#11

{
"name" : "U9QKykR",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "G0uiXeQURmqXJwy5OB_agQ",
"version" : {
"number" : "5.2.2",
"build_hash" : "f9d9b74",
"build_date" : "2017-02-24T17:26:45.835Z",
"build_snapshot" : false,
"lucene_version" : "6.4.1"
},
"tagline" : "You Know, for Search"
}

#12

I just came across another thing regarding forbidden characters in field names which appearently can also lead to this exception.

I found out that there is a difference in the actual data structure of the affected client.
It has additional parameters, that should not exist anymore.

"batchName" : "ECB2018 (3)",
"parameterName" : "Foam",
"phaseName" : "",
"deviceName" : "Minifors 2",
"deviceType" : "Minifors2"

Could this be somehow an issue?

#13

Okay problem solved.
There was an index that wasn't supposed to be there. After deleting it, ES 6.8 starts up as expected.