Upgrade from 1.7.x to 2.2.0 _timestamp mapping issue

Elastic Stack Elasticsearch
1

Hey,

I used the migration plugin and it said I can migrate without re-indexing, but I'm getting this error :

java.lang.IllegalStateException: unable to upgrade the mappings for the index [mi_2015_07_w4], reason: [Mapper for [_timestamp] conflicts with existing mapping in other types:
 [mapper [_timestamp] has different [store] values, mapper [_timestamp] has different [doc_values] values, cannot change from disabled to enabled]]
 at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:339)
 at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:116)
 at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:228)
 at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:87)

I don't understand what it means exactly and whether I have to re-index everything (2B records). This says that the _timefield is just deprecated :

Could pls anybody help me with this?

2

It means you have _timestamp configured differently in two different types. In one type, you have timestamp stored, in the other it is not stored. Unfortunately this will require reindexing: the index contains a mix of segments with these conflicting settings.

I opened an issue to correctly mark this as requiring reindexing in the migration plugin:

3

Hey, perfect, just one question. I have 74 indices and when I started 2.2.0 there were these errors logged, but just for a subset of the indices, like 15 of them. Do you think that I need to reindex just those? Can I rely on it?

Regarding the error, you are talking about "Document types" within index having _timestamp field configured differently, right? In my case :

 "manifest": {
    "_timestamp": {
       "enabled": true
    }
   "record": { 
    "_timestamp": {
       "enabled": true,
       "doc_values": true
    }
}

I guess that thte problem is the additional doc_values in the second mapping...

I was just thinking, how about if I delete one of the types?

https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-delete-mapping.html

That would fix the binary incompatibility, right?

4

when I started 2.2.0 there were these errors logged, but just for a subset of the indices, like 15 of them. Do you think that I need to reindex just those?

Yes, only the indexes that have errors (which should not have been loaded) would need to be reindexed.

I was just thinking, how about if I delete one of the types?

Deleting types is no longer possible in 2.0+. While it might work in this case, it depends on a lot of things (because all documents using that type need to be completely purged from the lucene segments), I would still recommend re-indexing.