# Old ElasticSearch Data & Upgrading Major Versions?

**URL:** <https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964>\
**Category:** Elasticsearch\
**Created:** [November 30, 2015, 10:18pm UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964 "2015-11-30T22:18:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ninjada](https://avatars.discourse-cdn.com/v4/letter/n/5daacb/32.png) [@ninjada](https://discuss.elastic.co/u/ninjada)\
**Post date:** [November 30, 2015, 10:18pm UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964/1 "2015-11-30T22:18:59Z")

</div>

Hello,

Approaching the second time I'm attempting a major update across my elasticsearch, logstash & kibana stacks.

Previously I did this around version 1.7x of elasticsearch & logstash 1.5 -- I had to jettison all previous data (not sure the running version, but earlier) to get elasticsearch to start.

While this wasn't a huge deal as data was backed up and I had states to redeploy the older elk stack, I am again attempting to move to latest versions with elasticsearch 2.x & logstash 2.x and kibana 4.30. When testing on dev with the older data, elasticsearch refuses to start.

> Starting elasticsearch: Exception in thread "main" java.lang.IllegalStateException: unable to upgrade the mappings for the index [logstash-2015.11.29], reason: [Mapper for [timestamp] conflicts with existing mapping in other types:  
> [mapper [timestamp] cannot be changed from type [date] to [string]]]  
> Likely root cause: java.lang.IllegalArgumentException: Mapper for [timestamp] conflicts with existing mapping in other types:  
> [mapper [timestamp] cannot be changed from type [date] to [string]]  
> at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)  
> at org.elasticsearch.index.mapper.MapperService.checkNewMappersCompatibility(MapperService.java:364)  
> at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:315)  
> at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:261)  
> at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility(MetaDataIndexUpgradeService.java:329)  
> at org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.java:112)  
> at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:228)  
> at org.elasticsearch.gateway.GatewayMetaState.(GatewayMetaState.java:87)  
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)  
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)  
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)  
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)  
> at \<\<\>\>  
> at org.elasticsearch.node.Node.(Node.java:202)  
> at org.elasticsearch.node.Node.(Node.java:129)  
> at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)  
> at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)  
> at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:285)  
> at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)  
> [FAILED]

Obviously some mapping issue, but I'm not sure how to resolve this. I'm assuming this is a logstash issue, the way its mapping data? Just wondering a) what causes this exactly & is there a way to resolve that and keep my old data? and b) if not, why and is it the fault of logstash or elasticsearch?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 30, 2015, 10:31pm UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964/2 "2015-11-30T22:31:26Z")

</div>

You should be running [https://github.com/elastic/elasticsearch-migration](https://github.com/elastic/elasticsearch-migration) before upgrading.

---

<div class="post-metadata">

**Author:** ![ninjada](https://avatars.discourse-cdn.com/v4/letter/n/5daacb/32.png) [@ninjada](https://discuss.elastic.co/u/ninjada)\
**Post date:** [December 1, 2015, 3:27am UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964/3 "2015-12-01T03:27:35Z")

</div>

I've run the elasticsearch-migration tool and received the following issues on every index:

> Conflicting field mappings  
> Mapping for field nginx-access:timestamp conflicts with: nginx-error:timestamp. Check parameters: format, norms.enabled, type  
> Mapping for field nginx-access:timestamp conflicts with: nginx-error:timestamp. Check parameters: format, norms.enabled, type  
> Fields with dots  
> Dots in field names lead to ambiguous field resolution, in fields: logs:metrics.http.%{response}.count, logs:metrics.http.%{response}.rate\_15m, logs:metrics.http.%{response}.rate\_1m, logs:metrics.http.%{response}.rate\_5m, logs:metrics.http.200.count,

Have resolved the issue with dots in field names for metrics. However, I'm stuck on the timestamp conflicts.

What is the correct way to setup a logstash filter with respect to timestamping? Here is my basic config:

> filter {  
> if [type] == "nginx-access" {  
> grok {  
> match =\> { "message" =\> "%{NGINXACCESS}" }  
> }

> ```
> date {
> match => ["timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" , "YYYY-MM-dd HH:mm:ss,SSS" , "dd-MM-YYYY HH:mm:ss"]
> }
> 
> ```

> ```
> geoip {
> source => "clientip"
> }
> 
> ```

> ```
> if [agent] != "-" and [agent] != "" {
> useragent {
> source => "agent"
> target => "ua"
> }
> }
> 
> ```

> ```
> metrics {
> meter => ["http_%{response}"]
> add_tag => "metric"
> flush_interval => "60"
> }
> }
> 
> ```
> 
> }  
> filter {  
> if [type] == "nginx-error" {  
> grok {  
> match =\> { "message" =\> "%{NGINXERROR}" }  
> }

> ```
> date {
> match => ["timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" , "YYYY-MM-dd HH:mm:ss,SSS" , "dd-MM-YYYY HH:mm:ss"]
> }
> 
> ```

> ```
> geoip {
> source => "clientip"
> }
> 
> ```

> ```
> if [agent] != "-" and [agent] != "" {
> useragent {
> source => "agent"
> target => "ua"
> }
> }
> 
> ```

> ```
> metrics {
> meter => ["http_%{response}"]
> add_tag => "metric"
> flush_interval => "60"
> }
> }
> 
> ```

Seems i have a date entry per filter, is this whats leading to the conflict?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 1, 2015, 4:43am UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964/4 "2015-12-01T04:43:38Z")

</div>

And what's the mappings for those two fields? Do they have the same type?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:34pm UTC](https://discuss.elastic.co/t/old-elasticsearch-data-upgrading-major-versions/35964/5 "2017-07-05T23:34:37Z")

</div>