Need help upgrading XWiki's ElasticSearch instance

Elastic Stack Elasticsearch
1

Hi guys,

XWiki (http://xwiki.org) uses ElasticSearch. We're still using an old version of ES: v1.1.1. I've tried in the past to upgrade but it failed. I'm now trying again and this time I want to spend the time it takes to perform this upgrade to conclusion.

By reading the docs I understood that I need to upgrade to 2.x first, before upgrading to 5.x. So I installed 2.4.4 and copied the config/ and data/ dirs. When I restarted ES, I get:

Caused by: java.lang.IllegalArgumentException: Mapper for [_timestamp] conflicts with existing mapping in other types:
[mapper [_timestamp] has different [store] values]

I found the reason by reading Upgrade from 1.7.x to 2.2.0 _timestamp mapping issue

Indeed I have 2 types (installs/install and installs/install2) and they're not using _timestamp in a similar way I guess. We probably have the timestamp stored in one type and not in the other.

Now I read that I need to reindex and AFAIU this means moving existing data under a new type, right?

I need some guidance about what to do, provided the following:

  • There are several instances of XWiki out there, some using the installs/install type and others using the installs/install2 type. Every day these instances send a ping that gets stored in ES in one of those types.
  • Thus we need to continue supporting these types to not break the XWiki instances and still collect the ping data.
  • We also have substantial code in various places that queries the XWiki ES instance and thus using the existing types, and thus we would rather not break those codes.
  • We can introduce a new type that would be used by new versions of XWiki but globally we need to be able to sum up all the pings from all the types.

What are our options?

Thanks a lot for any guidance
-Vincent
XWiki committer

2

Is the only option to keep the XWiki ES 1.1.1 instance and install a new ES 5.x instance with a new index and then modify new XWiki versions to use that new instance+index.

Then modify our querying codes to query both the old instance + new one?

It's not very easy to do. Do you see an alternative?

Thx

3

There is no way to keep multiple mappings for the same field in different types.

Have you thought about going to Elasticsearch 5.x directly? You can do a remote reindex from older versions and only need to do one migration.
Related to that: timestamp has been removed in 5.x and there is only date. If you do a reindex, could you define both possible formats and have them parsed to milliseconds-since-the-epoch ? https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html#multiple-date-formats

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.