Migration Plugin - How to change dynamic setting?

Elastic Stack Elasticsearch
1

Hello,

as we currently planning our upgrade from 2.4 to 5.x, we run into some kind of problem regarding our old indices.

Index settings
Replaced settings
index.translog.flush_threshold_ops has been replaced by index.translog.flush_threshold_size

This message showed up in the migration plugin.

Looking at the index itself we have both settings set:

"logstash-2016.11.29" : {
    "settings" : {
      "index" : {
        "refresh_interval" : "30s",
        "translog" : {
          "flush_threshold_ops" : "60000",
          "flush_threshold_size" : "1gb",
          "durability" : "async"
        },

So i am curious now of how to "unset" the "threshold_ops"-settings in order to allow a upgrade to 5.x index?

I tried setting it to "0" or "NULL" - the first one did not change anything, the 2nd did not work.
Since it is a dynamic setting, i dont think reindexing would be required.

Any feedback is appreciated.

Regards

2

Don't post pictures please but formatted text. Hardly readable.

3

thanks for the hint - updated the post

4

You can update to null an index settings in 5.x series.
In 2.x you have to set it to the default values but I'm unsure if this will solve the problem you are facing.

Can you try with a non production cluster? Create in 2.x an index with that setting and then update to 5.x without changing that setting and see what are the error messages if any.

5

Thanks for your response.

I am afraid i cant set it to the default.

Regarding the 2.X documentation the default would be "unlimited" - the update API only allow integer values tho.

I am gonna try it out with an test cluster and let you know the results.

6

The default value is actually 2147483647 (Max INT). But unsure if this helps.

7

I tried setting it to the max Int value, no changes.

I set up an 2.4.5 cluster and created the index with the setting, migration plugin showed an error, as expected.

I than upgraded the ES on the server to 5.4.0 and after restart the following logs showed up:

[2017-05-11T11:10:44,358][INFO ][o.e.c.u.IndexFolderUpgrader] [logstash-2016.11.29/1SJ-qAI4SiugEzLx-rfW_Q] upgrading [/es_index/cluster/nodes/0/indices/logstash-2016.11.29] to new naming convention
[2017-05-11T11:10:44,359][INFO ][o.e.c.u.IndexFolderUpgrader] [logstash-2016.11.29/1SJ-qAI4SiugEzLx-rfW_Q] moved from [/es_index/cluster/nodes/0/indices/logstash-2016.11.29] to [/es_index/cluster/nodes/0/indices/1SJ-qAI4SiugEzLx-rfW_Q]
[2017-05-11T11:10:44,465][WARN ][o.e.c.m.MetaDataIndexUpgradeService] [servername] [logstash-2016.11.29/1SJ-qAI4SiugEzLx-rfW_Q] ignoring unknown index setting: [index.translog.flush_threshold_ops] with value [21483647]; archiving

ES did upgrade the index without errors and it still is searchable.

I guess that "solves" my problem here - one can ignore these notifications of the migration plugin.

Thanks for your help here.

8

I believe you need to set the new setting thought ?

9

Sorry, i dont understand what you mean actually.

The "index.translog.flush_threshold_ops" was set before the upgrade to 5.x - the upgrade went fine even tho the migration plugin displayed it otherwise.

After the index was upgraded to 5.x it is no longer possible to set this setting, this message appears:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown setting [index.translog.flush_threshold_ops] did you mean [index.translog.flush_threshold_size]?"}],"type":"illegal_argument_exception","reason":"unknown setting [index.translog.flush_threshold_ops] did you mean [index.translog.flush_threshold_size]?"},

10

Sorry if I was unclear.

I meant that you need to set index.translog.flush_threshold_size if you want to have the same behavior you had before.

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.