Upgrading ES 2.x to 5.x - Questions about elasticsearch-migration output

[Roland_L]

Hi,

I'm preparing our migration from ES 2.4.1 to 5.0 and I have a few questions about the results the elasticsearch-migration plugin has given me:

• node.datacenter_location should be rewritten as node.attr.datacenter_location: we have been using node.datacenter_location in cluster.routing.allocation.awareness.attributes: datacenter_location: so if the first one will be changed what do I change the second one to?
• foreground will be moved to the archived namespace on upgrade: what is this parameter? We don't have it in out elasticsearch.yml and I've never seen it before.
• indices.recovery.concurrent_streams will be moved to the archived namespace on upgrade: is there an alternative to this property? It's documented up until 2.x, but it's missing in the 5.0 documentation (https://www.elastic.co/guide/en/elasticsearch/reference/5.0/recovery.html). We have 10 nodes (5 per DC) and if the default is still 3 streams it's very low. It would be great if we could raise the number of concurrent streams, our hardware can handle more streams.

Thanks,
Roland

[tanguy]

Hi Roland,

@Roland_L

node.datacenter_location should be rewritten as node.attr.datacenter_location: we have been using node.datacenter_location in cluster.routing.allocation.awareness.attributes: datacenter_location: so if the first one will be changed what do I change the second one to?

In 5.0 node attributes must have the prefix node.attr. so you need to change node.datacenter_location to node.attr.datacenter_locationin the configuration file.

But node.datacenter_location in cluster.routing.allocation.awareness.attributes: datacenter_location indicates the name of the node attribute to use for shard allocation. In your case it is unchanged.

See Shard Allocation Awareness | Elasticsearch Guide [5.0] | Elastic as an example on 5.0.

@Roland_L

foreground will be moved to the archived namespace on upgrade: what is this parameter? We don't have it in out elasticsearch.yml and I've never seen it before.

This means that the migration plugin has found a parameter foreground that is unknown and will be moved under the archived. namespace in cluster settings after the node starts up. This parameter is unknown to me so I think it's a leftover in your configuration. From Elasticsearch 5.0 on all settings are validated before they are applied.

@Roland_L

indices.recovery.concurrent_streams will be moved to the archived namespace on upgrade:

This settings has been completely removed, it does not exist anymore. Recoveries are now controlled (and throttled) on each node using an allocation decider. You might want to tune the settings as documented in Cluster Level Shard Allocation | Elasticsearch Guide [5.0] | Elastic.

[Roland_L]

Hi,

thank you for the answers! It helped me a lot!

@tanguy

it's a leftover in your configuration

Well, that's my exact problem. I don't find the parameter anywhere. I've looked at all the files in /etc/elasticsearch, I've looked at /_cluster/state, /_cluster/status, /_cluster/health - no foreground anywhere...

Edit: well, I've found "foreground": it's under /_nodes/settings for each node. Still no idea where it comes from, because we definitely don't set this. Maybe a default value from 2.x? (I also have a 1.7 cluster running, that one doesn't have this property.)