Upgrading ES 2.x to 5.x - Questions about elasticsearch-migration output


(Roland L) #1

Hi,

I'm preparing our migration from ES 2.4.1 to 5.0 and I have a few questions about the results the elasticsearch-migration plugin has given me:

  • node.datacenter_location should be rewritten as node.attr.datacenter_location: we have been using node.datacenter_location in cluster.routing.allocation.awareness.attributes: datacenter_location: so if the first one will be changed what do I change the second one to?
  • foreground will be moved to the archived namespace on upgrade: what is this parameter? We don't have it in out elasticsearch.yml and I've never seen it before.
  • indices.recovery.concurrent_streams will be moved to the archived namespace on upgrade: is there an alternative to this property? It's documented up until 2.x, but it's missing in the 5.0 documentation (https://www.elastic.co/guide/en/elasticsearch/reference/5.0/recovery.html). We have 10 nodes (5 per DC) and if the default is still 3 streams it's very low. It would be great if we could raise the number of concurrent streams, our hardware can handle more streams.

Thanks,
Roland


(Tanguy) #2

Hi Roland,

In 5.0 node attributes must have the prefix node.attr. so you need to change node.datacenter_location to node.attr.datacenter_locationin the configuration file.

But node.datacenter_location in cluster.routing.allocation.awareness.attributes: datacenter_location indicates the name of the node attribute to use for shard allocation. In your case it is unchanged.

See https://www.elastic.co/guide/en/elasticsearch/reference/5.0/allocation-awareness.html#allocation-awareness as an example on 5.0.

This means that the migration plugin has found a parameter foreground that is unknown and will be moved under the archived. namespace in cluster settings after the node starts up. This parameter is unknown to me so I think it's a leftover in your configuration. From Elasticsearch 5.0 on all settings are validated before they are applied.

This settings has been completely removed, it does not exist anymore. Recoveries are now controlled (and throttled) on each node using an allocation decider. You might want to tune the settings as documented in https://www.elastic.co/guide/en/elasticsearch/reference/5.0/shards-allocation.html#_shard_allocation_settings.


(Roland L) #3

Hi,

thank you for the answers! It helped me a lot!

Well, that's my exact problem. :slight_smile: I don't find the parameter anywhere. I've looked at all the files in /etc/elasticsearch, I've looked at /_cluster/state, /_cluster/status, /_cluster/health - no foreground anywhere...

Edit: well, I've found "foreground": it's under /_nodes/settings for each node. Still no idea where it comes from, because we definitely don't set this. Maybe a default value from 2.x? (I also have a 1.7 cluster running, that one doesn't have this property.)


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.