Curator Restore Extra Settings

In the most recent curator documentation, I see that you can now add extra settings for the restore action (https://www.elastic.co/guide/en/elasticsearch/client/curator/current/restore.html#_extra_settings_2).

In the example, it used the following:

extra_settings:
  settings:
    number_of_shards: 1
    number_of_replicas: 0

My understanding is that this would restore the indices and make all indices have 1 primary shard and 0 replicas. However, when I tried to use these settings to restore a test index, it had no impact on either the # of primary shards or the # of replica shards (e.g. the original index had 5 primary and 1 replica and the restored index had the same amount for both).

Am I misunderstanding how this extra setting is supposed to work with regards to the Curator restore action?

Thank you,
-LM

There are a few possibilities that come immediately to mind.

  1. You had an index template with different default settings for those indices.
  2. You had include_global_state set to True, which would preserve the original settings due to the full cluster state being restored with the index.

For any other troubleshooting to be done, this would require running Curator with DEBUG level logging, the logging blacklist set to [], and increased logging on the Elasticsearch side as well.

Hi @theuntergeek,

Thanks for the quick reply. I will investigate further and see if I can figure out what might be happening.

-LM

A few quick updates on my end. I tried this with our current version of 5.2.1 and then upgraded to 5.4.3 as well. Everything described below applies to both versions.

To start, I made sure to remove all of our templates and started with no data:

curl localhost:9200/_template/_all?pretty 
{ }

I then added the following index into the cluster:

curl -XPUT localhost:9200/twitter/tweet/1 -d '{"msg": "Testing curator!"}'

The _cat/indices endpoint gave the following results:

health status index   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   twitter JDSWq6HlRimaSWqHyiGdMg   5   1          1            0      7.6kb          3.8kb

I then created a snapshot using the following action file:

---
actions:
  1:
    action: snapshot
    options:
      repository: 'repo'
      name: 'twitter_snapshot'
      ignore_empty_list: True
      continue_if_exception: True
      disable_action: False
      include_global_state: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: twitter

I then deleted the index and restored the index using this action file:

---
  actions:
    1:
      action: restore
      options:
        repository: repo
        name: twitter_snapshot
        include_global_state: False
        extra_settings:
          settings:
            number_of_shards: 1
            number_of_replicas: 0
      filters:
      - filtertype: state
        state: SUCCESS 

When I curled for indices I got the following results:

  curl localhost:9200/_cat/indices?v
  health status index   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
  green  open   twitter pLRSOENJQzirAMZM5rkadA   5   1          1            0      7.8kb          3.9kb

Which suggests that it did not work. Interestingly, when I used the regular restore curl command, it did work:

curl -XPOST localhost:9200/_snapshot/repo/twitter_snapshot/_restore -d '{ "indices": "twitter","index_settings": {"index.number_of_replicas": 0},"include_global_state": false}'
curl localhost:9200/_cat/indices?v
health status index   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   twitter VSJCAHEgSGKBT2LTGGKhcA   5   0          1            0      3.9kb          3.9kb

Since I had no templates and made sure include_global_state was set to False, perhaps something else may be at play? I cranked up logging for both Curator and Elasticsearch, but didn't know if I should just put all the logs directly here. Is there anything in particular I should be looking for or should I just post everything I had here?

-LM

A quick addendum: one reason I was felt the need to try the example out was because it appeared that Curator was offering a capability that is not present in the restore document:

Please note, that some settings such as index.number_of_shards cannot be changed during restore operation.

from: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html#_changing_index_settings_during_restore

Interesting. I must have missed a change, as it was (theoretically) possible at some point in the past. I will change the docs to reflect this reality.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.