# Curator Snapshot Error with Closed indices **URL:** \ **Category:** Elasticsearch\ **Created:** [June 1, 2018, 4:43pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259 "2018-06-01T16:43:34Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![wanshishi](https://avatars.discourse-cdn.com/v4/letter/w/fbc32d/32.png) [@wanshishi](https://discuss.elastic.co/u/wanshishi)\ **Post date:** [June 1, 2018, 4:43pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/1 "2018-06-01T16:43:35Z")
Snapshot is having an error with close indices. I have this curator script that runs multiple actions. I am not sure if I am doing it correctly or the order of the action affects how it ran. 1. allocation 2. forcemerge 3. close 4. snapshot 5. delete What happens is when the cron for curator runs, it will perform action 1,2 and 3, then when starting snapshot, I am having this error. `ERROR Failed to complete action: snapshot. : Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: TransportError(403, u'cluster_block_exception', u'blocked by: [FORBIDDEN/4/index closed];')` To go around this error, i tried to re-open the indices, then ran snapshot afterwards and it went fine. Now when my script re-run again, it will fail, because it will close the indices again. Is there a way for snapshot action to ignore close indices? Or is there a better solution? > actions: > 1: > action: allocation > description: "Apply shard allocation filtering rules to the specified indices. Move files older than 2 days" > options: > key: node\_type > value: warm > allocation\_type: require > wait\_for\_completion: true > timeout\_override: > ignore\_empty\_list: true > continue\_if\_exception: false > disable\_action: false > filters: > > - filtertype: pattern > kind: prefix > value: logstash- > - filtertype: age > source: name > direction: older > timestring: '%Y.%m.%d' > unit: days > unit\_count: 2 > > 2: > action: forcemerge > description: \>- > forceMerge logstash- prefixed indices older than 1 day (based on index > creation\_date) to 1 segments per shard. Delay 120 seconds between each > forceMerge operation to allow the cluster to quiesce. > This action will ignore indices already forceMerged to the same or fewer > number of segments per shard, so the 'forcemerged' filter is unneeded. > options: > max\_num\_segments: 1 > delay: 120 > timeout\_override: > ignore\_empty\_list: true > continue\_if\_exception: false > disable\_action: False > filters: > > - filtertype: pattern > kind: prefix > value: logstash- > exclude: > - filtertype: age > source: creation\_date > direction: older > unit: days > unit\_count: 1 > exclude: > > 3: > action: close > description: \>- > Close indices older than 90 days (based on index name), for logstash- and syslog- prefixed indices. > options: > delete\_aliases: False > timeout\_override: > ignore\_empty\_list: true > continue\_if\_exception: false > disable\_action: False > filters: > > - filtertype: pattern > kind: prefix > value: logstash- > exclude: > - filtertype: age > source: name > direction: older > timestring: '%Y.%m.%d' > unit: days > unit\_count: 90 > exclude: > > 4: > action: snapshot > description: "Snapshot logstash- older than 91 day based on index creation\_date.Wait for snapshot to complete." > options: > repository: es\_backups > name: logstash-%Y%m%d%H%M%S > ignore\_unavailable: false > include\_global\_state: True > partial: False > wait\_for\_completion: True > skip\_repo\_fs\_check: False > disable\_action: False > filters: > > - filtertype: pattern > kind: prefix > value: logstash- > - filtertype: age > source: name > timestring: '%Y.%m.%d' > direction: older > unit: days > unit\_count: 91 > > 5: > action: delete\_indices > description: \>- > Delete indices older than 91 days (based on index name), logstash- and syslog- > prefixed indices. Ignore the error if the filter does not result in an > actionable list of indices (ignore\_empty\_list) and exit cleanly. > options: > ignore\_empty\_list: True > timeout\_override: > continue\_if\_exception: False > disable\_action: False > filters: > > - filtertype: pattern > kind: prefix > value: logstash- > exclude: > - filtertype: age > source: name > direction: older > timestring: '%Y.%m.%d' > unit: days > unit\_count: 91 > exclude: Specifications Version: ES Version 6.0.0 Platform: Ubuntu Curator 5.5 ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [June 1, 2018, 5:06pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/2 "2018-06-01T17:06:33Z")
Elasticsearch cannot snapshot a closed index, which the error message you received makes plain. You have two options. Which you choose will depend on the desired outcome. 1. Re-order your actions so `snapshot` comes before `close`. Actions are performed in the order in which they appear. 2. Add a [`closed` filter](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/filtertype_closed.html) to your `snapshot` filter list. Option 1 guarantees that the indices you just forcemerged get snapshotted. This is what I presume you wanted to have happen. Option 2 will simply tell the `snapshot` action to ignore closed indices. If you _want_ the closed indices to be snapshotted, this is perhaps not the option you want to go with. You should simply re-order the actions to close the indices after they've been snapshotted. ---
**Author:** ![wanshishi](https://avatars.discourse-cdn.com/v4/letter/w/fbc32d/32.png) [@wanshishi](https://discuss.elastic.co/u/wanshishi)\ **Post date:** [June 1, 2018, 5:35pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/3 "2018-06-01T17:35:18Z")
> [@theuntergeek](#): > > Elasticsearch cannot snapshot a closed index, which the error message you received makes plain. > > You have two options. Which you choose will depend on the desired outcome. > > 1. Re-order your actions so `snapshot` comes before `close`. Actions are performed in the order in which they appear. > 2. Add a [`closed` filter](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/filtertype_closed.html) to your `snapshot` filter list. > > Option 1 guarantees that the indices you just forcemerged get snapshotted. This is what I presume you wanted to have happen. Option 2 will simply tell the `snapshot` action to ignore closed indices. If you _want_ the closed indices to be snapshotted, this is perhaps not the option you want to go with. You should simply re-order the actions to close the indices after they've been snapshotted. I tried the option 1 and it works the first run. but the next schedule to run the cron again, it will fail because there are closed indices now. Will the delete action deletes the closed indices too? ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [June 1, 2018, 6:35pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/4 "2018-06-01T18:35:03Z")
The `delete_indices` action will only delete indices you have identified by your filters. Snapshotting at older than 91 is just to save before deletion? Since you are snapshotting, then deleting indices older than `91` days, and you are closing older than 90 days, it seems to me that the `close` action is unnecessary. A closed index for only 1 day is not going to change much. These numbers are rather large. Can your cluster handle 90 days of open indices, in terms of shard count per node? ---
**Author:** ![wanshishi](https://avatars.discourse-cdn.com/v4/letter/w/fbc32d/32.png) [@wanshishi](https://discuss.elastic.co/u/wanshishi)\ **Post date:** [June 1, 2018, 6:56pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/5 "2018-06-01T18:56:52Z")
> [@theuntergeek](#): > > The `delete_indices` action will only delete indices you have identified by your filters. > > Snapshotting at older than 91 is just to save before deletion? Since you are snapshotting, then deleting indices older than `91` days, and you are closing older than 90 days, it seems to me that the `close` action is unnecessary. A closed index for only 1 day is not going to change much. > > These numbers are rather large. Can your cluster handle 90 days of open indices, in terms of shard count per node? Yes the snapshot is to save indices before deletion (somewhat like backup). And yes my cluster can handle 90 days of open indices. So in your perspective, there is no point in closing the indices. Thank you, I am learning something... ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [June 1, 2018, 7:34pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/6 "2018-06-01T19:34:16Z")
Closing the indices is something to do if you couldn't handle that many _open_ indices, even though your storage _can_ handle it. Each open shard has a resource cost (heap space), so the more you have, the more memory constrained your nodes become for indexing operations. In a cluster that could hold 90 days worth of indices, but could not keep more than 30 days worth of indices in an open state, you'd snapshot at 30 days, then close the indices to keep them present, but not used. ---
**Author:** ![wanshishi](https://avatars.discourse-cdn.com/v4/letter/w/fbc32d/32.png) [@wanshishi](https://discuss.elastic.co/u/wanshishi)\ **Post date:** [June 1, 2018, 8:51pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/7 "2018-06-01T20:51:37Z")
Thank you for the clarification master @theuntergeek 😃 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 29, 2018, 8:51pm UTC](https://discuss.elastic.co/t/curator-snapshot-error-with-closed-indices/134259/8 "2018-06-29T20:51:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.