# Restore a snapshot error ULTRAWARM INTERNAL EXCEPTION

**URL:** <https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470>\
**Category:** Elasticsearch\
**Created:** [April 10, 2020, 9:51am UTC](https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470 "2020-04-10T09:51:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![michel1](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@michel1](https://discuss.elastic.co/u/michel1)\
**Post date:** [April 10, 2020, 9:51am UTC](https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470/1 "2020-04-10T09:51:50Z")

</div>

I'm trying to restore an index with curator on an Amazon Elasticsearch domain (version 6.8), but the operation seems to failed with the error ULTRAWARM INTERNAL EXCEPTION.  
The curator restore yaml is:

```auto
    actions:
   1:
    action: restore
    description: >-
      Restore all indices in the most recent snapshot with state SUCCESS. Wait
      for the restore to complete before continuing. Do not skip the repository
      filesystem access check. Use the other options to define the index/shard
      settings for the restore.
    options:
      repository: my-repo
      name:
      indices:
        - my_index
      extra_settings:
       index_settings:
        number_of_replicas: 0
        index.auto_expand_replicas: true
       ignore_index_settings:
       - index.refresh_interval
      wait_for_completion: True
      max_wait: 3600
      wait_interval: 10
    filters:
    - filtertype: pattern
      kind: prefix
      value: curator-
    - filtertype: state
      state: SUCCESS

```

The curator output is:

```auto
    2020-04-10 11:24:29,779 INFO Preparing Action ID: 1, "restore"
    2020-04-10 11:24:29,779 INFO Creating client object and testing connection
    2020-04-10 11:24:29,871 INFO Credentials found in config file: ~/.aws/config
    2020-04-10 11:24:29,874 INFO Instantiating client object
    2020-04-10 11:24:29,874 INFO Testing client connectivity
    2020-04-10 11:24:30,223 INFO Successfully created Elasticsearch client object with provided 
    settings
    2020-04-10 11:24:30,278 INFO Trying Action ID: 1, "restore": Restore all indices in the most 
    recent snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do 
    not skip the repository filesystem access check. Use the other options to define the 
    index/shard 
    settings for the restore.
    2020-04-10 11:24:31,017 INFO Restoring indices "['my_index']" from snapshot: 
    curator-20200410072900
    2020-04-10 11:24:31,239 INFO Provided indices: ['my_index']
    2020-04-10 11:24:31,239 INFO Found indices: ['my_index']
    2020-04-10 11:24:31,239 INFO Index "my_index" is still in stage "INDEX"
    2020-04-10 11:24:41,299 INFO _recovery returned an empty response. Trying again.
    2020-04-10 11:24:51,359 INFO _recovery returned an empty response. Trying again.
    2020-04-10 11:25:01,422 INFO _recovery returned an empty response. Trying again.

```

This goes on indefinitely.

When I call

```auto
    POST _cluster/allocation/explain?pretty

```

I get the following error:

```auto
    {
      "index": "my_index",
      "shard": 4,
    "primary": true,
    "current_state": "unassigned",
    "unassigned_info": {
      "reason": "ALLOCATION_FAILED",
      "at": "2020-04-10T09:24:34.518Z",
      "failed_allocation_attempts": 1,
      "details": "[ULTRAWARM INTERNAL EXCEPTION]",
      "last_allocation_status": "no"
     },
     "can_allocate": "no",
     "allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
    "node_allocation_decisions": [
      {
        "node_id": "xhpwVdgrR9ia7ly_jXAYhA",
        "node_name": "xhpwVdg",
        "node_decision": "no",
        "weight_ranking": 1,
        "deciders": [
          {
            "decider": "restore_in_progress",
            "decision": "NO",
            "explanation": "[ULTRAWARM INTERNAL EXCEPTION]"
          }
        ]
      }
    ]
}

```

Both the source and destination domains aren't using the ULTRAWARM feature of AWS.  
Is there anything I need to consider?

Thanks!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 10, 2020, 9:59am UTC](https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470/2 "2020-04-10T09:59:18Z")

</div>

You will likely need to ask this question of your AWS support contacts, since this message is coming from some of AWS's unsupported customisations of Elasticsearch that we can't see or help debug. We don't often see AWS folk around here, sorry.

---

<div class="post-metadata">

**Author:** ![michel1](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@michel1](https://discuss.elastic.co/u/michel1)\
**Post date:** [April 10, 2020, 10:49am UTC](https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470/3 "2020-04-10T10:49:00Z")

</div>

Hi David,  
thanks anyway for your super fast reply.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 8, 2020, 10:49am UTC](https://discuss.elastic.co/t/restore-a-snapshot-error-ultrawarm-internal-exception/227470/4 "2020-05-08T10:49:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.