# Timeout on deleting a snapshot repository

**URL:** <https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936>\
**Category:** Elasticsearch\
**Created:** [December 24, 2016, 3:53am UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936 "2016-12-24T03:53:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![darinhoward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darinhoward/32/14033_2.png) [@darinhoward](https://discuss.elastic.co/u/darinhoward)\
**Post date:** [December 24, 2016, 3:53am UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936/1 "2016-12-24T03:53:00Z")

</div>

I'm running elasticsearch 1.7.5 w/ 19 nodes (12 data nodes).

Attempting to setup snapshots for backup and recovery - but am getting a 503 on creation and deletion of a snapshot repository.

```
curl -XDELETE 'localhost:9200/_snapshot/backups?pretty'

```

returns:

```
{
 "error" : "RemoteTransportException[[masternodename][inet[/10.0.0.20:9300]][cluster:admin/repository/delete]]; nested: ProcessClusterEventTimeoutException[failed to process cluster event (delete_repository [backups]) within 30s]; ",
 "status" : 503
}

```

I was able to adjust the query w/ a master\_timeout=10m - still getting a timeout. Is there a way to debug the cause of this request failing?

Thanks.

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [December 26, 2016, 4:12pm UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936/2 "2016-12-26T16:12:48Z")

</div>

Do you see a lot of [pending tasks](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/cat-pending-tasks.html) on your master node when it happens? If you do, how many tasks are there and what's their most common source?

---

<div class="post-metadata">

**Author:** ![darinhoward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/darinhoward/32/14033_2.png) [@darinhoward](https://discuss.elastic.co/u/darinhoward)\
**Post date:** [December 27, 2016, 3:49pm UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936/3 "2016-12-27T15:49:28Z")

</div>

Originally the cluster had ~10 running tasks (with a higher priority than the put/delete repo). Trying the action again today w/ 0 running tasks - it runs w/ out delay.

I'll monitor - but that may have been the issue.

Thanks.

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [December 27, 2016, 4:01pm UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936/4 "2016-12-27T16:01:23Z")

</div>

Yes, I typically see it on large cluster states when cluster state update tasks in front of a delete repo task take too long to finish (because of a large number of nodes, indices, types, aliases etc.). We have improved the situation in later versions of elasticsearch by switching to shipping cluster state diffs instead of a complete cluster state and implementing cluster state task batching in more places.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 24, 2017, 4:01pm UTC](https://discuss.elastic.co/t/timeout-on-deleting-a-snapshot-repository/69936/5 "2017-01-24T16:01:24Z")

</div>

Closed on January 24, 2017, 4:01pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.