# Shard allocation for backup / archive **URL:** \ **Category:** Elasticsearch\ **Created:** [October 5, 2012, 1:16pm UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251 "2012-10-05T13:16:44Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![James\_Dooley](https://avatars.discourse-cdn.com/v4/letter/j/a8b319/32.png) [@James\_Dooley](https://discuss.elastic.co/u/James_Dooley)\ **Post date:** [October 5, 2012, 1:16pm UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251/1 "2012-10-05T13:16:44Z")
I am hoping to get a system set up where I can allocate a full copy of a indexes shards to a single server for backup / archiving purposes. My understanding is that cluster.routing.allocation.[in|ex]clude.tag or .ip should do the trick. Basically tell all of the indexes to not use the backup server unless I am looking to run backups on that index. I can set the exclude value on each active index using: _curl **-XPUT localhost:9200/** \_settings_ But this has to be set every time a new index is created, and I expect this to get messy since we have multiple indexes per day. I can set a cluster wide setting using: _curl -XPUT localhost:9200/\_cluster/settings_ But this seems to take this as a mandatory setting instead of a default. So for example if the cluster setting says to ignore the backup server I can not tell a individual index to include only the backup server. Are there any alternatives or am I stuck with having to update each indexes settings every time? -- ---
**Author:** ![Marcin\_Dojwa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcin_dojwa/32/2307_2.png) [@Marcin\_Dojwa](https://discuss.elastic.co/u/Marcin_Dojwa)\ **Post date:** [October 5, 2012, 1:41pm UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251/2 "2012-10-05T13:41:33Z")
Hi, I guess you can use templates here. Each new index would get settings from the template automatically. Check > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Best regards. 2012/10/5 James Dooley [smsldoo@gmail.com](mailto:smsldoo@gmail.com) > I am hoping to get a system set up where I can allocate a full copy of a > indexes shards to a single server for backup / archiving purposes. > > My understanding is that cluster.routing.allocation.[in|ex]clude.tag or > .ip should do the trick. Basically tell all of the indexes to not use the > backup server unless I am looking to run backups on that index. > I can set the exclude value on each active index using: > > _curl **-XPUT localhost:9200/** \_settings_ > > But this has to be set every time a new index is created, and I expect this to get messy since we have multiple indexes per day. > > I can set a cluster wide setting using: > > _curl -XPUT localhost:9200/\_cluster/settings_ > > But this seems to take this as a mandatory setting instead of a default. So for example if the cluster setting says to ignore the backup server I can not tell a individual index to include only the backup server. > > Are there any alternatives or am I stuck with having to update each indexes settings every time? > > -- -- ---
**Author:** ![James\_Dooley](https://avatars.discourse-cdn.com/v4/letter/j/a8b319/32.png) [@James\_Dooley](https://discuss.elastic.co/u/James_Dooley)\ **Post date:** [October 5, 2012, 1:53pm UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251/3 "2012-10-05T13:53:37Z")
Hm, looks like that is exactly what I need. Thanks One last question, I just noticed that when I force the index to the backup server using allocation.include: "backup" I am unable to remove the index.routing.allocation.include value when I am done with the value. If I set it to "" it takes it as a literal include no nodes. I looked around a bit, is there any way to actually remove a setting? Example: curl -XPUT 'localhost:9200/index/settings' -d '{"index.routing.allocation.include.tag": "backup"}}' moves one replica worth of shards to the backup server. curl -XPUT 'localhost:9200/index/settings' -d '{"index.routing.allocation.include.tag": ""}}' drops the replica from the backup server but does not allocate them anywhere. I still have the primary shards allocated, but no replicas. I imagine that if I could just remove the index.routing.allocation.include.tag setting all together it would start working normally again. On Friday, October 5, 2012 9:41:47 AM UTC-4, Marcin Dojwa wrote: > Hi, I guess you can use templates here. Each new index would get settings > from the template automatically. Check > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > Best regards. > > 2012/10/5 James Dooley \<[sms...@gmail.com](mailto:sms...@gmail.com) \\> > > > I am hoping to get a system set up where I can allocate a full copy of a > > indexes shards to a single server for backup / archiving purposes. > > > > My understanding is that cluster.routing.allocation.[in|ex]clude.tag or > > .ip should do the trick. Basically tell all of the indexes to not use the > > backup server unless I am looking to run backups on that index. > > I can set the exclude value on each active index using: > > > > _curl **-XPUT localhost:9200/** \_settings_ > > > > But this has to be set every time a new index is created, and I expect this to get messy since we have multiple indexes per day. > > > > I can set a cluster wide setting using: > > > > _curl -XPUT localhost:9200/\_cluster/settings_ > > > > But this seems to take this as a mandatory setting instead of a default. So for example if the cluster setting says to ignore the backup server I can not tell a individual index to include only the backup server. > > > > Are there any alternatives or am I stuck with having to update each indexes settings every time? > > > > -- -- ---
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [October 8, 2012, 1:06am UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251/4 "2012-10-08T01:06:31Z")
James Just a small test to myself, to check if I've understood the docs correctly. What you intend to do is: - tell elasticsearch to "move" a full index (i.e. all of its shards and replicas) to a single node by means of the "routing.allocation.include.\*" setting - tell elasticsearch to exclude all the remaining indexes from that node you just move to previous index to, via the "routing.allocation.exclude.\*" setting - make a backup of that server using any suitable external tool (tar -xcvf of the index file directories, by instance), knowing that the only data that host has is the data of the index you're backing up. - during that time, the index is still available for indexing/searching requests made to the cluster, unless you have used the "block read" and/or "block write" settings for that index. Is that your strategy?? txs for the answer On Friday, October 5, 2012 9:53:37 AM UTC-4, James Dooley wrote: > Hm, looks like that is exactly what I need. Thanks > > One last question, I just noticed that when I force the index to the > backup server using allocation.include: "backup" I am unable to remove the > index.routing.allocation.include value when I am done with the value. > > If I set it to "" it takes it as a literal include no nodes. I looked > around a bit, is there any way to actually remove a setting? > > Example: > > curl -XPUT 'localhost:9200/index/settings' -d > '{"index.routing.allocation.include.tag": "backup"}}' moves one replica > worth of shards to the backup server. > > curl -XPUT 'localhost:9200/index/settings' -d > '{"index.routing.allocation.include.tag": ""}}' drops the replica from the > backup server but does not allocate them anywhere. I still have the primary > shards allocated, but no replicas. > > I imagine that if I could just remove the > index.routing.allocation.include.tag setting all together it would start > working normally again. > > On Friday, October 5, 2012 9:41:47 AM UTC-4, Marcin Dojwa wrote: > > > Hi, I guess you can use templates here. Each new index would get settings > > from the template automatically. Check > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > Best regards. > > > > 2012/10/5 James Dooley [sms...@gmail.com](mailto:sms...@gmail.com) > > > > > I am hoping to get a system set up where I can allocate a full copy of a > > > indexes shards to a single server for backup / archiving purposes. > > > > > > My understanding is that cluster.routing.allocation.[in|ex]clude.tag or > > > .ip should do the trick. Basically tell all of the indexes to not use the > > > backup server unless I am looking to run backups on that index. > > > I can set the exclude value on each active index using: > > > > > > _curl **-XPUT localhost:9200/** \_settings_ > > > > > > But this has to be set every time a new index is created, and I expect this to get messy since we have multiple indexes per day. > > > > > > I can set a cluster wide setting using: > > > > > > _curl -XPUT localhost:9200/\_cluster/settings_ > > > > > > But this seems to take this as a mandatory setting instead of a default. So for example if the cluster setting says to ignore the backup server I can not tell a individual index to include only the backup server. > > > > > > Are there any alternatives or am I stuck with having to update each indexes settings every time? > > > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:09am UTC](https://discuss.elastic.co/t/shard-allocation-for-backup-archive/9251/5 "2017-07-06T03:09:49Z")