# Snapshot, Hot/Warm Architecture and Upgrade **URL:** \ **Category:** Elasticsearch\ **Tags:** slm-snapshot-lifecycle-management, snapshot-and-restore\ **Created:** [October 12, 2023, 8:55am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887 "2023-10-12T08:55:43Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 12, 2023, 8:55am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/1 "2023-10-12T08:55:43Z")
Hi there, I have a few questions here: - First, if I have an index of 4.5 TB, what is the best way to back up that much data? - Second, my existing cluster has 25 data nodes in total, if I want to apply hot/warm architecture, what is the best practice for it? - Third, currently, my cluster is running on version 7.17. if I want to upgrade my cluster which contains 25 data nodes, 3 master nodes, 2 coordinate nodes, and 1 monitoring site. what is the best practice to upgrade it? your answer will mean a lot Thanks ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 12, 2023, 9:08am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/2 "2023-10-12T09:08:18Z")
> [@yuswanul](#): > > First, if I have an index of 4.5 TB, what is the best way to back up that much data? The only supported way to backup Elasticsearch is through the [snapshot API](https://www.elastic.co/guide/en/elasticsearch/reference/8.10/snapshot-restore-apis.html). Is this a single index or a set of time-based indices? How many primary and replica shards do you have? > [@yuswanul](#): > > Second, my existing cluster has 25 data nodes in total, if I want to apply hot/warm architecture, what is the best practice for it? A hot warm architecture generally assumes you have time-based indices of some sort and a defined retention period. Is that the case for your use case? ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 12, 2023, 9:29am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/3 "2023-10-12T09:29:24Z")
> [@Christian\_Dahlqvist](#): > > The only supported way to backup Elasticsearch is through the [snapshot API](https://www.elastic.co/guide/en/elasticsearch/reference/8.10/snapshot-restore-apis.html). I want to know: - What is the performance of backup & restore based on, and - What are the parameters to speed up the process? > [@Christian\_Dahlqvist](#): > > Is this a single index or a set of time-based indices? How many primary and replica shards do you have? that is a single index and it has 25 primary and replicas. the store.size is 4.5 TB and pri.store.size is 2.2 TB > [@Christian\_Dahlqvist](#): > > A hot warm architecture generally assumes you have time-based indices of some sort and a defined retention period. Is that the case for your use case? umm no, what I want to know is, with my current cluster condition, which is all data nodes in the hot tier by default (CMIIW). because I don't explicitly set the tier for each data node, how do I make the transition to implement this hot/warm architecture? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 12, 2023, 9:35am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/4 "2023-10-12T09:35:50Z")
> [@yuswanul](#): > > I want to know: > > - What is the performance of backup & restore based on, and > - What are the parameters to speed up the process? It depends on a lot of factors, e.g. infrastructure and available resources, so I would recommend you set up a repository and test. > [@yuswanul](#): > > that is a single index and it has 25 primary and replicas. the store.size is 4.5 TB and pri.store.size is 2.2 TB Sounds like you have very large shards, which can cause performance problems. > [@yuswanul](#): > > umm no, what I want to know is, with my current cluster condition, which is all data nodes in the hot tier by default (CMIIW). because I don't explicitly set the tier for each data node, how do I make the transition to implement this hot/warm architecture? If you have a large single index a hot-warm architecture does not make any sense. What are you hoping to achieve? What is the problem you are looking to solve? ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 12, 2023, 10:28am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/5 "2023-10-12T10:28:09Z")
> [@Christian\_Dahlqvist](#): > > Sounds like you have very large shards, which can cause performance problems. yeah, pretty large. each shard has a size of 93-94 GB > [@Christian\_Dahlqvist](#): > > If you have a large single index a hot-warm architecture does not make any sense. What are you hoping to achieve? What is the problem you are looking to solve? actually, it's not the only one. there are a few indexes with a size of 4.2 - 4.5 TB and what I want to achieve is to extend the retention of it ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 13, 2023, 9:13am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/6 "2023-10-13T09:13:12Z")
regarding extending retention, do you have any other suggestions for that? currently, all those big indexes have a retention of 15 days ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 13, 2023, 9:54am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/7 "2023-10-13T09:54:23Z")
It sounds like you are not using time-based indices, is that correct? If so, do you delete data through delete-by-query? Is your data immutable or are you performing updates? ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 13, 2023, 10:32am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/8 "2023-10-13T10:32:36Z")
> [@Christian\_Dahlqvist](#): > > It sounds like you are not using time-based indices, is that correct? no, we have been using time-based indices from the start till now. that big data is really the data that comes in 1 day. here is the capture ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/e/be09ec5934c6d27e94655de5d33e26f48b8716f1.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 13, 2023, 10:42am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/9 "2023-10-13T10:42:25Z")
I can see a few deleted documents in the indices. Does this mean that you are performing updates/deletes or may this be a side effect of you specifying your own document ID? ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 13, 2023, 10:55am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/10 "2023-10-13T10:55:42Z")
yeah, but what is the correlation between deleted documents and backup all those indexes? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 13, 2023, 10:58am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/11 "2023-10-13T10:58:48Z")
This is not related to backup, but switching to a hot/warm architecture. For backup speed you will need to test. ---
**Author:** ![yuswanul](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuswanul/32/101662_2.png) [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Post date:** [October 13, 2023, 11:25am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/12 "2023-10-13T11:25:36Z")
> [@Christian\_Dahlqvist](#): > > a side effect of you specifying your own document ID OK, I can confirm that it is a side effect of specifying my own document ID ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 10, 2023, 11:25am UTC](https://discuss.elastic.co/t/snapshot-hot-warm-architecture-and-upgrade/344887/13 "2023-11-10T11:25:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.