# Hot Cold architecture question? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 31, 2020, 7:56am UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811 "2020-03-31T07:56:23Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [March 31, 2020, 7:56am UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/1 "2020-03-31T07:56:24Z")
Dear all I have a question about hot warm cold architecture. So basically i want to skip the warm node and just have the hot and the cold node only, is it possible, and also i hear from a friend that cold node has better compress ratio that hot and warm, is it true because i have a lot of old data and they take up quite alot of space. Thanks you for your help and time. ---
**Author:** ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)\ **Post date:** [April 1, 2020, 3:28pm UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/2 "2020-04-01T15:28:54Z")
@lusynda if you're using [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html) then yes, super easy to just define a policy with only hot/cold defined. Before giving a more indepth answer, some quick background: - hot/warm/cold/etc uses a capability called [index-level shard allocation filtering](https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-allocation-filtering.html) which is a way to tell Elasticsearch to put shards of a given index on to specific machine based on an arbitrary label you assign to that machine. - per above, "hot" and "cold" are really just arbitrary labels assigned to the ES nodes, so it's totally possible to have a "hot" node that runs way slower than "cold". - ILM basically maps the labels you assign (or we do, if running on Elastic Cloud) to your nodes to steps in the lifecycle, like `"my-awesome-node" == "hot"` or `"my-busted-node" == "cold` If you're running your own cluster (not hosted), then "cold" is whatever you want it to be. In other words, it doesn't automatically mean better compression or better anything, really. On Elastic Cloud, the compression ratio doesn't change, but the memory-to-disk ratio does, which will reduce the overall cost of given volume of data. Somewhat more apropos, ILM will gain the ability to change the compression codec during a phase change in v7.7.0. See: [https://github.com/elastic/elasticsearch/pull/49974](https://github.com/elastic/elasticsearch/pull/49974) ---
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [April 3, 2020, 7:48am UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/3 "2020-04-03T07:48:41Z")
Thank you for your respond So what you are telling me that the cold node has no better compression over data so if i have an index of lets say 500GB on the hot node, it will remain the same 500GB on the cold node? And if i have old indices that has not been in to an ILM then how will i be able to move them to cold node. I heard that i can use curator to move them but that was in the old version, is the current version still support curator? ---
**Author:** ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)\ **Post date:** [April 3, 2020, 12:49pm UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/4 "2020-04-03T12:49:34Z")
> So what you are telling me that the cold node has no better compression over data so if i have an index of lets say 500GB on the hot node, it will remain the same 500GB on the cold node? Correct. Compression ratio doesn't change between lifecycle steps, but the memory-to-disk ratio does, meaning you need less memory (fewer nodes) to serve the same amount of data. Also, cold nodes generally use slower, less expensive disk meaning the **cost** (not the size) for data on a cold node is less than it would be on hot or warm. I probably was giving you a deeper answer than you wanted with my first response. I was saying that Elasticsearch does not ship with different types of data nodes. What you do (and what we do in Elastic Cloud) is use different types of hardware in a single cluster, labeling the faster hardware "hot", the slower hardware "slow", and then use ILM to do make the indices move between them and do other optimizations at the same time (like a [forced merge](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html)) > And if i have old indices that has not been in to an ILM then how will i be able to move them to cold node. If you all you want to do is have older data with a higher compression ratio than newer data, you'd need to [reindex](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) data from your "hot" indices to your "cold" ones, where the cold indices have been configured with `"index.codec": "best_compression"`. If you've setup [shard allocation filtering](https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-allocation-filtering.html), then you can move indices from "hot" to "cold" by changing the [index routing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-allocation-filtering.html#index-allocation-settings) on the hot index when ready for them to move. Curator does these API calls for you, but it isn't required. ---
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [April 15, 2020, 7:21am UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/5 "2020-04-15T07:21:23Z")
Sorry for bring this post up this late but my boss has a question that i really dont know how to anwser, So beside the cold node is there any other way for us to archive our old indices and some how compress them even further or iam stuck with the old indices that takes a lot of spaces Thanks you. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 15, 2020, 7:52am UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/6 "2020-04-15T07:52:16Z")
Not that I am aware of as indices already are compressed. ---
**Author:** ![Mike.Barretta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike.barretta/32/16688_2.png) [@Mike.Barretta](https://discuss.elastic.co/u/Mike.Barretta)\ **Post date:** [April 15, 2020, 1:42pm UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/7 "2020-04-15T13:42:07Z")
@lusynda Like Christian said, best compression is best compression. Your only other options are: 1. Drop those cold indices down to 0 replicas, which will cut your data size in half, but leave you vulnerable to data loss 2. Pull the data off the cluster via a [snapshot](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/modules-snapshots.html) and store it in something cheap like an AWS S3 bucket, local tape array, etc. When you need it, you restore back. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 13, 2020, 1:42pm UTC](https://discuss.elastic.co/t/hot-cold-architecture-question/225811/8 "2020-05-13T13:42:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.