# Using replicas for long term storage in SAN

**URL:** <https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781>\
**Category:** Elasticsearch\
**Created:** [February 25, 2016, 9:39pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781 "2016-02-25T21:39:13Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dkota](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dkota](https://discuss.elastic.co/u/dkota)\
**Post date:** [February 25, 2016, 9:39pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/1 "2016-02-25T21:39:14Z")

</div>

Hi,

We are looking at a production implementation of ELK to replace our logging solution. We are looking at creating daily indices, which would make it easy to manage them.

Here is the issue we would like to have 90 days worth of logs/ indices on the faster disk. And would like to 365 days worth indices in slower disk. But we need all the logs to be searchable.

Is it possible to have 2 replica shards for every primary shard, allocate one of the replica shards to the slow disk( which will also have ES instance running on)?

Can we customize the search engine to not to read from the slower disk node unless we specifically say so?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 25, 2016, 10:02pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/2 "2016-02-25T22:02:25Z")

</div>

> [@dkota](#):
>
> Is it possible to have 2 replica shards for every primary shard, allocate one of the replica shards to the slow disk( which will also have ES instance running on)?

Nope.

> [@dkota](#):
>
> Can we customize the search engine to not to read from the slower disk node unless we specifically say so?

Nope.

Basically you need to move any "old" data onto nodes that are cold, you cannot put a specific shard set (ie replica or primary) on a specific group of machines.

---

<div class="post-metadata">

**Author:** ![dkota](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dkota](https://discuss.elastic.co/u/dkota)\
**Post date:** [February 25, 2016, 10:04pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/3 "2016-02-25T22:04:29Z")

</div>

can explain on what you mean by cold? Would cold nodes be searchable?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 26, 2016, 12:07am UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/4 "2016-02-26T00:07:37Z")

</div>

Cold nodes being those with more, slower storage - ie spinning disks versus SSD.

Yes they are still searchable.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 26, 2016, 1:36am UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/5 "2016-02-26T01:36:13Z")

</div>

You can do this by using a [Hot/Warm architecture](https://www.elastic.co/blog/hot-warm-architecture) where a set of nodes with fast storage handle all indexing as well as querying on the most recent data and a different set of nodes, often with slower disk, handle older data which is generally read-only.

---

<div class="post-metadata">

**Author:** ![dkota](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dkota](https://discuss.elastic.co/u/dkota)\
**Post date:** [February 26, 2016, 5:34pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/6 "2016-02-26T17:34:45Z")

</div>

Thanks Christian, looks like this will solve my problem.

---

<div class="post-metadata">

**Author:** ![dkota](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dkota](https://discuss.elastic.co/u/dkota)\
**Post date:** [February 26, 2016, 9:57pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/7 "2016-02-26T21:57:38Z")

</div>

Will the search be quicker on the hot nodes..what are the configuration changes need. The article doesn't mention.

Also how much compression does elasticsearch offer?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 26, 2016, 10:23pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/8 "2016-02-26T22:23:22Z")

</div>

Search is usually quicker on hot nodes as they have more/faster resources to work with.

ES compressed data by default, that ratio depends entirely on the dataset.

---

<div class="post-metadata">

**Author:** ![dkota](https://avatars.discourse-cdn.com/v4/letter/d/46a35a/32.png) [@dkota](https://discuss.elastic.co/u/dkota)\
**Post date:** [May 20, 2016, 8:45pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/9 "2016-05-20T20:45:49Z")

</div>

Is there a way to remove replicas once the indices are moved to cold node?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 23, 2016, 3:47am UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/10 "2016-05-23T03:47:51Z")

</div>

Just set them to 0 using the APIs.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:49pm UTC](https://discuss.elastic.co/t/using-replicas-for-long-term-storage-in-san/42781/11 "2017-07-05T22:49:53Z")

</div>


