# High availability when creating "in memory" index without persisting the index into a storage **URL:** \ **Category:** Elasticsearch\ **Created:** [October 5, 2011, 1:15pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522 "2011-10-05T13:15:02Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Spico\_Florin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spico_florin/32/49309_2.png) [@Spico\_Florin](https://discuss.elastic.co/u/Spico_Florin)\ **Post date:** [October 5, 2011, 1:15pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/1 "2011-10-05T13:15:02Z")
Hello! We have the following scenario: 1. we would like to maintain an index of documents in memory 2. we don't want to keep the index persisted in a storage. Practically, our index works as temporary index in memory where we add new documents and delete the obsoleted ones. Under these circumstances, if a node fails, the indexed data contained in it will be available on the cluster? Meaning, is the in-memory index replicated over the cluster? We look forward for your answers. Thank you. Regards, Florin ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [October 5, 2011, 4:32pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/2 "2011-10-05T16:32:40Z")
Front ending ES with Hazelcast wont be a bad idea. LINK - [http://www.hazelcast.com/](http://www.hazelcast.com/) Thanks Vineeth On Wed, Oct 5, 2011 at 6:45 PM, Spico Florin [spicoflorin@gmail.com](mailto:spicoflorin@gmail.com) wrote: > Hello! > We have the following scenario: > > 1. we would like to maintain an index of documents in memory > 2. we don't want to keep the index persisted in a storage. > Practically, our index works as temporary index in memory where we add new > documents and delete the obsoleted ones. > Under these circumstances, if a node fails, the indexed data contained in > it will be available on the cluster? Meaning, is the in-memory index > replicated over the cluster? > > We look forward for your answers. > > Thank you. > Regards, > Florin ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [October 5, 2011, 5:28pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/3 "2011-10-05T17:28:13Z")
Well, if you don't need persistence then either Hazelcast or ElasticSearch will work fine as is. If you want to query by something other than key, then you would want to include ElasticSearch. Using ElasticSearch, your data is replicated (if you configure your index such) whether you use a memory store or file-based storage. Look at the store types here: [http://www.elasticsearch.org/guide/reference/index-modules/store.html](http://www.elasticsearch.org/guide/reference/index-modules/store.html) ---
**Author:** ![Ridvan\_Gyundogan](https://avatars.discourse-cdn.com/v4/letter/r/e495f1/32.png) [@Ridvan\_Gyundogan](https://discuss.elastic.co/u/Ridvan_Gyundogan)\ **Post date:** [October 6, 2011, 7:21am UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/4 "2011-10-06T07:21:21Z")
Can you explain in more detail, what do you mean by "front ending". I have to say that I like both elasticsearch and hazelcast. On Oct 5, 7:32 pm, Vineeth Mohan [vineethmo...@algotree.com](mailto:vineethmo...@algotree.com) wrote: > Front ending ES with Hazelcast wont be a bad idea. > > LINK -[http://www.hazelcast.com/](http://www.hazelcast.com/) > > Thanks > Vineeth > > On Wed, Oct 5, 2011 at 6:45 PM, Spico Florin [spicoflo...@gmail.com](mailto:spicoflo...@gmail.com) wrote: > > > Hello! > > We have the following scenario: > > > > 1. we would like to maintain an index of documents in memory > > 2. we don't want to keep the index persisted in a storage. > > Practically, our index works as temporary index in memory where we add new > > documents and delete the obsoleted ones. > > Under these circumstances, if a node fails, the indexed data contained in > > it will be available on the cluster? Meaning, is the in-memory index > > replicated over the cluster? > > > We look forward for your answers. > > > Thank you. > > Regards, > > Florin ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [October 6, 2011, 2:14pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/5 "2011-10-06T14:14:49Z")
By "front-ending" he was referring to the option of using Hazelcast as your memcache layer, and having it use ES for persistence. Hazelcast has a simple MapStore interface you can implement to read/write objects from ES. Since you don't need persistence, there is little need for that set up. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 6, 2011, 7:04pm UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/6 "2011-10-06T19:04:47Z")
Yes, an in memory index is replicated across the cluster, and if one node fails, then you have a replica. Note, the internal transaction log used in elasticsearch is always file based one. I highly suggest just using the file system based one, its simpler, and you can use the mmapfs one for better perf. why do you want it to be in memory one? On Wed, Oct 5, 2011 at 3:15 PM, Spico Florin [spicoflorin@gmail.com](mailto:spicoflorin@gmail.com) wrote: > Hello! > We have the following scenario: > > 1. we would like to maintain an index of documents in memory > 2. we don't want to keep the index persisted in a storage. > Practically, our index works as temporary index in memory where we add new > documents and delete the obsoleted ones. > Under these circumstances, if a node fails, the indexed data contained in > it will be available on the cluster? Meaning, is the in-memory index > replicated over the cluster? > > We look forward for your answers. > > Thank you. > Regards, > Florin ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:52am UTC](https://discuss.elastic.co/t/high-availability-when-creating-in-memory-index-without-persisting-the-index-into-a-storage/5522/7 "2017-07-06T03:52:41Z")