# How to switch from file storage to memory storage?
**URL:** https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263
**Category:** Elasticsearch
**Created:** [August 26, 2010, 3:49pm UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263 "2010-08-26T15:49:39Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@ppearcy](https://discuss.elastic.co/u/ppearcy)
#### Post date: [August 26, 2010, 3:49pm UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/1 "2010-08-26T15:49:39Z")
I wanted to run some performance tests against in memory indexes using
the index module store. I set up my elasticsearch.yaml file similar to
what is in this thread:
[http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-index-store-fs-memory-enabled-is-true-td923306.html#a923306](http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-index-store-fs-memory-enabled-is-true-td923306.html#a923306)
So, the index portion of my config looks like this:
index:
analysis :
analyzer :
verity\_tokenizer :
type: pattern
lowercase: true
pattern: '(?:(?!\w).)+'
stopwords: _none_
store:
type: niofs
fs:
memory:
enabled: true
However, after restarting the cluster, it doesn't appear the setting
is taking effect. I've tried playing around with these settings, but
can't get anything to stick.
Any idea what I am missing?
Thanks,
Paul
---
### Author:  [@David\_Jensen\_2](https://discuss.elastic.co/u/David_Jensen_2)
#### Post date: [August 26, 2010, 7:20pm UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/2 "2010-08-26T19:20:49Z")
I was actually able to switch from disk to memory. I have about 14M
records in my Gateway. When I started up brand new Elasticsearch
servers, I could watch the memory usage on each server go up.
Going through the docs and also my configuration. I'm wondering if you
need to change "type: niofs" to "type: memory"
On Aug 26, 8:49 am, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
> I wanted to run some performance tests against in memory indexes using
> the index module store. I set up my elasticsearch.yaml file similar to
> what is in this thread:[http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde](http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde)...
>
> So, the index portion of my config looks like this:
> index:
> analysis :
> analyzer :
> verity\_tokenizer :
> type: pattern
> lowercase: true
> pattern: '(?:(?!\w).)+'
> stopwords: _none_
> store:
> type: niofs
> fs:
> memory:
> enabled: true
>
> However, after restarting the cluster, it doesn't appear the setting
> is taking effect. I've tried playing around with these settings, but
> can't get anything to stick.
>
> Any idea what I am missing?
>
> Thanks,
> Paul
---
### Author:  [@ppearcy](https://discuss.elastic.co/u/ppearcy)
#### Post date: [August 26, 2010, 7:54pm UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/3 "2010-08-26T19:54:38Z")
Nice, thanks! Seems obvious in retrospect.
On Aug 26, 1:20 pm, David Jensen [djense...@gmail.com](mailto:djense...@gmail.com) wrote:
> I was actually able to switch from disk to memory. I have about 14M
> records in my Gateway. When I started up brand new Elasticsearch
> servers, I could watch the memory usage on each server go up.
>
> Going through the docs and also my configuration. I'm wondering if you
> need to change "type: niofs" to "type: memory"
>
> On Aug 26, 8:49 am, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
>
> > I wanted to run some performance tests against in memory indexes using
> > the index module store. I set up my elasticsearch.yaml file similar to
> > what is in this thread:[http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde](http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde)...
>
> > So, the index portion of my config looks like this:
> > index:
> > analysis :
> > analyzer :
> > verity\_tokenizer :
> > type: pattern
> > lowercase: true
> > pattern: '(?:(?!\w).)+'
> > stopwords: _none_
> > store:
> > type: niofs
> > fs:
> > memory:
> > enabled: true
>
> > However, after restarting the cluster, it doesn't appear the setting
> > is taking effect. I've tried playing around with these settings, but
> > can't get anything to stick.
>
> > Any idea what I am missing?
>
> > Thanks,
> > Paul
---
### Author:  [@ppearcy](https://discuss.elastic.co/u/ppearcy)
#### Post date: [August 26, 2010, 9:42pm UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/4 "2010-08-26T21:42:07Z")
So, my config file for all indexes in memory looks like:
index:
store:
type: memory
fs:
memory:
enabled: true
This results in all indexes going to memory. I am looking to go with a
hybrid approach, some in memory and some out of memory.
I believe I should be able to accomplish this by hacking the metadata
file to include this setting, eg:
"settings" : {
"index.number\_of\_replicas" : "0",
"index.number\_of\_shards" : "2",
"index.store.fs.memory.enabled" : "true",
"index.mapper.dynamic" : "false"
},
However, this doesn't seem to work. I also tried out the index
settings update API([GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine](http://github.com/elasticsearch/elasticsearch/)
issues/302), but it doesn't appear to support this option, yet.
Any ideas on how to update a specific index to go straight to memory?
This is probably all a moot point, but I wanted to run performance
tests to prove it, as I believe that the OS's disk cache should be
able to perform similarly to a true in memory index, as long as,
enough RAM is available. I'd be happy to be proven wrong about that,
though.
Thanks,
Paul
On Aug 26, 1:54 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
> Nice, thanks! Seems obvious in retrospect.
>
> On Aug 26, 1:20 pm, David Jensen [djense...@gmail.com](mailto:djense...@gmail.com) wrote:
>
> > I was actually able to switch from disk to memory. I have about 14M
> > records in my Gateway. When I started up brand new Elasticsearch
> > servers, I could watch the memory usage on each server go up.
>
> > Going through the docs and also my configuration. I'm wondering if you
> > need to change "type: niofs" to "type: memory"
>
> > On Aug 26, 8:49 am, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
>
> > > I wanted to run some performance tests against in memory indexes using
> > > the index module store. I set up my elasticsearch.yaml file similar to
> > > what is in this thread:[http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde](http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde)...
>
> > > So, the index portion of my config looks like this:
> > > index:
> > > analysis :
> > > analyzer :
> > > verity\_tokenizer :
> > > type: pattern
> > > lowercase: true
> > > pattern: '(?:(?!\w).)+'
> > > stopwords: _none_
> > > store:
> > > type: niofs
> > > fs:
> > > memory:
> > > enabled: true
>
> > > However, after restarting the cluster, it doesn't appear the setting
> > > is taking effect. I've tried playing around with these settings, but
> > > can't get anything to stick.
>
> > > Any idea what I am missing?
>
> > > Thanks,
> > > Paul
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [August 27, 2010, 10:32am UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/5 "2010-08-27T10:32:24Z")
Hi,
Do you want to change it on the gateway side? When you create an index,
you can specific the settings for it, including if it will use memory index
or not. If you want to go and hack the metadata at the gateway side, you can
shutdown the cluster, add a setting (for the indices you want) -
"index.store.type" : "memory".
-shay.banon
On Fri, Aug 27, 2010 at 12:42 AM, Paul [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote:
> So, my config file for all indexes in memory looks like:
> index:
> store:
> type: memory
> fs:
> memory:
> enabled: true
>
> This results in all indexes going to memory. I am looking to go with a
> hybrid approach, some in memory and some out of memory.
>
> I believe I should be able to accomplish this by hacking the metadata
> file to include this setting, eg:
> "settings" : {
> "index.number\_of\_replicas" : "0",
> "index.number\_of\_shards" : "2",
> "index.store.fs.memory.enabled" : "true",
> "index.mapper.dynamic" : "false"
> },
>
> However, this doesn't seem to work. I also tried out the index
> settings update API([GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine](http://github.com/elasticsearch/elasticsearch/)
> issues/302), but it doesn't appear to support this option, yet.
>
> Any ideas on how to update a specific index to go straight to memory?
>
> This is probably all a moot point, but I wanted to run performance
> tests to prove it, as I believe that the OS's disk cache should be
> able to perform similarly to a true in memory index, as long as,
> enough RAM is available. I'd be happy to be proven wrong about that,
> though.
>
> Thanks,
> Paul
>
> On Aug 26, 1:54 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
>
> > Nice, thanks! Seems obvious in retrospect.
> >
> > On Aug 26, 1:20 pm, David Jensen [djense...@gmail.com](mailto:djense...@gmail.com) wrote:
> >
> > > I was actually able to switch from disk to memory. I have about 14M
> > > records in my Gateway. When I started up brand new Elasticsearch
> > > servers, I could watch the memory usage on each server go up.
> >
> > > Going through the docs and also my configuration. I'm wondering if you
> > > need to change "type: niofs" to "type: memory"
> >
> > > On Aug 26, 8:49 am, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote:
> >
> > > > I wanted to run some performance tests against in memory indexes
> > > > using
> > > > the index module store. I set up my elasticsearch.yaml file similar
> > > > to
> > > > what is in this thread:
> > > > [http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde](http://elasticsearch-users.115913.n3.nabble.com/Out-of-Memory-if-inde)...
> >
> > > > So, the index portion of my config looks like this:
> > > > index:
> > > > analysis :
> > > > analyzer :
> > > > verity\_tokenizer :
> > > > type: pattern
> > > > lowercase: true
> > > > pattern: '(?:(?!\w).)+'
> > > > stopwords: _none_
> > > > store:
> > > > type: niofs
> > > > fs:
> > > > memory:
> > > > enabled: true
> >
> > > > However, after restarting the cluster, it doesn't appear the setting
> > > > is taking effect. I've tried playing around with these settings, but
> > > > can't get anything to stick.
> >
> > > > Any idea what I am missing?
> >
> > > > Thanks,
> > > > Paul
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:20am UTC](https://discuss.elastic.co/t/how-to-switch-from-file-storage-to-memory-storage/3263/6 "2017-07-06T04:20:15Z")