# Index in memory **URL:** \ **Category:** Elasticsearch\ **Created:** [April 12, 2012, 1:45pm UTC](https://discuss.elastic.co/t/index-in-memory/7307 "2012-04-12T13:45:51Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [April 12, 2012, 1:45pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/1 "2012-04-12T13:45:51Z")
Hello, Please, anybody can check is the config below is correct, for use index in memory. { "topicoindex" : { "settings" : { "index.analysis.filter.myngram.max\_gram" : "200", "index.analysis.filter.jus\_stop.type" : "stop", - ``` "index.store.type" : "memory",* ``` "index.analysis.filter.jus\_stop.enable\_position\_increments" : "false", "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : "/opt/elasticsearch/config/delimiter.txt", "index.number\_of\_replicas" : "0", "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : "true", "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : "true", "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : "true", "index.analysis.filter.myngram.min\_gram" : "1", "index.analysis.filter.myngram.type" : "edgeNGram", "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : "true", "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", "index.analysis.filter.jus\_stop.ignore\_case" : "true", "index.analysis.filter.jus\_stemmer.name" : "brazilian", "index.number\_of\_shards" : "1", "index.analysis.filter.jus\_stop._lang_" : "brazilian", "index.analysis.analyzer.default.filter.3" : "lowercase", "index.analysis.analyzer.default.filter.4" : "myngram", "index.analysis.analyzer.default.tokenizer" : "whitespace", "index.analysis.filter.jus\_word\_delimiter.type" : "word\_delimiter", "index.analysis.filter.myngram.side" : "front", "index.analysis.analyzer.default.filter.1" : "standard", "index.analysis.analyzer.default.filter.2" : "jus\_word\_delimiter", "index.analysis.filter.my\_synonym.synonyms\_path" : "/opt/elasticsearch/config/synonyms\_atos.txt", "index.analysis.analyzer.default.filter.0" : "accentFilter", "index.analysis.filter.my\_synonym.type" : "synonym", "index.analysis.filter.jus\_stemmer.type" : "stemmer", "index.analysis.filter.jus\_stop.stopwords\_path" : "/opt/elasticsearch/config/stopwords.txt", "index.version.created" : "190199" } } } Thanks for all. -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 13, 2012, 12:21pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/2 "2012-04-13T12:21:17Z")
Yes, setting index.store.type to memory will store it in memory. On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > Hello, > > Please, anybody can check is the config below is correct, for use index in > memory. > > { > "topicoindex" : { > "settings" : { > "index.analysis.filter.myngram.max\_gram" : "200", > "index.analysis.filter.jus\_stop.type" : "stop", > > - > > ``` > "index.store.type" : "memory",* > > ``` > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : "false", > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > "/opt/elasticsearch/config/delimiter.txt", > "index.number\_of\_replicas" : "0", > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : "true", > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : "true", > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : "true", > "index.analysis.filter.myngram.min\_gram" : "1", > "index.analysis.filter.myngram.type" : "edgeNGram", > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : "true", > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > "index.number\_of\_shards" : "1", > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > "index.analysis.analyzer.default.filter.3" : "lowercase", > "index.analysis.analyzer.default.filter.4" : "myngram", > "index.analysis.analyzer.default.tokenizer" : "whitespace", > "index.analysis.filter.jus\_word\_delimiter.type" : "word\_delimiter", > "index.analysis.filter.myngram.side" : "front", > "index.analysis.analyzer.default.filter.1" : "standard", > "index.analysis.analyzer.default.filter.2" : "jus\_word\_delimiter", > "index.analysis.filter.my\_synonym.synonyms\_path" : > "/opt/elasticsearch/config/synonyms\_atos.txt", > "index.analysis.analyzer.default.filter.0" : "accentFilter", > "index.analysis.filter.my\_synonym.type" : "synonym", > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > "index.analysis.filter.jus\_stop.stopwords\_path" : > "/opt/elasticsearch/config/stopwords.txt", > "index.version.created" : "190199" > } > } > } > > Thanks for all. > > -- > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [April 13, 2012, 12:37pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/3 "2012-04-13T12:37:17Z")
Thanks, Shay When i user store in memory, when i restart the ES i will lost all my data ? When i stop and start the ES(0.19.1) i have the massage below. [2012-04-13 09:32:46,713][INFO][gateway] [Death Adder] recovered [1] indices into cluster\_state [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death Adder] [topicoindex][0] failed to start shard org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death Adder] [topicoindex][0] failed to start shard org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > Yes, setting index.store.type to memory will store it in memory. > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > Hello, > > > > Please, anybody can check is the config below is correct, for use index > > in memory. > > > > { > > "topicoindex" : { > > "settings" : { > > "index.analysis.filter.myngram.max\_gram" : "200", > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > - > > > > ``` > > "index.store.type" : "memory",* > > > > ``` > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > "false", > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > "/opt/elasticsearch/config/delimiter.txt", > > "index.number\_of\_replicas" : "0", > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : "true", > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > "true", > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : "true", > > "index.analysis.filter.myngram.min\_gram" : "1", > > "index.analysis.filter.myngram.type" : "edgeNGram", > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > "true", > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > "index.number\_of\_shards" : "1", > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > "index.analysis.analyzer.default.filter.4" : "myngram", > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > "index.analysis.filter.jus\_word\_delimiter.type" : "word\_delimiter", > > "index.analysis.filter.myngram.side" : "front", > > "index.analysis.analyzer.default.filter.1" : "standard", > > "index.analysis.analyzer.default.filter.2" : "jus\_word\_delimiter", > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > "index.analysis.filter.my\_synonym.type" : "synonym", > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > "/opt/elasticsearch/config/stopwords.txt", > > "index.version.created" : "190199" > > } > > } > > } > > > > Thanks for all. > > > > -- > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 13, 2012, 12:46pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/4 "2012-04-13T12:46:53Z")
Yea, its a failure that should be fixed, you won't be able to retain the data when using local gateway and in memory index. So we either need to delete the index or have it come up empty. Btw, why do you need to have the index in memory? On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > Thanks, Shay > > When i user store in memory, when i restart the ES i will lost all my data > ? > > When i stop and start the ES(0.19.1) i have the massage below. > > [2012-04-13 09:32:46,713][INFO][gateway] [Death Adder] > recovered [1] indices into cluster\_state > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death Adder] > [topicoindex][0] failed to start shard > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > [topicoindex][0] shard allocated for local recovery (post api), should > exists, but doesn't > at > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > at > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] > sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], > [P], s[INITIALIZING], reason [Failed to start shard, message > [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for > local recovery (post api), should exists, but doesn't]]] > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] > received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], > [P], s[INITIALIZING], reason [Failed to start shard, message > [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for > local recovery (post api), should exists, but doesn't]]] > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death Adder] > [topicoindex][0] failed to start shard > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > [topicoindex][0] shard allocated for local recovery (post api), should > exists, but doesn't > at > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > at > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] > sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], > [P], s[INITIALIZING], reason [Failed to start shard, message > [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for > local recovery (post api), should exists, but doesn't]]] > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] > received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], > [P], s[INITIALIZING], reason [Failed to start shard, message > [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for > local recovery (post api), should exists, but doesn't]]] > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > Yes, setting index.store.type to memory will store it in memory. > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > > > Hello, > > > > > > Please, anybody can check is the config below is correct, for use index > > > in memory. > > > > > > { > > > "topicoindex" : { > > > "settings" : { > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > > > - > > > > > > ``` > > > "index.store.type" : "memory",* > > > > > > ``` > > > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > > "false", > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > > "/opt/elasticsearch/config/delimiter.txt", > > > "index.number\_of\_replicas" : "0", > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : > > > "true", > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > > "true", > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : "true", > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > > "true", > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > "index.number\_of\_shards" : "1", > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > "index.analysis.filter.jus\_word\_delimiter.type" : "word\_delimiter", > > > "index.analysis.filter.myngram.side" : "front", > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > "index.analysis.analyzer.default.filter.2" : "jus\_word\_delimiter", > > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > > "/opt/elasticsearch/config/stopwords.txt", > > > "index.version.created" : "190199" > > > } > > > } > > > } > > > > > > Thanks for all. > > > > > > -- > > > Gustavo Maia > > -- > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [April 13, 2012, 12:56pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/5 "2012-04-13T12:56:00Z")
I wanted to make the index in memory. To get better performance. Would any other way to do this and a similar performance? If I have 30GB of memory and using FS, the ES will use all the memory if necessary? Em 13 de abril de 2012 09:46, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > Yea, its a failure that should be fixed, you won't be able to retain the data when using local gateway and in memory index. So we either need to delete the index or have it come up empty. Btw, why do you need to have the index in memory? > > On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > Thanks, Shay > > > > When i user store in memory, when i restart the ES i will lost all my data ? > > > > When i stop and start the ES(0.19.1) i have the massage below. > > > > [2012-04-13 09:32:46,713][INFO][gateway] [Death Adder] recovered [1] indices into cluster\_state > > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death Adder] [topicoindex][0] failed to start shard > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't > > at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:636) > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death Adder] received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] > > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death Adder] [topicoindex][0] failed to start shard > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't > > at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:636) > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] sending failed shard for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death Adder] received shard failed for [topicoindex][0], node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard allocated for local recovery (post api), should exists, but doesn't]]] > > > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > Yes, setting index.store.type to memory will store it in memory. > > > > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > Hello, > > > > > > > > Please, anybody can check is the config below is correct, for use index in memory. > > > > > > > > { > > > > "topicoindex" : { > > > > "settings" : { > > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > "index.store.type" : "memory", > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : "false", > > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : "/opt/elasticsearch/config/delimiter.txt", > > > > "index.number\_of\_replicas" : "0", > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : "true", > > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : "true", > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : "true", > > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : "true", > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", > > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > > "index.number\_of\_shards" : "1", > > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > > "index.analysis.filter.jus\_word\_delimiter.type" : "word\_delimiter", > > > > "index.analysis.filter.myngram.side" : "front", > > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > > "index.analysis.analyzer.default.filter.2" : "jus\_word\_delimiter", > > > > "index.analysis.filter.my\_synonym.synonyms\_path" : "/opt/elasticsearch/config/synonyms\_atos.txt", > > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > > "index.analysis.filter.jus\_stop.stopwords\_path" : "/opt/elasticsearch/config/stopwords.txt", > > > > "index.version.created" : "190199" > > > > } > > > > } > > > > } > > > > > > > > Thanks for all. > > > > > > > > -- > > > > Gustavo Maia > > > > -- > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 17, 2012, 7:56am UTC](https://discuss.elastic.co/t/index-in-memory/7307/6 "2012-04-17T07:56:44Z")
The OS will use the file system cache to boost performance. On Fri, Apr 13, 2012 at 3:56 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > I wanted to make the index in memory. To get better performance. > Would any other way to do this and a similar performance? > If I have 30GB of memory and using FS, the ES will use all the memory > if necessary? > > Em 13 de abril de 2012 09:46, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > Yea, its a failure that should be fixed, you won't be able to retain the > > data when using local gateway and in memory index. So we either need to > > delete the index or have it come up empty. Btw, why do you need to have the > > index in memory? > > > > On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > wrote: > > > > > Thanks, Shay > > > > > > When i user store in memory, when i restart the ES i will lost all my > > > data ? > > > > > > When i stop and start the ES(0.19.1) i have the massage below. > > > > > > [2012-04-13 09:32:46,713][INFO][gateway] [Death > > > Adder] recovered [1] indices into cluster\_state > > > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death > > > Adder] [topicoindex][0] failed to start shard > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > exists, but doesn't > > > at > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > at > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > at java.lang.Thread.run(Thread.java:636) > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > Adder] sending failed shard for [topicoindex][0], > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > Adder] received shard failed for [topicoindex][0], > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death > > > Adder] [topicoindex][0] failed to start shard > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > exists, but doesn't > > > at > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > at > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > at java.lang.Thread.run(Thread.java:636) > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > Adder] sending failed shard for [topicoindex][0], > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > Adder] received shard failed for [topicoindex][0], > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > Yes, setting index.store.type to memory will store it in memory. > > > > > > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > wrote: > > > > > > > > > Hello, > > > > > > > > > > Please, anybody can check is the config below is correct, for use > > > > > index in memory. > > > > > > > > > > { > > > > > "topicoindex" : { > > > > > "settings" : { > > > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > > "index.store.type" : "memory", > > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > > > > "false", > > > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > > > > "/opt/elasticsearch/config/delimiter.txt", > > > > > "index.number\_of\_replicas" : "0", > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : > > > > > "true", > > > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > > > > "true", > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : > > > > > "true", > > > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > > > > "true", > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : "true", > > > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > > > "index.number\_of\_shards" : "1", > > > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > > > "index.analysis.filter.jus\_word\_delimiter.type" : > > > > > "word\_delimiter", > > > > > "index.analysis.filter.myngram.side" : "front", > > > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > > > "index.analysis.analyzer.default.filter.2" : > > > > > "jus\_word\_delimiter", > > > > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > > > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > > > > "/opt/elasticsearch/config/stopwords.txt", > > > > > "index.version.created" : "190199" > > > > > } > > > > > } > > > > > } > > > > > > > > > > Thanks for all. > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia > > -- > Gustavo Maia ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [April 17, 2012, 8:58am UTC](https://discuss.elastic.co/t/index-in-memory/7307/7 "2012-04-17T08:58:34Z")
Gustavo, just FYI I switched to in-memory indices as well due to performance reasons (having slower disk IO can be a real problem). However, I was able to use index.store.type: mmapfs with local gateway with success (at least it looks like it is working fine so far). Shay, is there any issue with using mmapfs indices and local gateway in multi-node cluster? Regards, Lukas On Tue, Apr 17, 2012 at 9:56 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > The OS will use the file system cache to boost performance. > > On Fri, Apr 13, 2012 at 3:56 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > I wanted to make the index in memory. To get better performance. > > Would any other way to do this and a similar performance? > > If I have 30GB of memory and using FS, the ES will use all the memory > > if necessary? > > > > Em 13 de abril de 2012 09:46, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > Yea, its a failure that should be fixed, you won't be able to retain > > > the data when using local gateway and in memory index. So we either need to > > > delete the index or have it come up empty. Btw, why do you need to have the > > > index in memory? > > > > > > On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > Thanks, Shay > > > > > > > > When i user store in memory, when i restart the ES i will lost all my > > > > data ? > > > > > > > > When i stop and start the ES(0.19.1) i have the massage below. > > > > > > > > [2012-04-13 09:32:46,713][INFO][gateway] [Death > > > > Adder] recovered [1] indices into cluster\_state > > > > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death > > > > Adder] [topicoindex][0] failed to start shard > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > exists, but doesn't > > > > at > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > at > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > at > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > at java.lang.Thread.run(Thread.java:636) > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > Adder] sending failed shard for [topicoindex][0], > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > Adder] received shard failed for [topicoindex][0], > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death > > > > Adder] [topicoindex][0] failed to start shard > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > exists, but doesn't > > > > at > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > at > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > at > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > at java.lang.Thread.run(Thread.java:636) > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > Adder] sending failed shard for [topicoindex][0], > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > Adder] received shard failed for [topicoindex][0], > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > > > Yes, setting index.store.type to memory will store it in memory. > > > > > > > > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia \< > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > Please, anybody can check is the config below is correct, for use > > > > > > index in memory. > > > > > > > > > > > > { > > > > > > "topicoindex" : { > > > > > > "settings" : { > > > > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > > > "index.store.type" : "memory", > > > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > > > > > "false", > > > > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > > > > > "/opt/elasticsearch/config/delimiter.txt", > > > > > > "index.number\_of\_replicas" : "0", > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : > > > > > > "true", > > > > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > > > > > "true", > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : > > > > > > "true", > > > > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > > > > > "true", > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : > > > > > > "true", > > > > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > > > > "index.number\_of\_shards" : "1", > > > > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > > > > "index.analysis.filter.jus\_word\_delimiter.type" : > > > > > > "word\_delimiter", > > > > > > "index.analysis.filter.myngram.side" : "front", > > > > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > > > > "index.analysis.analyzer.default.filter.2" : > > > > > > "jus\_word\_delimiter", > > > > > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > > > > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > > > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > > > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > > > > > "/opt/elasticsearch/config/stopwords.txt", > > > > > > "index.version.created" : "190199" > > > > > > } > > > > > > } > > > > > > } > > > > > > > > > > > > Thanks for all. > > > > > > > > > > > > -- > > > > > > Gustavo Maia > > > > > > > > -- > > > > Gustavo Maia > > > > -- > > Gustavo Maia ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [April 17, 2012, 9:01am UTC](https://discuss.elastic.co/t/index-in-memory/7307/8 "2012-04-17T09:01:04Z")
Should have said I tested this on 0.19.0 and 0.19.2. On Tue, Apr 17, 2012 at 10:58 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Gustavo, > > just FYI I switched to in-memory indices as well due to performance > reasons (having slower disk IO can be a real problem). However, I was able > to use index.store.type: mmapfs with local gateway with success (at least > it looks like it is working fine so far). > > Shay, is there any issue with using mmapfs indices and local gateway in > multi-node cluster? > > Regards, > Lukas > > On Tue, Apr 17, 2012 at 9:56 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > The OS will use the file system cache to boost performance. > > > > On Fri, Apr 13, 2012 at 3:56 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > > > I wanted to make the index in memory. To get better performance. > > > Would any other way to do this and a similar performance? > > > If I have 30GB of memory and using FS, the ES will use all the memory > > > if necessary? > > > > > > Em 13 de abril de 2012 09:46, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > Yea, its a failure that should be fixed, you won't be able to retain > > > > the data when using local gateway and in memory index. So we either need to > > > > delete the index or have it come up empty. Btw, why do you need to have the > > > > index in memory? > > > > > > > > On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > wrote: > > > > > > > > > Thanks, Shay > > > > > > > > > > When i user store in memory, when i restart the ES i will lost all my > > > > > data ? > > > > > > > > > > When i stop and start the ES(0.19.1) i have the massage below. > > > > > > > > > > [2012-04-13 09:32:46,713][INFO][gateway] [Death > > > > > Adder] recovered [1] indices into cluster\_state > > > > > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death > > > > > Adder] [topicoindex][0] failed to start shard > > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > > exists, but doesn't > > > > > at > > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > > at > > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > > at java.lang.Thread.run(Thread.java:636) > > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > > Adder] sending failed shard for [topicoindex][0], > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > > Adder] received shard failed for [topicoindex][0], > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death > > > > > Adder] [topicoindex][0] failed to start shard > > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > > exists, but doesn't > > > > > at > > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > > at > > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > > at java.lang.Thread.run(Thread.java:636) > > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > > Adder] sending failed shard for [topicoindex][0], > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > > Adder] received shard failed for [topicoindex][0], > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > > > > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > > > > > Yes, setting index.store.type to memory will store it in memory. > > > > > > > > > > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia \< > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > Please, anybody can check is the config below is correct, for use > > > > > > > index in memory. > > > > > > > > > > > > > > { > > > > > > > "topicoindex" : { > > > > > > > "settings" : { > > > > > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > > > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > > > > "index.store.type" : "memory", > > > > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > > > > > > "false", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > > > > > > "/opt/elasticsearch/config/delimiter.txt", > > > > > > > "index.number\_of\_replicas" : "0", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : > > > > > > > "true", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > > > > > > "true", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : > > > > > > > "true", > > > > > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > > > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > > > > > > "true", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : > > > > > > > "true", > > > > > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > > > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > > > > > "index.number\_of\_shards" : "1", > > > > > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > > > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > > > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > > > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > > > > > "index.analysis.filter.jus\_word\_delimiter.type" : > > > > > > > "word\_delimiter", > > > > > > > "index.analysis.filter.myngram.side" : "front", > > > > > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > > > > > "index.analysis.analyzer.default.filter.2" : > > > > > > > "jus\_word\_delimiter", > > > > > > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > > > > > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > > > > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > > > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > > > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > > > > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > > > > > > "/opt/elasticsearch/config/stopwords.txt", > > > > > > > "index.version.created" : "190199" > > > > > > > } > > > > > > > } > > > > > > > } > > > > > > > > > > > > > > Thanks for all. > > > > > > > > > > > > > > -- > > > > > > > Gustavo Maia > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [April 17, 2012, 1:27pm UTC](https://discuss.elastic.co/t/index-in-memory/7307/9 "2012-04-17T13:27:48Z")
Thanks Lukas, I will try mmapfs. Thanks for all. Em 17 de abril de 2012 06:01, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) escreveu: > Should have said I tested this on 0.19.0 and 0.19.2. > > On Tue, Apr 17, 2012 at 10:58 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > > > Gustavo, > > > > just FYI I switched to in-memory indices as well due to performance > > reasons (having slower disk IO can be a real problem). However, I was able > > to use index.store.type: mmapfs with local gateway with success (at least it > > looks like it is working fine so far). > > > > Shay, is there any issue with using mmapfs indices and local gateway in > > multi-node cluster? > > > > Regards, > > Lukas > > > > On Tue, Apr 17, 2012 at 9:56 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > > > The OS will use the file system cache to boost performance. > > > > > > On Fri, Apr 13, 2012 at 3:56 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > I wanted to make the index in memory. To get better performance. > > > > Would any other way to do this and a similar performance? > > > > If I have 30GB of memory and using FS, the ES will use all the memory > > > > if necessary? > > > > > > > > Em 13 de abril de 2012 09:46, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > > > Yea, its a failure that should be fixed, you won't be able to retain > > > > > the data when using local gateway and in memory index. So we either need to > > > > > delete the index or have it come up empty. Btw, why do you need to have the > > > > > index in memory? > > > > > > > > > > On Fri, Apr 13, 2012 at 3:37 PM, Gustavo Maia > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > Thanks, Shay > > > > > > > > > > > > When i user store in memory, when i restart the ES i will lost all my > > > > > > data ? > > > > > > > > > > > > When i stop and start the ES(0.19.1) i have the massage below. > > > > > > > > > > > > [2012-04-13 09:32:46,713][INFO][gateway] [Death > > > > > > Adder] recovered [1] indices into cluster\_state > > > > > > [2012-04-13 09:32:46,726][WARN][indices.cluster] [Death > > > > > > Adder] [topicoindex][0] failed to start shard > > > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > > > exists, but doesn't > > > > > > at > > > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > > > at > > > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > > > at java.lang.Thread.run(Thread.java:636) > > > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > > > Adder] sending failed shard for [topicoindex][0], > > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > [2012-04-13 09:32:46,756][WARN][cluster.action.shard] [Death > > > > > > Adder] received shard failed for [topicoindex][0], > > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > [2012-04-13 09:32:55,983][WARN][indices.cluster] [Death > > > > > > Adder] [topicoindex][0] failed to start shard > > > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > > > > > [topicoindex][0] shard allocated for local recovery (post api), should > > > > > > exists, but doesn't > > > > > > at > > > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:107) > > > > > > at > > > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > > > > > at java.lang.Thread.run(Thread.java:636) > > > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > > > Adder] sending failed shard for [topicoindex][0], > > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > [2012-04-13 09:32:56,008][WARN][cluster.action.shard] [Death > > > > > > Adder] received shard failed for [topicoindex][0], > > > > > > node[i1FbZPaYQTSZ663icY\_4Ww], [P], s[INITIALIZING], reason [Failed to start > > > > > > shard, message [IndexShardGatewayRecoveryException[[topicoindex][0] shard > > > > > > allocated for local recovery (post api), should exists, but doesn't]]] > > > > > > > > > > > > Em 13 de abril de 2012 09:21, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) escreveu: > > > > > > > > > > > > > Yes, setting index.store.type to memory will store it in memory. > > > > > > > > > > > > > > On Thu, Apr 12, 2012 at 4:45 PM, Gustavo Maia > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > Please, anybody can check is the config below is correct, for use > > > > > > > > index in memory. > > > > > > > > > > > > > > > > { > > > > > > > > "topicoindex" : { > > > > > > > > "settings" : { > > > > > > > > "index.analysis.filter.myngram.max\_gram" : "200", > > > > > > > > "index.analysis.filter.jus\_stop.type" : "stop", > > > > > > > > "index.store.type" : "memory", > > > > > > > > "index.analysis.filter.jus\_stop.enable\_position\_increments" : > > > > > > > > "false", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.type\_table\_path" : > > > > > > > > "/opt/elasticsearch/config/delimiter.txt", > > > > > > > > "index.number\_of\_replicas" : "0", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_numbers" : > > > > > > > > "true", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.preserve\_original" : > > > > > > > > "true", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_words" : > > > > > > > > "true", > > > > > > > > "index.analysis.filter.myngram.min\_gram" : "1", > > > > > > > > "index.analysis.filter.myngram.type" : "edgeNGram", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.split\_on\_numerics" : > > > > > > > > "true", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.catenate\_all" : > > > > > > > > "true", > > > > > > > > "index.analysis.filter.jus\_stop.ignore\_case" : "true", > > > > > > > > "index.analysis.filter.jus\_stemmer.name" : "brazilian", > > > > > > > > "index.number\_of\_shards" : "1", > > > > > > > > "index.analysis.filter.jus\_stop._lang_" : "brazilian", > > > > > > > > "index.analysis.analyzer.default.filter.3" : "lowercase", > > > > > > > > "index.analysis.analyzer.default.filter.4" : "myngram", > > > > > > > > "index.analysis.analyzer.default.tokenizer" : "whitespace", > > > > > > > > "index.analysis.filter.jus\_word\_delimiter.type" : > > > > > > > > "word\_delimiter", > > > > > > > > "index.analysis.filter.myngram.side" : "front", > > > > > > > > "index.analysis.analyzer.default.filter.1" : "standard", > > > > > > > > "index.analysis.analyzer.default.filter.2" : > > > > > > > > "jus\_word\_delimiter", > > > > > > > > "index.analysis.filter.my\_synonym.synonyms\_path" : > > > > > > > > "/opt/elasticsearch/config/synonyms\_atos.txt", > > > > > > > > "index.analysis.analyzer.default.filter.0" : "accentFilter", > > > > > > > > "index.analysis.filter.my\_synonym.type" : "synonym", > > > > > > > > "index.analysis.filter.jus\_stemmer.type" : "stemmer", > > > > > > > > "index.analysis.filter.jus\_stop.stopwords\_path" : > > > > > > > > "/opt/elasticsearch/config/stopwords.txt", > > > > > > > > "index.version.created" : "190199" > > > > > > > > } > > > > > > > > } > > > > > > > > } > > > > > > > > > > > > > > > > Thanks for all. > > > > > > > > > > > > > > > > -- > > > > > > > > Gustavo Maia > > > > > > > > > > > > -- > > > > > > Gustavo Maia > > > > > > > > -- > > > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:32am UTC](https://discuss.elastic.co/t/index-in-memory/7307/10 "2017-07-06T03:32:15Z")