I would like to use MMapDirectory at the data indexing phase (in a batch).
And then switch to index to in-memory and read only at time of serving real
user queries to optimize the search latency. I used to achieve that when
directly deal with Lucene by using RAMDirectory and read-only Searcher. Is
-Des.index.store.type=memory the ElasticSearch equivalent of ensure
RAMDirectory is used underline?
MMapDirectory is already using as much memory as possible, for reading data.
RamDirectory store is when you want to push all data onto the heap,
typically for volatile unit tests. For large index, it puts only burden on
the heap and your performance will suffer from GC.
I would like to use MMapDirectory at the data indexing phase (in a batch).
And then switch to index to in-memory and read only at time of serving real
user queries to optimize the search latency. I used to achieve that when
directly deal with Lucene by using RAMDirectory and read-only Searcher. Is
-Des.index.store.type=memory the Elasticsearch equivalent of ensure
RAMDirectory is used underline?
Does ES allow switching from file-based store to memory-based store without
re-indexing? We used to use file-based store to run a batch index, then
switch to read-only memory-based index when using Lucene directly. We've
found that read-only RAMDirectory is 20% faster than MmapDirecotry. That's
why we still want to use the similar type of configuration when switching
from Lucene to ES.
Thanks,
-Mark
On Tuesday, June 10, 2014 1:58:13 PM UTC-7, Jörg Prante wrote:
Yes, it is equivalent.
MMapDirectory is already using as much memory as possible, for reading
data.
RamDirectory store is when you want to push all data onto the heap,
typically for volatile unit tests. For large index, it puts only burden on
the heap and your performance will suffer from GC.
Jörg
On Tue, Jun 10, 2014 at 7:43 PM, Mark Lu <mark...@gmail.com <javascript:>>
wrote:
I would like to use MMapDirectory at the data indexing phase (in a
batch). And then switch to index to in-memory and read only at time of
serving real user queries to optimize the search latency. I used to achieve
that when directly deal with Lucene by using RAMDirectory and read-only
Searcher. Is -Des.index.store.type=memory the Elasticsearch equivalent of
ensure RAMDirectory is used underline?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.