Out Of Memory When Store Index In Memory

Elastic Stack Elasticsearch
1

Hi All,

To increase the suggestion I has changed store type to memory as setting
bellow:

cluster.name: es_zautosuggest_dev
node.name: "es_zasuggest_124"
node.master: true
node.data: true
index.number_of_shards: 5
index.number_of_replicas: 0
index.store.type: memory
index.gateway.type: none
gateway.type: fs
gateway.fs.location: /zserver/java-projects/dev_suggest/data

I have 3 servers like this, #1 I store default, #2 & #3 I store "memory"

The problem is the memory increase too fast when I indexing 4M data for
testing.
The data contains two field:
name (< 10 word, <50 character)
data ( < 50 word, < 300 character)

My server 1 has 40Gb Ram.

I allocate it for 20Gb Ram by setting ES_HEAP_SIZE

set.default.ES_HEAP_SIZE=20480

wrapper.java.initmemory=%ES_HEAP_SIZE%

wrapper.java.maxmemory=%ES_HEAP_SIZE%

For first 2M data, the memory is ok.
But for later 2M data, the memory going too fast.

I check in head plugin and the size is:

My_Index:
size: 606.1mb (1.7gb)
docs: 2356911 (2356912)

I think 1.7Gb is the actually size of data after analysis. so with 20Gb
allocated my server should ok.

Is anything in ES I have missed ?

P/S: Is there a way to set a specify index to store in memory and other is
store in disk ?

Thanks In Advance.

--

2

Hi,

You can set index.store.type to the specified index. Check 'Index Settings'
section here:

It can surely be set at index creation time, but I am not sure it can be
changed while index exists.

Best regards.
Marcin Dojwa

2012/12/19 kidkid zkidkid@gmail.com

index.store.type

--

3

Thanks Marcin Dojwa.

I still have this type of problem. I would try to analysis it later.
I found some other also have problem with store in memory.

Hope kimchi will take a look in it.

On Wednesday, December 19, 2012 4:42:32 PM UTC+7, Marcin Dojwa wrote:

Hi,

You can set index.store.type to the specified index. Check 'Index
Settings' section here:
Elasticsearch Platform — Find real-time answers at scale | Elastic

It can surely be set at index creation time, but I am not sure it can be
changed while index exists.

Best regards.
Marcin Dojwa

2012/12/19 kidkid <zki...@gmail.com <javascript:>>

index.store.type

--