Large synonyms file - Cluster State Exception

Elastic Stack Elasticsearch
1

Hi - Similar issues were discussed earlier by others, however I am unable to resolve this from the solutions mentioned in those discussions. Recommendation for large synonym file

I have 2 synonyms/phrases files which are 100MB each currently and may increase in size as we add more data.

When I start the ES instance, I get the below exceptions -

[2018-08-20T06:03:47,450][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [tAd08LA] timed out while retrying [indices:admin/exists] after failure (timeout [30s])
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];

I have only one node right now, hence its not failing because of any replication of synonyms files between nodes. Won't ES be able to handle 100 or a 300MB synonyms file? What needs to be done to handle this?

2

same issue here , in my case i have synonyms file with 1 GB and 512 MB , at the time of starting elasticsearch it gives this logs
[o.e.m.j.JvmGcMonitorService] [tAd08LA] [gc][369] overhead, spent [1.3s] collecting in the last [2.3s]

and finally terminate with OutOfMemory error ,

3

How much heap do you have assigned to the node?

4

it is 6gb
.when i start es with less synonym rules i didn't get any issue , but now it's not even starting

5

Have you tried increasing the heap size? I am not sure how much heap such large dictionaries will require, but would not be surprised if processing them will require more than 6GB.

6

yeah , it was 2gb before , i increased it upto this one(6Gb) , but still not working

is there any other issues causes this ?any other alternative ways i can take ?

7

I do not know, but would probably try with an even larger heap.

8

ok thanks for the suggestion

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.