Large synonyms file - Cluster State Exception

Hi - Similar issues were discussed earlier by others, however I am unable to resolve this from the solutions mentioned in those discussions. Recommendation for large synonym file

I have 2 synonyms/phrases files which are 100MB each currently and may increase in size as we add more data.

When I start the ES instance, I get the below exceptions -

[2018-08-20T06:03:47,450][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [tAd08LA] timed out while retrying [indices:admin/exists] after failure (timeout [30s])
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];

I have only one node right now, hence its not failing because of any replication of synonyms files between nodes. Won't ES be able to handle 100 or a 300MB synonyms file? What needs to be done to handle this?

same issue here , in my case i have synonyms file with 1 GB and 512 MB , at the time of starting elasticsearch it gives this logs
[o.e.m.j.JvmGcMonitorService] [tAd08LA] [gc][369] overhead, spent [1.3s] collecting in the last [2.3s]

and finally terminate with OutOfMemory error ,

How much heap do you have assigned to the node?

it is 6gb
.when i start es with less synonym rules i didn't get any issue , but now it's not even starting

Have you tried increasing the heap size? I am not sure how much heap such large dictionaries will require, but would not be surprised if processing them will require more than 6GB.

yeah , it was 2gb before , i increased it upto this one(6Gb) , but still not working

is there any other issues causes this ?any other alternative ways i can take ?

I do not know, but would probably try with an even larger heap.

ok thanks for the suggestion

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.