Hi - Similar issues were discussed earlier by others, however I am unable to resolve this from the solutions mentioned in those discussions. Recommendation for large synonym file
I have 2 synonyms/phrases files which are 100MB each currently and may increase in size as we add more data.
When I start the ES instance, I get the below exceptions -
[2018-08-20T06:03:47,450][DEBUG][o.e.a.a.i.e.i.TransportIndicesExistsAction] [tAd08LA] timed out while retrying [indices:admin/exists] after failure (timeout [30s])
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
I have only one node right now, hence its not failing because of any replication of synonyms files between nodes. Won't ES be able to handle 100 or a 300MB synonyms file? What needs to be done to handle this?
same issue here , in my case i have synonyms file with 1 GB and 512 MB , at the time of starting elasticsearch it gives this logs
[o.e.m.j.JvmGcMonitorService] [tAd08LA] [gc][369] overhead, spent [1.3s] collecting in the last [2.3s]
and finally terminate with OutOfMemory error ,
How much heap do you have assigned to the node?
it is 6gb
.when i start es with less synonym rules i didn't get any issue , but now it's not even starting
Have you tried increasing the heap size? I am not sure how much heap such large dictionaries will require, but would not be surprised if processing them will require more than 6GB.
yeah , it was 2gb before , i increased it upto this one(6Gb) , but still not working
is there any other issues causes this ?any other alternative ways i can take ?
I do not know, but would probably try with an even larger heap.
ok thanks for the suggestion