Hi,
Kindly help me with some tips on tuning elasticseach for faster searching. Sorry for the long post.
I have two elasticsearch clusters
Two bare metal nodes, each with 24 cores, 64 GiB RAM, rotational 1TB HDDAround 5853319 docs (10GiB) per day with 4 primary shards and replication=116GiB heap, no other special configuration
In this cluster, if I do a 7 day search in Kibana 4 dashboard with
around 30 graphs, it takes more than a minute to return all results,
then on applying a filter, it takes another minute to give back results.
Can it do it faster than this?
HQ plugin shows the following stats in warning/error stages for the two nodes:
Search - Query: 308.35ms 355.12ms
Search - Fetch: 104.61ms 84.6ms
Refresh: 34.63ms 29.37ms
Three EC2 c4.4xlarge data nodes, each with 16 cores, 30 GiB RAM, EBS volumesAround 188361074 docs (400GiB) per day with 5 primary shards and replication=116GiB heap
Was getting error in Kibana, so increased circuitbreaker to 75%. Then
did a 7 day search, brought the whole cluster down due to OOM
exceptions. Was that overkill? What can I do to make this search
successful, and to prevent the cluster from going down?
Hoping for some pointers here. Thanks