First I will apologize for most likely a naive question. I am currently building out our elasticsearch implementation from a single node, to a 3 node cluster. I have followed most of the tutorials I have found on the internet, but it seems my cluster is slower by up to 3x versus the single node we are currently using. When I do a search on our cluster, I see all the drives get pegged to 100% utilization, and the iowait gets up to around 70 on each node. When I do the same query on the single node, I don't even see a hit to the disk, and very little cpu. Here is our current setup in AWS:
1 750GB 200 PIOPS drive
1 200GB 400 PIOPS drive
The data for the cluster is spread across all 3 nodes, and I mean each node has the complete set of the data. The single node is using all default configuration, but the cluster nodes are just using these added configs:
'cluster.routing.allocation.node_initial_primaries_recoveries' => '4'
'cluster.routing.allocation.node_concurrent_recoveries' => '2'
'indices.recovery.max_bytes_per_sec' => '20mb'
'indices.recovery.concurrent_streams' => '5'
'index.number_of_replicas' => '2'
Are there any other configuration options I should add to get better search performance?
Thank you for your time.