I have a cluster of ES nodes on AWS. Let's say 20 nodes using EBS and 3 using i3en (local disks).
I am testing with i3en nodes to improve IO.
But I am still seeing slow indexing and search logs showing up (up to tens of seconds) on those i3en nodes. It seems there's very little help migrating to local disks. Nodes with EBS is also showing similar slowness.
Question: Will mixing nodes with EBS cause false positive like this? Meaning shards residing on EBS causing slowness with shards residing on local disk?
Do I need all shards to use local disks to see the IO improvement for an index?
iotop also shows the disks are busy only about 1% of time. What could be the slowness? I've increased cpu from 8 core to 16 cores before and it didn't help. Now moving from EBS to local disk and it doesn't seem to help either. Is there another potential bottleneck I over looked?
