We have set up elasticsearch of 6 nodes cluster, each having 40 cores and 512 Gb Ram. Our index size is around 2.6 Tb and we have created 90 shards for that index.
We are having on OLTP application but we are observing that the performance of elasticsearch queries are not performing good when the number of queries is getting increased. We have set up queue size of 100 in threadpool.
ES heap size is 30 GB. (Min and Max equal), swappiness is set to 1 in the linux system.
Please let us know how can we achieve at least 500 searches in parallel, what are we doing wrong here?