We try to benchmark the frozen indices part, and recognize some strange CPU behavior:
We have Hot - Cold architecture
In the test time, we test the nodes only with frozen indices.
We do not write in the test time, so there are no data changes in the data set.
We have 28 indices of 2 shards with one replica. Total data set 2.4TB include replica
The Cold part VMs have 4 Cores 16GB RAM 4TB local SAS storage
We run one or two searches from the Kibana discovery to the whole data set.
Not so important how many time it gets, but the maximal CPU usage - 25%.
We try to query for all data or part of data - the same 25%
Now, we decrease the number of cores from 4 to 2
Now in the same queries, we have 50% CPU usage.
In the "Top" of the servers, I see that more than half of the time Elastic use 1 core(not all time)
Is it possible, that Elastic can use only one core in the frozen indices?
Maybe there is some CPU limit in the cluster/node configuration?
And again, we try to run two parallel discoveries and got the same results.
Thank a lot