Does Rally support benchmark test to multi-instance elasticsearch in single node?

[Qinghe12]

Hi,I want to start on 2 elasticseach instances（ES0 and ES1） in a single node with different port，and use 2 esrally ( esrally0 and esrally1) to benchmark these 2 elasticseach instances.

But I find esrally1 cannot start benchmark after esrally0 has start-up benchmark. At last，esrally0 benchamrk to ES0 runs well with normal output-log while esrally2 stucks(whithout any run log), and CPU cores bind to ES0 and ES1 run in 100% load.

By the way, I do configure different cluster name and node name in 2 elsaticsearch instances.

So I want to know is there any problems to prevent this multi-instance benchmark layout ？

Thanks for any suggestion ！

[Bradley_Deam]

Hi Qinghe12,

It's not clear to me from your post what your ultimate goal here, but as far as I can tell you want to benchmark two seperate single-node Elasticsearch clusters in parallel? Can you explain your reasoning for this? There may be a better way to achieve the same end-goal without needing to launch two instances of Rally on the same machine.

Rally works by launching subprocesses of itself (1 python process per available core), so you cannot run multiple instances of Rally within the same process namespace.

It would help a lot if you could post the command(s) you're using to launch Rally, as well as the logs.

[Qinghe12]

Thanks a lot for your reply.
For the reason：I want to evaluate the overall throughput in a SMP physical machine with 16 cores(core0-core15). ES0(elasticsarch instance-0：bind to core0-core3) and ES1 (elasticsarch instance-1：bind to core4-core7) run as two seperate process with different port.

Rally1（bind to core8-core11） and Rally2 （bind to core12-core15）were used to benchmark the two elasticsearch seperatly with commands following.

Rally1 conmand:
esrally race --track=geonames --target-hosts=localhost:9200
--pipeline=benchmark-only
--offline --on-error=abort
--challenge=append-no-conflicts

Rally2 conmand:
esrally race --track=geonames --target-hosts=localhost:9200
--pipeline=benchmark-only
--offline --on-error=abort
--challenge=append-no-conflicts

It's expected that I can get two “Max Throughput” metric result log , while not.

Maybe I have to evaluate it in two VM only？

thanks again ...

[Bradley_Deam]

For the reason：I want to evaluate the overall throughput in a SMP physical machine with 16 cores(core0-core15). ES0(elasticsarch instance-0：bind to core0-core3) and ES1 (elasticsarch instance-1：bind to core4-core7) run as two seperate process with different port.

Typically we recommend users do not run any load generation tool (Rally included) on the same machine as the target application, even if you're pinning CPUs there's still many other shared resources on the same machine that might end up skewing the end results.

Maybe I have to evaluate it in two VM only？

I mentioned earlier that in order to run two seperate, concurrent instances of Rally they'll need to be in seperate process namespaces - i.e. each Rally shouldn't be able to see the other's processes. However, this isn't something we test with or an approach that we use at all, so your mileage may vary.

I strongly recommend launching Rally(s) on seperate machines in order to keep some level of isolation from each other and the systems under test. This presentation covers a lot of the common pitfalls when benchmarking Elasticsearch.

[Qinghe12]

OK, get it，thanks for reply！

I can launch two rally in two different machines to benchmark two different ES instances.

Another question, Is there any way to run two tracks task in a same rally process to benchmark the two ES instances in parallel?

[Bradley_Deam]

Another question, Is there any way to run two tracks task in a same rally process to benchmark the two ES instances in parallel?

Rally can't run two tracks at once, no. You can have Rally target multiple hosts at the same time via the --target-hosts argument, but they should be part of the same cluster.