Throughput "docs/s" vs "ops/s"?

[Chris1234]

I've been running Rally (which is great thank you!) on clusters to test different setups. I've been watching the Throughput results as a rough "This is faster/slower" indicator. It's reported in "docs/s" for all the runs.

However, my most recent execution (with no Rally setting changed and the same benchmark-only test) reported throughput in "ops/s" Why would this unit change between runs? Did I accidentally fat-finger a setting and not realize it?

[danielmitterdorfer]

Hi Chris,

you definitely did not fat-finger anything.

Rally defines the unit per operation-type (bulk, query, force-merge, etc.). The default is "ops" which is turned into "ops/s" in the command line report, i.e. how many requests have been send. For example, the bulk-operation overrides this default with "docs" which ends up as "docs/s" in the command line report. The reason why it does this, is that the number of documents that have been ingested per second is a better indicator than the number of requests that it has sent.

Hope that helps.

Daniel

[Chris1234]

I suppose then my question is why does this one report return "ops/s" when the other test runs returned "docs/s". (I've run six tests thus far.) All tests have been run on different setups but the exact same command-line:

esrally --pipeline=benchmark-only --target-host=10.1.2.181:9200 --track=geonames --challenge=append-no-conflicts-index-only --on-error=continue --race-id=${RACE_ID}

Making it even more confusing, I ran the exact same test again on the same setup and THIS time it returned "docs/s" like all the others.

[danielmitterdorfer]

Hi,

I can imagine a scenario where it is indeed possible that you get "ops/s" as unit even though it should be something else, e.g. "docs/s". Is it possible that the error rate for Rally invocations where you got "ops/s" was greater than zero? Regardless of what unit you see though, the number should always be interpreted as "docs/s".

Daniel