Huge Difference in the throughput for index

[amitsa (AMIT KUMAR)]

I am running es-cluster on kubernetes on aws 14i_16xlarge instance.
These below pods are running on different aws instances.

track :- nyc_taxis

1 master
1 data nodes
1 esrally client

i am trying to calculate index througput only.

i see difference in result when using --include-tasks="index"

for example:-

without --include-tasks="index"

throughput for index :- 220000 docs/sec

with --include-tasks="index"

throughput for index :- 920000 docs/sec

[Bradley_Deam]

Hi @amitsa -

The --include-tasks argument means you'll only ever execute tasks of type index. Without seeing your esrally invocation or the results summary, it's difficult to reason about what might be happening.

That said, this seems a little like an XY problem. Unless your production dataset looks like that of the NYC Taxis (mappings, fields etc.), then the indexing throughput numbers are likely to be unrealistic, and in some cases completely invalid.

If you're trying to ascertain which instance types provide the best cost performance for your cluster, then it's imperative that you spend the time to model something akin to your production workload to ensure that any benchmarks are at least somewhat representative of what your cluster may need to handle once in production.

You can do this by creating your own track:

• Define Custom Workloads: Tracks — Rally 2.5.0 documentation
• (Note: You can create a track from existing data with the `create-track` subcommand`)

[amitsa (AMIT KUMAR)]

Hi @Bradley_Deam

small correction in the values

without --include-tasks="index"

throughput for index :- 220000 docs/sec

with --include-tasks="index"

throughput for index :- 92000 docs/sec

I haven seen similar differences on Bare Metal platform while performing the test.
I am not looking for any particular production datasets matching to NYC_Taxis track.
I am just checking out performance on different platforms with nyc_taxis.

whole execution takes time so i was only looking into indexing performance so opted out the flag --include-tasks but i see difference in the throughput .

I don't see any error and issue in logs.