# Huge Difference in the throughput for index **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [June 28, 2022, 5:57am UTC](https://discuss.elastic.co/t/huge-difference-in-the-throughput-for-index/308320 "2022-06-28T05:57:20Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![amitsa](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@amitsa](https://discuss.elastic.co/u/amitsa)\ **Post date:** [June 28, 2022, 5:57am UTC](https://discuss.elastic.co/t/huge-difference-in-the-throughput-for-index/308320/1 "2022-06-28T05:57:20Z")
I am running es-cluster on kubernetes on aws 14i\_16xlarge instance. These below pods are running on different aws instances. track :- nyc\_taxis 1 master 1 data nodes 1 esrally client i am trying to calculate index througput only. i see difference in result when using --include-tasks="index" for example:- # without --include-tasks="index" throughput for index :- 220000 docs/sec # with --include-tasks="index" throughput for index :- 920000 docs/sec ---
**Author:** ![Bradley\_Deam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley_deam/32/52991_2.png) [@Bradley\_Deam](https://discuss.elastic.co/u/Bradley_Deam)\ **Post date:** [July 4, 2022, 1:01am UTC](https://discuss.elastic.co/t/huge-difference-in-the-throughput-for-index/308320/2 "2022-07-04T01:01:26Z")
Hi @amitsa - The[`--include-tasks` argument](https://esrally.readthedocs.io/en/stable/command_line_reference.html?highlight=include%20task#include-tasks) means you'll only ever execute tasks of type `index`. Without seeing your `esrally` invocation or the results summary, it's difficult to reason about what might be happening. That said, this seems a little [like an XY problem](https://en.wikipedia.org/wiki/XY_problem). Unless your production dataset looks like that of the NYC Taxis (mappings, fields etc.), then the indexing throughput numbers are likely to be unrealistic, and in some cases completely invalid. If you're trying to ascertain which instance types provide the best cost performance for your cluster, then it's imperative that you spend the time to model something akin to your production workload to ensure that any benchmarks are at least somewhat representative of what your cluster may need to handle once in production. You can do this by creating your own track: - [Define Custom Workloads: Tracks — Rally 2.5.0 documentation](https://esrally.readthedocs.io/en/stable/adding_tracks.html) - (Note: [You can create a track from existing data with the `create-track` subcommand`](https://esrally.readthedocs.io/en/stable/command_line_reference.html?highlight=include%20task#create-track)) ---
**Author:** ![amitsa](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@amitsa](https://discuss.elastic.co/u/amitsa)\ **Post date:** [July 5, 2022, 5:39am UTC](https://discuss.elastic.co/t/huge-difference-in-the-throughput-for-index/308320/3 "2022-07-05T05:39:15Z")
Hi @Bradley_Deam small correction in the values # without --include-tasks="index" throughput for index :- 220000 docs/sec # with --include-tasks="index" throughput for index :- 92000 docs/sec I haven seen similar differences on Bare Metal platform while performing the test. I am not looking for any particular production datasets matching to NYC\_Taxis track. I am just checking out performance on different platforms with nyc\_taxis. whole execution takes time so i was only looking into indexing performance so opted out the flag --include-tasks but i see difference in the throughput . I don't see any error and issue in logs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 2, 2022, 5:39am UTC](https://discuss.elastic.co/t/huge-difference-in-the-throughput-for-index/308320/4 "2022-08-02T05:39:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.