# Rally Track Report Analysis (follow up)

**URL:** <https://discuss.elastic.co/t/rally-track-report-analysis-follow-up/127266>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [April 9, 2018, 7:50am UTC](https://discuss.elastic.co/t/rally-track-report-analysis-follow-up/127266 "2018-04-09T07:50:17Z")\
**Posts on this page:** 1\
**Showing post:** 11

<div class="post-metadata">

**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\
**Post date:** [May 8, 2018, 8:48pm UTC](https://discuss.elastic.co/t/rally-track-report-analysis-follow-up/127266/11 "2018-05-08T20:48:39Z")

</div>

Hey @ize0j10,

I followed this thread and am also rather baffled about what's going on. I will recap my understanding below as a summary (please correct errors) and proceed with some suggestions.

My understanding is that you are using the latest released Rally (currently `0.10.1`) against a two node 6.3 Elasticsearch cluster.

Are you using a custom track or one of the [standard ones](https://github.com/elastic/rally-tracks)?

IIUC your race takes some time to complete and while the various operations complete successfully you are experiencing problems (read time outs) towards the end of the benchmark, when Rally collects index stats (and GC times).

If the track is a lengthy one, it would make sense to limit the amount of operations and see if the problem happens again.  
If you are using any of the standard Rally tracks, they support a number of options that can be passed using [--target-params](https://esrally.readthedocs.io/en/stable/command_line_reference.html#track-params); for example `geonames` supports `ingest_percentage` and `bulk_size`/`bulk_indexing_clients`. Reducing significantly the `ingest_percentage` (say to `1`) should index less documents and reduce the total running time of the benchmark.

If you are using a custom track (specified with `--track-path`) you could modify it to just run fewer operations and index less data (again with the `ingest-percentage` property of the [bulk](http://esrally.readthedocs.io/en/stable/track.html?highlight=ingest_percentage#bulk) operation).

Should the benchmark finish successfully we will have an indication that there is some strange network problem.

The elasticsearch python client (that Rally uses) uses [persistent connections](http://elasticsearch-py.readthedocs.io/en/master/index.html?highlight=persistent%20connections#persistent-connections) and there is a chance that something is terminating long living connections (firewall?). The client however supports `retry_on_timeout`; @danielmitterdorfer what do you think, would passing `retry_on_timeout=True` in `--client-options` be worth trying as this parameter will be used by the client\_factory used by telemetry devices, such as `IndexStats` and `GcTimesSummary`?

Regards,  
Dimitris

---

_[View the full topic](https://discuss.elastic.co/t/rally-track-report-analysis-follow-up/127266)._