# IO / Disc "tear down" for elastic search **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [September 9, 2020, 1:33pm UTC](https://discuss.elastic.co/t/io-disc-tear-down-for-elastic-search/248013 "2020-09-09T13:33:41Z")\ **Posts on this page:** 3\ **Page:** 3
**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\ **Post date:** [September 29, 2020, 8:13am UTC](https://discuss.elastic.co/t/io-disc-tear-down-for-elastic-search/248013/41 "2020-09-29T08:13:27Z")
> [@Oleg\_Ruchovets](#): > > esrally --track-params=./track-params.json --challenge=daily-log-volume-index-and-query --track=eventdata --track-repository=eventdata --pipeline=benchmark-only --target-hosts=MyIp:9200 This has the wrong challenge name. The challenge's name is shown [here](https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/challenges/daily-log-volume-index-and-query.json#L13) therefore you should use: `esrally --challenge=index-and-query-logs-fixed-daily-volume --track=eventdata --track-repository=eventdata --pipeline=benchmark-only --target-hosts=myip:myport` > [@Oleg\_Ruchovets](#): > > Is it possible to pass `id_seq_low_id_bias` to `true` and probability as a parameter. No it's not (at least not yet), but it's easy to make it parametrizable yourself by simply adding a few jinja2 variables as shown [here](https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/challenges/bulk-update.json#L30). > [@Oleg\_Ruchovets](#): > > May you please give me an example of how to deal with probability tweaks and what is the logic behind them. There are described briefly [here](https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/parameter_sources/elasticlogs_bulk_source.py#L56-L59) and [here](https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/parameter_sources/elasticlogs_bulk_source.py#L157-L166). If you set `id_seq_low_id_bias` to `true` it'll use a different distribution for picking the doc ids to update that heavily favors lower ids (by default it's the uniform distribution). As index size grows, updating the oldest ids creates a heavy workload. ---
**Author:** ![Oleg\_Ruchovets](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oleg_ruchovets/32/60137_2.png) [@Oleg\_Ruchovets](https://discuss.elastic.co/u/Oleg_Ruchovets)\ **Post date:** [October 13, 2020, 7:08pm UTC](https://discuss.elastic.co/t/io-disc-tear-down-for-elastic-search/248013/42 "2020-10-13T19:08:49Z")
Hello, I ran the: esrally --pipeline=benchmark-only --track=eventdata --track-repository=eventdata --challenge=bulk-update --track-params=bulk\_size:10000,bulk\_indexing\_clients:64 --target-hosts=localhost:9200 --client-options="timeout:240" --kill-running-processes After reach 58% of execution I've got an exception: \*\*and got \*\* \*\* java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519 it is actually a Lucene index limitation \*\* I ran the single node with a single index. The question, what is the option to overcome this problem? I run a number of benchmarks and I need the final result of Rally execution,elastic process is failed - rally is failed too and there is no output of the Rally execution. Also a couple of questions about specific rally parameters: I configured bulk\_size:10000, bulk\_indexing\_clients:64 looking at the **top** command I saw concurrently running max 15 rally processes? The question - how can I affect on higher load? is it process/core bound - in order to have running 64 rally processes I need at least 64 CPU cores machine? ``` bulk_size:10000 - making this parameter 50000 for example, will it increase the overall load on the write IO? Is there a way to generate a bigger document (in case of this challenge I saw JSON document is Nginx log which is not so big) ``` `id_seq_low_id_bias` to `true` - what are the steps - I mean if it is not supported parameters passing what file should I edit and does it affect on the command line to execute the challenge. I saw the example here: [https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/challenges/bulk-update.json#L30](https://github.com/elastic/rally-eventdata-track/blob/e23b994e69b7b3ff8b0d386aa3b9a0a37f3e597d/eventdata/challenges/bulk-update.json#L30) but how should I apply it to my local environment? is this the place to edit the file: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/1/31285e7930bd10c99959cee832532059480c0d43.png) what is the logic behind this challenge: how often updates happened? Is it just writes the first phase and after some threshold starts to update the documents? What is the best way to create/extend Rally challenges/plugins? Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 10, 2020, 7:08pm UTC](https://discuss.elastic.co/t/io-disc-tear-down-for-elastic-search/248013/43 "2020-11-10T19:08:59Z")
Closed on November 10, 2020, 7:08pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/io-disc-tear-down-for-elastic-search/248013.md?page=2)