# The error rate is 100% **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [September 28, 2017, 12:45pm UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117 "2017-09-28T12:45:52Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![luolinsun](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@luolinsun](https://discuss.elastic.co/u/luolinsun)\ **Post date:** [September 28, 2017, 12:45pm UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/1 "2017-09-28T12:45:52Z")
I use the geonames and the challenge is append-no-conflicts-index-only. the command: ./esrally race --offline --track=geonames --pipeline=benchmark-only --target-hosts=10.202.7.169:9200,10.202.7.170:9200,10.202.7.171:9200 --challenge=append-no-conflicts-index-only But, the result about the index-append is: | All | Min Throughput | index-append | 927.67 | docs/s | | All | Median Throughput | index-append | 14590.5 | docs/s | | All | Max Throughput | index-append | 14890.2 | docs/s | | All | 50th percentile latency | index-append | 51.2863 | ms | | All | 90th percentile latency | index-append | 65.4431 | ms | | All | 99th percentile latency | index-append | 78.9863 | ms | | All | 99.9th percentile latency | index-append | 115.625 | ms | | All | 100th percentile latency | index-append | 452.019 | ms | | All | 50th percentile service time | index-append | 51.2863 | ms | | All | 90th percentile service time | index-append | 65.4431 | ms | | All | 99th percentile service time | index-append | 78.9863 | ms | | All | 99.9th percentile service time | index-append | 115.625 | ms | | All | 100th percentile service time | index-append | 452.019 | ms | | All | error rate | index-append | 100 | % | I don't know the why the error rate is 100% ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [September 28, 2017, 1:29pm UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/2 "2017-09-28T13:29:40Z")
Hi @luolinsun, this problem is very likely environment specific. Can you please share the corresponding log file (Rally prints the path to the log file at the beginning: "Writing logs to ~/.rally/logs/rally\_out\_SOME\_TIMESTAMP\_HERE.log") Can you please also try to run a small subset of just 1000 documents by specifying `--test-mode` in addition to the other command line parameters? Does it still report an error rate of 100%? Another test would be to run it against your local machine (for test purposes) with: ```auto ./esrally --offline --track=geonames --distribution-version=5.6.0 --challenge=append-no-conflicts-index-only ``` This requires that Java 8 is installed on your local machine (as Elasticsearch is started by Rally on the local machine in the background). Daniel ---
**Author:** ![luolinsun](https://avatars.discourse-cdn.com/v4/letter/l/8e8cbc/32.png) [@luolinsun](https://discuss.elastic.co/u/luolinsun)\ **Post date:** [September 29, 2017, 2:32am UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/3 "2017-09-29T02:32:23Z")
Thanks,I try a small subset and a local machine with es5.4, the error rate also is 100%. From the log I can't find the reason. There are some logs: 2017-09-29 01:50:29,956 PID:3849 rally.driver INFO Main driver has notified all load generators of termination. 2017-09-29 01:50:29,979 PID:3838 rally.racecontrol INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:36732)] (ignoring). 2017-09-29 01:50:29,980 PID:3839 rally.telemetry INFO Gathering indices stats. 2017-09-29 01:50:30,7 PID:3839 rally.metrics INFO Compression changed size of metric store from [264] bytes to [1264] bytes 2017-09-29 01:50:30,8 PID:3838 rally.racecontrol INFO Bulk adding system metrics to metrics store. 2017-09-29 01:50:30,8 PID:3838 rally.metrics INFO Restoring in-memory representation of metrics store. 2017-09-29 01:50:30,8 PID:3838 rally.racecontrol INFO Flushing metrics data... 2017-09-29 01:50:30,8 PID:3838 rally.racecontrol INFO Flushing done 2017-09-29 01:50:30,8 PID:3838 rally.racecontrol INFO Finished lap [1/1] 2017-09-29 01:50:30,9 PID:3838 rally.racecontrol INFO Asking mechanic to stop the engine. 2017-09-29 01:50:30,9 PID:3839 rally.actor INFO Transitioning from [benchmark\_stopped] to [cluster\_stopping]. 2017-09-29 01:50:30,10 PID:3844 rally.mechanic INFO Stopping nodes [\, \, \]. 2017-09-29 01:50:30,10 PID:3844 rally.metrics INFO Compression changed size of metric store from [64] bytes to [47] bytes 2017-09-29 01:50:30,10 PID:3839 rally.metrics INFO Restoring in-memory representation of metrics store. 2017-09-29 01:50:30,10 PID:3839 rally.actor INFO [1] of [1] child actors have responded for transition from [cluster\_stopping] to [cluster\_stopped]. 2017-09-29 01:50:30,10 PID:3839 rally.actor INFO All [1] child actors have responded. Transitioning now from [cluster\_stopping] to [cluster\_stopped]. 2017-09-29 01:50:30,11 PID:3839 rally.metrics INFO Compression changed size of metric store from [64] bytes to [47] bytes 2017-09-29 01:50:30,11 PID:3838 rally.racecontrol INFO Mechanic has stopped engine successfully. 2017-09-29 01:50:30,11 PID:3838 rally.racecontrol INFO Bulk adding system metrics to metrics store. 2017-09-29 01:50:30,11 PID:3838 rally.metrics INFO Restoring in-memory representation of metrics store. 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO Summarizing results. 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m------------------------------------------------------e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m \_\_\_\_\_\_\_ \_\_ \_\_\_\_\_ e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m / **_()_** \_\_\_\_ _/ / / **/** _\_\_\_\_\_ \_\_\_\_\_\_\_\_ e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m / /\_ / / \_\_ / \_\_ `/ / \_\_ / _/ \_\_ / / \_ \e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m / \_\_/ / / / / / // / / **/ / /** / // / / / \_\_/e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m// /// //\ __,_/_/ /\_\__ /\_ **/\_** /_/ \_\_\_/ e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO e[1m------------------------------------------------------e[0m 2017-09-29 01:50:30,23 PID:3838 rally.reporting INFO 2017-09-29 01:50:30,26 PID:3838 rally.reporting INFO | Lap | Metric | Operation | Value | Unit | |------:|-------------------------------:|-------------:|-----------:|-------😑 | All | Total Young Gen GC | | 0.272 | s | | All | Total Old Gen GC | | 0 | s | | All | Heap used for segments | | 0.420527 | MB | | All | Heap used for doc values | | 0.0634766 | MB | | All | Heap used for terms | | 0.330464 | MB | | All | Heap used for norms | | 0.00128174 | MB | | All | Heap used for points | | 0.00135517 | MB | | All | Heap used for stored fields | | 0.0239487 | MB | | All | Segment count | | 80 | | | All | Min Throughput | index-append | 1234.81 | docs/s | | All | Median Throughput | index-append | 11730.3 | docs/s | | All | Max Throughput | index-append | 13368.1 | docs/s | | All | 50th percentile latency | index-append | 51.3351 | ms | | All | 90th percentile latency | index-append | 67.9726 | ms | | All | 99th percentile latency | index-append | 98.6408 | ms | | All | 99.9th percentile latency | index-append | 687.572 | ms | | All | 100th percentile latency | index-append | 748.216 | ms | | All | 50th percentile service time | index-append | 51.3351 | ms | | All | 90th percentile service time | index-append | 67.9726 | ms | | All | 99th percentile service time | index-append | 98.6408 | ms | | All | 99.9th percentile service time | index-append | 687.572 | ms | | All | 100th percentile service time | index-append | 748.216 | ms | | All | error rate | index-append | 100 | % | | All | Min Throughput | force-merge | 114.74 | ops/s | | All | Median Throughput | force-merge | 114.74 | ops/s | | All | Max Throughput | force-merge | 114.74 | ops/s | | All | 100th percentile latency | force-merge | 8.69539 | ms | | All | 100th percentile service time | force-merge | 8.69539 | ms | | All | error rate | force-merge | 0 | % | 2017-09-29 01:50:30,28 PID:3838 rally.metrics INFO Closing metrics store. 2017-09-29 01:50:30,30 PID:3812 rally.racecontrol INFO Benchmark has finished successfully. 2017-09-29 01:50:30,30 PID:3812 rally.racecontrol INFO Telling benchmark actor to exit. 2017-09-29 01:50:30,30 PID:3838 rally.racecontrol INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:39664)] (ignoring). 2017-09-29 01:50:30,31 PID:3812 rally.main INFO Attempting to shutdown internal actor system. 2017-09-29 01:50:30,37 PID:3812 rally.main INFO Actor system is still running. Waiting... 2017-09-29 01:50:30,37 PID:3836 root INFO ---- Actor System shutdown 2017-09-29 01:50:31,39 PID:3812 rally.main INFO Shutdown completed. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [September 29, 2017, 8:38am UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/4 "2017-09-29T08:38:12Z")
Hi @luolinsun, that leaves two possibilities to me: (1) Either your data set is corrupted or (2) the machine does not have enough resources? Regarding (1): Can you please do `ls -l ~/.rally/benchmarks/data/geonames` and check whether you see the same file sizes? Rally should verify the file sizes but I just want to double-check: ```auto total 7445944 -rw-r--r-- 1 daniel staff 3547614383 Sep 29 10:38 documents-2.json -rw-r--r-- 1 daniel staff 264698741 Sep 29 10:37 documents-2.json.bz2 -rw-r--r-- 1 daniel staff 4250 Sep 29 10:39 documents-2.json.offset ``` If not, then please delete all files in that folder and rerun Rally. It should download and uncompress the files again (but not if you specify `--offline`). Regarding (2): - Are you on a machine with a spinning disk instead of an SSD? - Can you try to rerun the benchmark with an increased heap size, e.g. `--car=4gheap`? You could also check whether Elasticsearch logs in `~/.rally/benchmarks/races/YOUR_RACE_TIMESTAMP/rally-node-0/logs/server` reveal anything interesting. Daniel ---
**Author:** ![sanketshinde](https://avatars.discourse-cdn.com/v4/letter/s/b5ac83/32.png) [@sanketshinde](https://discuss.elastic.co/u/sanketshinde)\ **Post date:** [October 16, 2017, 3:02pm UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/5 "2017-10-16T15:02:00Z")
I am getting the same error: Like i posted in my post : [Premature end of Benchmark run](https://discuss.elastic.co/t/premature-end-of-benchmark-run/102684/5) Can't seem to figure this out. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 13, 2017, 3:02pm UTC](https://discuss.elastic.co/t/the-error-rate-is-100/102117/6 "2017-11-13T15:02:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.