Esrally failed to show result (Cannot race. Load generator [14] has exited prematurely.)

Elastic Stack Elasticsearch
1

All tasks shows done 100%, but failed to show the result. Is it the memory problem? It happens when run large number of iterations to build index.
Parameters: bulk_indexing_clients:32,iterations:250000

The rally log show force-merge failed, will it caused the issue?

Running log:

[INFO] Racing on track [email_track], challenge [route-part-body-append-only] and car ['external'] with version [7.4.2].

[WARNING] merges_total_time is 346974 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] merges_total_throttled_time is 77 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] indexing_total_time is 153967 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 356965 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] flush_total_time is 5098 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index-route-part-body-template                                  [100% done]
Running create-index-route-part-body-template                                  [100% done]
Running check-cluster-health                                                   [100% done]
Running route-part-body-email-append                                           [100% done]
Running force-merge                                                            [100% done]
[ERROR] Cannot race. Load generator [14] has exited prematurely.

Getting further help:
*********************
* Check the log files in /home/jinlian/.rally/logs for errors.
* Read the documentation at https://esrally.readthedocs.io/en/1.3.0/
* Ask a question on the forum at https://discuss.elastic.co/c/elasticsearch/rally
* Raise an issue at https://github.com/elastic/rally/issues and include the log files in /home/jinlian/.rally/logs.

-----------------------------------
[INFO] FAILURE (took 26049 seconds)
-----------------------------------

Rally log: rally.log

2

Hi,

Checking the log file attached, we see that after issuing:

POST http://xxxxxx:80/_all/_forcemerge

the elasticsearch client that Rally uses received:

http.client.RemoteDisconnected: Remote end closed connection without response

indicating that something very bad happened to Elasticsearch after this operation; was Elasticsearch still operational after force-merge? You could also check the Elasticsearch logs.

3

After the races, the elasticsearch works fine, I can access it through the kibana, and other races work fine.And if I set iterations to 100000 it work fine too.
I can't check the log now because the docker have closed.
I saw the doc said the report do not include force_merge operation. I'll try do race without force_merge.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.