esrally.exceptions.LaunchError: Daemon startup failed with exit code [1]

Elastic Stack Elasticsearch
1

I trying start:

esrally race --track-path=tracks/search_all --distribution-version=7.11.2 --car="4gheap" --preserve-install

But I got an error: esrally.exceptions.LaunchError: Daemon startup failed with exit code [1].
When I run elasticsearch from the .rally/races/{race}/install/elasticsearch-7.11.2 folder, it works and Elasticsearch starts successfully.

esrally version: 2.12.0

Log:

2025-06-02 11:15:13,494 -not-actor-/PID:20617 esrally.rally INFO OS [uname_result(system='Linux', node='n-sb-007', release='6.8.0-60-generic', version='#63~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 22 19:00:15 UTC 2', machine='x86_64')]
2025-06-02 11:15:13,494 -not-actor-/PID:20617 esrally.rally INFO Python [namespace(name='cpython', cache_tag='cpython-312', version=sys.version_info(major=3, minor=12, micro=8, releaselevel='final', serial=0), hexversion=51120368, _multiarch='x86_64-linux-gnu')]
2025-06-02 11:15:13,495 -not-actor-/PID:20617 esrally.rally INFO Rally version [2.12.0]
2025-06-02 11:15:13,495 -not-actor-/PID:20617 esrally.utils.net INFO Connecting directly to the Internet (no proxy support) for [all_proxy].
2025-06-02 11:15:13,495 -not-actor-/PID:20617 esrally.utils.net INFO Connecting directly to the Internet (no proxy support) for [all_proxy].
2025-06-02 11:15:13,495 -not-actor-/PID:20617 esrally.rally INFO Cleaning track dependency directory [/home/shuvalov/.rally/libs]...
2025-06-02 11:15:13,541 -not-actor-/PID:20617 esrally.rally INFO Actor system already running locally? [False]
2025-06-02 11:15:13,541 -not-actor-/PID:20617 esrally.actor INFO Starting actor system with system base [multiprocTCPBase] and capabilities [{'coordinator': True, 'ip': '127.0.0.1', 'Convention Address.IPv4': '127.0.0.1:1900'}].
2025-06-02 11:15:13,554 -not-actor-/PID:20622 root INFO ++++ Actor System gen (3, 10) started, admin @ ActorAddr-(T|:1900)
2025-06-02 11:15:13,562 -not-actor-/PID:20617 esrally.racecontrol INFO Race id is [d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5]
2025-06-02 11:15:13,563 -not-actor-/PID:20617 esrally.racecontrol INFO User specified no pipeline. Automatically derived pipeline [from-distribution].
2025-06-02 11:15:13,563 -not-actor-/PID:20617 esrally.racecontrol INFO Setting default host to [127.0.0.1:39200]
2025-06-02 11:15:13,572 ActorAddr-(T|:34717)/PID:20624 esrally.track.loader INFO Reading track specification file [some-path/tracks/search_all/track.json].
2025-06-02 11:15:13,953 ActorAddr-(T|:34717)/PID:20624 esrally.track.loader INFO Final rendered track for 'some-path/tracks/search_all/track.json' has been written to '/tmp/tmpqhi3yk2b.json'.
2025-06-02 11:15:13,965 ActorAddr-(T|:34717)/PID:20624 esrally.track.loader INFO Loading template [definition for index newsng_2016_03_v1 in newsng_2016_03_v1.json].
2025-06-02 11:15:13,974 ActorAddr-(T|:34717)/PID:20624 esrally.metrics INFO Creating in-memory metrics store
2025-06-02 11:15:13,974 ActorAddr-(T|:34717)/PID:20624 esrally.metrics INFO Opening metrics store for race timestamp=[20250602T111513Z], track=[search_all], challenge=[my-challenge], car=[['4gheap']]
2025-06-02 11:15:13,974 ActorAddr-(T|:34717)/PID:20624 esrally.metrics INFO Creating file race store
2025-06-02 11:15:13,974 ActorAddr-(T|:34717)/PID:20624 esrally.actor INFO Asking mechanic to start the engine.
2025-06-02 11:15:13,981 ActorAddr-(T|:40705)/PID:20636 esrally.actor INFO Received signal from race control to start engine.
2025-06-02 11:15:14,599 ActorAddr-(T|:40705)/PID:20636 esrally.utils.repo INFO Checking out [7.11] in [/home/shuvalov/.rally/benchmarks/teams/default] for distribution version [7.11.2].
2025-06-02 11:15:14,604 ActorAddr-(T|:40705)/PID:20636 esrally.utils.process INFO ΠŸΠ΅Ρ€Π΅ΠΊΠ»ΡŽΡ‡ΠΈΠ»ΠΈΡΡŒ Π½Π° Π²Π΅Ρ‚ΠΊΡƒ Β«7.11Β»
Π­Ρ‚Π° Π²Π΅Ρ‚ΠΊΠ° соотвСтствуСт Β«origin/7.11Β».

2025-06-02 11:15:14,604 ActorAddr-(T|:40705)/PID:20636 esrally.utils.repo INFO Rebasing on [7.11] in [/home/shuvalov/.rally/benchmarks/teams/default] for distribution version [7.11.2].
2025-06-02 11:15:14,611 ActorAddr-(T|:40705)/PID:20636 esrally.utils.process INFO Π£ΠΆΠ΅ Π½Π° Β«7.11Β»
Π­Ρ‚Π° Π²Π΅Ρ‚ΠΊΠ° соотвСтствуСт Β«origin/7.11Β».

2025-06-02 11:15:14,616 ActorAddr-(T|:40705)/PID:20636 esrally.utils.process INFO ВСкущая Π²Π΅Ρ‚ΠΊΠ° 7.11 ΡƒΠΆΠ΅ Π² Π°ΠΊΡ‚ΡƒΠ°Π»ΡŒΠ½ΠΎΠΌ состоянии.

2025-06-02 11:15:14,622 ActorAddr-(T|:40705)/PID:20636 esrally.actor INFO Cluster consisting of [{'host': '127.0.0.1', 'port': 39200}] will be provisioned by Rally.
2025-06-02 11:15:14,640 ActorAddr-(T|:41129)/PID:20683 esrally.actor INFO Starting node(s) [0] on [127.0.0.1].
2025-06-02 11:15:14,641 ActorAddr-(T|:41129)/PID:20683 esrally.metrics INFO Opening metrics store for race timestamp=[20250602T111513Z], track=[search_all], challenge=[my-challenge], car=[['4gheap']]
2025-06-02 11:15:15,204 ActorAddr-(T|:41129)/PID:20683 esrally.utils.repo INFO Checking out revision [d38ad85e8c1c636de254323a605a9d8d13c8d947] in [/home/shuvalov/.rally/benchmarks/teams/default].
2025-06-02 11:15:15,209 ActorAddr-(T|:41129)/PID:20683 esrally.utils.process INFO ΠŸΡ€ΠΈΠΌΠ΅Ρ‡Π°Π½ΠΈΠ΅: ΠΏΠ΅Ρ€Π΅ΠΊΠ»ΡŽΡ‡Π΅Π½ΠΈΠ΅ Π½Π° Β«d38ad85e8c1c636de254323a605a9d8d13c8d947Β».

Π’Ρ‹ сСйчас Π² состоянии «отсоСдинённого указатСля HEADΒ». ΠœΠΎΠΆΠ΅Ρ‚Π΅ ΠΎΡΠΌΠΎΡ‚Ρ€Π΅Ρ‚ΡŒΡΡ,
внСсти ΡΠΊΡΠΏΠ΅Ρ€ΠΈΠΌΠ΅Π½Ρ‚Π°Π»ΡŒΠ½Ρ‹Π΅ измСнСния ΠΈ Π·Π°Ρ„ΠΈΠΊΡΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ ΠΈΡ…, Ρ‚Π°ΠΊΠΆΠ΅ ΠΌΠΎΠΆΠ΅Ρ‚Π΅
ΠΎΡ‚ΠΌΠ΅Π½ΠΈΡ‚ΡŒ Π»ΡŽΠ±Ρ‹Π΅ ΠΊΠΎΠΌΠΌΠΈΡ‚Ρ‹, созданныС Π² этом состоянии, Π½Π΅ затрагивая Π΄Ρ€ΡƒΠ³ΠΈΠ΅
Π²Π΅Ρ‚ΠΊΠΈ, ΠΏΠ΅Ρ€Π΅ΠΊΠ»ΡŽΡ‡ΠΈΠ²ΡˆΠΈΡΡŒ ΠΎΠ±Ρ€Π°Ρ‚Π½ΠΎ Π½Π° Π»ΡŽΠ±ΡƒΡŽ Π²Π΅Ρ‚ΠΊΡƒ.

Если Ρ…ΠΎΡ‚ΠΈΡ‚Π΅ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ Π½ΠΎΠ²ΡƒΡŽ Π²Π΅Ρ‚ΠΊΡƒ для сохранСния созданных ΠΊΠΎΠΌΠΌΠΈΡ‚ΠΎΠ², ΠΌΠΎΠΆΠ΅Ρ‚Π΅
ΡΠ΄Π΅Π»Π°Ρ‚ΡŒ это (сСйчас ΠΈΠ»ΠΈ ΠΏΠΎΠ·ΠΆΠ΅), ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ ΠΊΠΎΠΌΠ°Π½Π΄Ρƒ switch с ΠΏΠ°Ρ€Π°ΠΌΠ΅Ρ‚Ρ€ΠΎΠΌ -c.
НапримСр:

  git switch -c <новая-Π²Π΅Ρ‚ΠΊΠ°>

Или ΠΎΡ‚ΠΌΠ΅Π½ΠΈΡ‚Π΅ эту ΠΎΠΏΠ΅Ρ€Π°Ρ†ΠΈΡŽ с ΠΏΠΎΠΌΠΎΡ‰ΡŒΡŽ:

  git switch -

ΠžΡ‚ΠΊΠ»ΡŽΡ‡ΠΈΡ‚Π΅ этот совСт, установив ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½ΡƒΡŽ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΡƒΡ€Π°Ρ†ΠΈΠΈ
advice.detachedHead Π² Π·Π½Π°Ρ‡Π΅Π½ΠΈΠ΅ false

HEAD сСйчас Π½Π° d38ad85 Update log4j configuration for 7.11

2025-06-02 11:15:24,484 ActorAddr-(T|:34717)/PID:20624 esrally.actor INFO Received a benchmark failure from [ActorAddr-(T|:40705)] and will forward it now.
2025-06-02 11:15:15,210 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.supplier INFO Enabling source artifact caching.
2025-06-02 11:15:24,485 -not-actor-/PID:20617 esrally.racecontrol ERROR A benchmark failure has occurred
2025-06-02 11:15:24,485 -not-actor-/PID:20617 esrally.racecontrol INFO Telling benchmark actor to exit.
2025-06-02 11:15:15,211 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.supplier INFO [/home/shuvalov/.rally/benchmarks/distributions/src] does not exist. Skipping pruning.
2025-06-02 11:15:15,211 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.java_resolver INFO Allowed JDK versions are [12, 11, 8].
2025-06-02 11:15:15,637 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.java_resolver INFO Detected JDK with major version [12] in [/usr/local/jdk-12.0.2+10].
2025-06-02 11:15:15,638 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.supplier INFO Resolved download URL [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.11.2-linux-x86_64.tar.gz?x-elastic-no-kpi=true] for version [7.11.2]
2025-06-02 11:15:24,486 ActorAddr-(T|:44223)/PID:20682 esrally.actor INFO mechanic.Dispatcher#receiveMessage unrecognized(msg = [<class 'thespian.actors.ActorExitRequest'>] sender = [ActorAddr-(T|:40705)])
2025-06-02 11:15:15,638 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.supplier INFO Skipping download for version [7.11.2]. Found an existing binary at [/home/shuvalov/.rally/benchmarks/distributions/elasticsearch-7.11.2-linux-x86_64.tar.gz].
2025-06-02 11:15:24,489 ActorAddr-(T|:44223)/PID:20682 esrally.actor INFO mechanic.Dispatcher#receiveMessage unrecognized(msg = [<class 'thespian.actors.ChildActorExited'>] sender = [ActorAddr-(T|:41129)])
2025-06-02 11:15:15,638 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Preparing candidate locally in [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install].
2025-06-02 11:15:24,490 ActorAddr-(T|:40705)/PID:20636 esrally.actor ERROR Child actor exited with [ChildActorExited:ActorAddr-(T|:44223)] while in status [starting].
2025-06-02 11:15:15,638 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Unzipping /home/shuvalov/.rally/benchmarks/distributions/elasticsearch-7.11.2-linux-x86_64.tar.gz to /home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install
2025-06-02 11:15:20,813 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Deleting pre-bundled Elasticsearch configuration at [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install/elasticsearch-7.11.2/config]
2025-06-02 11:15:20,813 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Reading config template file [/home/shuvalov/.rally/benchmarks/teams/default/cars/v1/vanilla/templates/config/log4j2.properties] and writing to [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install/elasticsearch-7.11.2/config/log4j2.properties].
2025-06-02 11:15:20,815 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Reading config template file [/home/shuvalov/.rally/benchmarks/teams/default/cars/v1/vanilla/templates/config/jvm.options] and writing to [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install/elasticsearch-7.11.2/config/jvm.options].
2025-06-02 11:15:20,820 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Reading config template file [/home/shuvalov/.rally/benchmarks/teams/default/cars/v1/vanilla/templates/config/elasticsearch.yml] and writing to [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install/elasticsearch-7.11.2/config/elasticsearch.yml].
2025-06-02 11:15:20,826 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.java_resolver INFO Allowed JDK versions are [12, 11, 8].
2025-06-02 11:15:21,180 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.java_resolver INFO Detected JDK with major version [12] in [/usr/local/jdk-12.0.2+10].
2025-06-02 11:15:21,180 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.launcher INFO Starting node [rally-node-0].
2025-06-02 11:15:24,478 ActorAddr-(T|:41129)/PID:20683 root ERROR Daemon startup failed with exit code [1]
2025-06-02 11:15:24,479 ActorAddr-(T|:41129)/PID:20683 esrally.actor ERROR Cannot process message [<esrally.mechanic.mechanic.StartNodes object at 0x74ab93f81cd0>]
Traceback (most recent call last):

  File "some-path/.venv/lib/python3.12/site-packages/esrally/mechanic/mechanic.py", line 595, in receiveMsg_StartNodes
    self.mechanic.start_engine()

  File "some-path/.venv/lib/python3.12/site-packages/esrally/mechanic/mechanic.py", line 724, in start_engine
    self.nodes = self.launcher.start(self.node_configs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "some-path/.venv/lib/python3.12/site-packages/esrally/mechanic/launcher.py", line 133, in start
    return [self._start_node(node_configuration, node_count_on_host) for node_configuration in node_configurations]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "some-path/.venv/lib/python3.12/site-packages/esrally/mechanic/launcher.py", line 163, in _start_node
    node_pid = self._start_process(binary_path, env)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "some-path/.venv/lib/python3.12/site-packages/esrally/mechanic/launcher.py", line 222, in _start_process
    raise exceptions.LaunchError(msg)

esrally.exceptions.LaunchError: Daemon startup failed with exit code [1]

2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.mechanic INFO Stopping nodes [].
2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.launcher INFO Shutting down [0] nodes on this host.
2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.metrics INFO Creating file race store
2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.mechanic WARNING Cannot store system metrics: No race with race id [d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5].
2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.metrics INFO Closing metrics store.
2025-06-02 11:15:24,488 ActorAddr-(T|:41129)/PID:20683 esrally.mechanic.provisioner INFO Preserving benchmark candidate installation at [/home/shuvalov/.rally/benchmarks/races/d3e233a4-ce74-48ba-9d0c-aa6b198dcfc5/rally-node-0/install/elasticsearch-7.11.2].
2025-06-02 11:15:27,486 -not-actor-/PID:20617 esrally.rally INFO Attempting to shutdown internal actor system.
2025-06-02 11:15:27,487 -not-actor-/PID:20623 root INFO ActorSystem Logging Shutdown
2025-06-02 11:15:27,508 -not-actor-/PID:20622 root INFO ---- Actor System shutdown
2025-06-02 11:15:27,508 -not-actor-/PID:20617 esrally.rally INFO Actor system is still running. Waiting...
2025-06-02 11:15:28,508 -not-actor-/PID:20617 esrally.rally INFO Shutdown completed.
2025-06-02 11:15:28,509 -not-actor-/PID:20617 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
  File "some-path/.venv/lib/python3.12/site-packages/esrally/rally.py", line 1195, in dispatch_sub_command
    race(cfg, args.kill_running_processes)
  File "some-path/.venv/lib/python3.12/site-packages/esrally/rally.py", line 942, in race
    with_actor_system(racecontrol.run, cfg)
  File "some-path/.venv/lib/python3.12/site-packages/esrally/rally.py", line 972, in with_actor_system
    runnable(cfg)
  File "some-path/.venv/lib/python3.12/site-packages/esrally/racecontrol.py", line 408, in run
    raise e
  File "some-path/.venv/lib/python3.12/site-packages/esrally/racecontrol.py", line 405, in run
    pipeline(cfg)
  File "some-path/.venv/lib/python3.12/site-packages/esrally/racecontrol.py", line 74, in __call__
    self.target(cfg)
  File "some-path/.venv/lib/python3.12/site-packages/esrally/racecontrol.py", line 337, in from_distribution
    return race(cfg, distribution=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "some-path/.venv/lib/python3.12/site-packages/esrally/racecontrol.py", line 302, in race
    raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: Daemon startup failed with exit code [1]
2

Resolve with --runtime-jdk="bundled"

1 Like