# Esrally error - Could not initialize internal actor system **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [July 13, 2020, 2:21pm UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993 "2020-07-13T14:21:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![pro.shivansh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pro.shivansh/32/71970_2.png) [@pro.shivansh](https://discuss.elastic.co/u/pro.shivansh)\ **Post date:** [July 13, 2020, 2:21pm UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993/1 "2020-07-13T14:21:03Z")
Hello everyone, I was running a benchmark-only for a cluster and ran into an error as described below. I would really appreciate it if someone can help me figure out why is this error occurring and a possible resolution. I am using pyenv to manage my version of python, I used python 3.6.8 and finally 3.8.2 after noticing the requirement on the esrally install page. Got the same error in both cases. I was running the following command - ```auto esrally --track-path= --target-hosts= --pipeline=benchmark-only --client-options="basic_auth_user:'',basic_auth_password:'',verify_certs:false" ``` I ran the command twice, the second time after it had failed so the errors after **2020-07-10 15:24:05,567** would be related to the time the command was ran the second time. These are the resulting logs - ```auto 2020-07-10 15:23:41,535 -not-actor-/PID:51873 esrally.rally INFO Race id [aaaaa-bbbb-cccc-dddd-eeeeeeee] 2020-07-10 15:23:41,535 -not-actor-/PID:51873 esrally.rally INFO OS [uname_result(system='Darwin', node='xxxxyyyy', release='19.3.0', version='Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64', machine='x86_64', processor='i386')] 2020-07-10 15:23:41,535 -not-actor-/PID:51873 esrally.rally INFO Python [namespace(_multiarch='darwin', cache_tag='cpython-38', hexversion=50856688, name='cpython', version=sys.version_info(major=3, minor=8, micro=2, releaselevel='final', serial=0))] 2020-07-10 15:23:41,535 -not-actor-/PID:51873 esrally.rally INFO Rally version [2.0.0] 2020-07-10 15:23:41,536 -not-actor-/PID:51873 esrally.utils.net INFO Connecting directly to the Internet (no proxy support). 2020-07-10 15:23:42,234 -not-actor-/PID:51873 esrally.rally INFO Detected a working Internet connection. 2020-07-10 15:23:42,301 -not-actor-/PID:51873 esrally.rally INFO Actor system already running locally? [False] 2020-07-10 15:23:42,301 -not-actor-/PID:51873 esrally.actor INFO Starting actor system with system base [multiprocTCPBase] and capabilities [{'coordinator': True, 'ip': 'xxxx', 'Convention Address.IPv4': 'xxxx'}]. 2020-07-10 15:23:42,388 -not-actor-/PID:51873 root WARNING Unable to get address info for address xxxxyyyy (AddressFamily.AF_INET, SocketKind.SOCK_DGRAM, 17, 0): [Errno 8] nodename nor servname provided, or not known 2020-07-10 15:23:42,390 -not-actor-/PID:51873 root WARNING Unable to get address info for address xxxxyyyy (AddressFamily.AF_INET, SocketKind.SOCK_DGRAM, 17, 0): [Errno 8] nodename nor servname provided, or not known 2020-07-10 15:23:42,392 -not-actor-/PID:51873 root WARNING Unable to get address info for address xxxxyyyy (AddressFamily.AF_INET, SocketKind.SOCK_DGRAM, 17, AddressInfo.AI_PASSIVE): [Errno 8] nodename nor servname provided, or not known 2020-07-10 15:23:42,394 -not-actor-/PID:51873 root WARNING Unable to get address info for address xxxxyyyy (AddressFamily.AF_INET, SocketKind.SOCK_DGRAM, 17, AddressInfo.AI_PASSIVE): [Errno 8] nodename nor servname provided, or not known 2020-07-10 15:23:42,410 -not-actor-/PID:51877 root INFO ++++ Actor System gen (3, 10) started, admin @ ActorAddr-(T|:1900) 2020-07-10 15:23:47,406 -not-actor-/PID:51873 esrally.actor ERROR Could not initialize internal actor system. Traceback (most recent call last): File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/actor.py", line 248, in bootstrap_actor_system return thespian.actors.ActorSystem(system_base, File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/actors.py", line 635, in __init__ systemBase = self._startupActorSys( File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/actors.py", line 676, in _startupActorSys systemBase = sbc(self, logDefs=logDefs) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/multiprocTCPBase.py", line 28, in __init__ super(ActorSystemBase, self). __init__ (system, logDefs) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/multiprocCommon.py", line 86, in __init__ super(multiprocessCommon, self). __init__ (system, logDefs) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/systemBase.py", line 326, in __init__ self._startAdmin(self.adminAddr, File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/multiprocCommon.py", line 114, in _startAdmin raise InvalidActorAddress(adminAddr, thespian.actors.InvalidActorAddress: ActorAddr-(T|:1900) is not a valid ActorSystem admin 2020-07-10 15:23:47,414 -not-actor-/PID:51873 esrally.rally INFO Falling back to offline actor system. 2020-07-10 15:23:47,415 -not-actor-/PID:51873 esrally.actor INFO Joining already running actor system with system base [multiprocQueueBase]. 2020-07-10 15:23:47,438 -not-actor-/PID:51880 root INFO ++++ Actor System gen (3, 10) started, admin @ ActorAddr-Q.ThespianQ 2020-07-10 15:23:47,460 -not-actor-/PID:51873 esrally.racecontrol INFO User specified pipeline [benchmark-only]. 2020-07-10 15:23:47,460 -not-actor-/PID:51873 esrally.racecontrol INFO Using configured hosts [{'host': 'xxxx', 'port': xxxx, 'use_ssl': True}] 2020-07-10 15:23:47,461 -not-actor-/PID:51873 esrally.actor INFO Joining already running actor system with system base [multiprocQueueBase]. 2020-07-10 15:23:50,474 -not-actor-/PID:51873 esrally.rally INFO Attempting to shutdown internal actor system. 2020-07-10 15:23:50,500 -not-actor-/PID:51880 root INFO ---- Actor System shutdown 2020-07-10 15:23:50,502 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:51,506 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:52,512 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:53,517 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:54,521 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:55,524 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:56,529 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:57,533 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:58,537 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:23:59,541 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:00,547 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:01,552 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:02,556 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:03,561 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:04,564 -not-actor-/PID:51873 esrally.rally INFO Actor system is still running. Waiting... 2020-07-10 15:24:05,567 -not-actor-/PID:51873 esrally.rally WARNING Shutdown timed out. Actor system is still running. 2020-07-10 15:24:05,568 -not-actor-/PID:51873 esrally.rally ERROR Cannot run subcommand [race]. Traceback (most recent call last): File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 362, in run pipeline(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 60, in __call__ self.target(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 298, in benchmark_only return race(cfg, external=True) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 241, in race benchmark_actor = actor_system.createActor(BenchmarkActor, targetActorRequirements={"coordinator": True}) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/actors.py", line 702, in createActor return self._systemBase.newPrimaryActor(actorClass, File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/systemBase.py", line 205, in newPrimaryActor raise NoCompatibleSystemForActor( thespian.actors.NoCompatibleSystemForActor: No compatible ActorSystem could be found for Actor During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/rally.py", line 766, in dispatch_sub_command race(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/rally.py", line 676, in race with_actor_system(racecontrol.run, cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/rally.py", line 704, in with_actor_system runnable(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 370, in run raise exceptions.RallyError("This race ended with a fatal crash.").with_traceback(tb) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 362, in run pipeline(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 60, in __call__ self.target(cfg) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 298, in benchmark_only return race(cfg, external=True) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/esrally/racecontrol.py", line 241, in race benchmark_actor = actor_system.createActor(BenchmarkActor, targetActorRequirements={"coordinator": True}) File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/actors.py", line 702, in createActor return self._systemBase.newPrimaryActor(actorClass, File "/Users/xxxx/temp/.venv/lib/python3.8/site-packages/thespian/system/systemBase.py", line 205, in newPrimaryActor raise NoCompatibleSystemForActor( esrally.exceptions.RallyError: ('This race ended with a fatal crash.', None) ``` I am very new to this forum, please excuse me if I missed something or my format is messed up. Suggestions are always welcome. ---
**Author:** ![Evgenia\_Badiyanova](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgenia_badiyanova/32/84068_2.png) [@Evgenia\_Badiyanova](https://discuss.elastic.co/u/Evgenia_Badiyanova)\ **Post date:** [July 14, 2020, 3:02pm UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993/2 "2020-07-14T15:02:53Z")
Hello Shivansh, It looks like you issue begins with: > [@pro.shivansh](#): > > `root WARNING Unable to get address info for address xxxxyyyy (AddressFamily.AF_INET, SocketKind.SOCK_DGRAM, 17, 0): [Errno 8] nodename nor servname provided, or not known` Looks like actor system can't resolve hostname on you local machine. You may need to add it to /etc/hosts. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 15, 2020, 6:12am UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993/3 "2020-07-15T06:12:51Z")
Hi, this might as well be because your load driver is connected to the Elasticsearch cluster via a VPN and the actor system cannot derive the appropriate IP address. We have documented a [workaround](https://esrally.readthedocs.io/en/latest/rally_daemon.html#troubleshooting). However, we strongly advice that the load generator is network-wise very close to the cluster, ideally on a server located in the same rack. Otherwise, you might introduce an accidental bottleneck in your benchmark setup. See our blog post [Seven Tips for Better Elasticsearch Benchmarks](https://www.elastic.co/blog/seven-tips-for-better-elasticsearch-benchmarks) for more pitfalls to avoid. Daniel ---
**Author:** ![pro.shivansh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pro.shivansh/32/71970_2.png) [@pro.shivansh](https://discuss.elastic.co/u/pro.shivansh)\ **Post date:** [July 24, 2020, 1:35pm UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993/4 "2020-07-24T13:35:58Z")
Thank you for this insight Daniel, I was able turn off the VPN and get things moving finally. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 21, 2020, 1:36pm UTC](https://discuss.elastic.co/t/esrally-error-could-not-initialize-internal-actor-system/240993/5 "2020-08-21T13:36:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.