How to setup and test node on different server?

Hello @asp,

The last message in your error log ( Node [rally-node-0] has terminated with exit code [78] ) refers to the elasticsearch node. In other words, the elasticsearch node is called rally-node-0.

The exit code 78 comes from Elasticsearch and points to a configuration error.

By investigating the elasticsearch logs under ~/.rally/benchmarks/races/<timestamp>/rally-node-0/logs/server/rally-benchmark.log[1] on the target host 10.0.2.4 we can identify the root cause preventing Elasticsearch from starting. It's possible this is due to bootstrap checks failing because vm.max_map_count didn't have a high enough value, but you should inspect your own logs to see the exact error. Details on configuring this can be found in the elasticsearch docs.

$ grep -R ERROR -A2 ~/.rally/benchmarks/races/*/rally-node-0/logs/server/rally-benchmark.log 
/home/dl/.rally/benchmarks/races/2018-03-15-08-57-46/rally-node-0/logs/server/rally-benchmark.log:[2018-03-15T08:58:07,774][ERROR][o.e.b.Bootstrap          ] [rally-node-0] node validation exception
/home/dl/.rally/benchmarks/races/2018-03-15-08-57-46/rally-node-0/logs/server/rally-benchmark.log-[1] bootstrap checks failed
/home/dl/.rally/benchmarks/races/2018-03-15-08-57-46/rally-node-0/logs/server/rally-benchmark.log-[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
--
/home/dl/.rally/benchmarks/races/2018-03-15-09-00-11/rally-node-0/logs/server/rally-benchmark.log:[2018-03-15T09:00:25,578][ERROR][o.e.b.Bootstrap          ] [rally-node-0] node validation exception
/home/dl/.rally/benchmarks/races/2018-03-15-09-00-11/rally-node-0/logs/server/rally-benchmark.log-[1] bootstrap checks failed
/home/dl/.rally/benchmarks/races/2018-03-15-09-00-11/rally-node-0/logs/server/rally-benchmark.log-[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[1] The elasticsearch errors can be seen in ~/.rally/rally-actor-messages.log in the target host as well.

Regards,
Dimitris

2 Likes