[Solved] Cannot start ES 7.0.1 on Fedora 30 unless "xpack.ml.enabled: false" is set

Solution: The solution was to install libxcrypt-compat (sudo dnf install libxcrypt-compat) as pointed out in the thread below. After installing that no further custom configuration was necessary and the service started without needing to set xpack.ml.enabled: false (i.e., default out-of-the-box config worked). Original Issue is below.

Just wondering if there are known issues with running ES 7.0.1 on Fedora 30? I was unable to start it until I set the xpack.ml.enabled: false in /etc/elasticsearch/elasticsearch.yml. Pretty much running the default settings aside from that. I installed it via the RPM package and am starting it with systemd. The journalctl output is below:

May 05 20:58:24 fedora-server elasticsearch[12275]: [2019-05-05T20:58:24,257][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [fedora-server] uncaught exception in thread [main]
May 05 20:58:24 fedora-server elasticsearch[12275]: org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-16369124385724695169/controller_log_12275 (No such fi>
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]: Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.node.Node.lambda$new$9(Node.java:438) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.node.Node.<init>(Node.java:441) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.node.Node.<init>(Node.java:251) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.0.1.jar:7.0.1]
May 05 20:58:24 fedora-server elasticsearch[12275]:         ... 6 more
May 05 20:58:24 fedora-server elasticsearch[12275]:         ... 6 more
May 05 20:58:24 fedora-server systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
May 05 20:58:24 fedora-server systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

Is it possible, that you are not allowed to write into the /tmp filesystem with the user that is starting Elasticsearch?

I actually thought that at first, but I tried specifying a custom tmp directory via ES_TMPDIR=/elasticsearch_tmp and ES_JAVA_OPTS="-Djna.tmpdir=/elasticsearch_tmp" in /etc/sysconfig/elasticsearch and gave the elasticsearch user and group ownership of it and did chmod 777 to the directory just to make sure there was not doubt it didn't have access. With this it still results in:

org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/elasticsearch_tmp/controller_log_4828 (No such file or directory)];

I also see a folder jna--1985354563 in that directory I created, so it is able to access it and create files/folders inside it. After that I reverted the changes to the tmp dir and proceeded to set xpack.ml.enabled: false to get it to work for now while I try and get this sorted.

you can try to create /elasticsearch_tmp/elasticsearch and give that directory all the permissions - this is IIRC where ML needs to have write permissions.

If that does not work either, please go ahead and open an issue in the Elasticsearch repo

This is likely elastic/ml-cpp#478. Please try installing the package libxcrypt-compat and let us know if the issue otherwise persists.

3 Likes

That's exactly what happened on my 6.8.0 with my brand new Fedora 30 install, installing libxcrypt-compat did the trick!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.