Error start elasticsearch in centos7

hello i found a problem when starting elasticsearch, and an error like this :

Jul 12 05:05:34 bigdata systemd-entrypoint[22275]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp
Jul 12 05:05:34 bigdata systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 12 05:05:34 bigdata systemd[1]: Failed to start Elasticsearch.
Jul 12 05:05:34 bigdata systemd[1]: Unit elasticsearch.service entered failed state.
Jul 12 05:05:34 bigdata systemd[1]: elasticsearch.service failed.

and this tail -f /var/log/elasticsearch/elasticsearch.log

[2021-07-12T03:15:14,441][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [bigdata] [controller/26724] [Main.cc@169] ML controller exiting
[2021-07-12T03:15:14,443][INFO ][o.e.x.m.p.NativeController] [bigdata] Native controller process has stopped - no new native processes can be started
[2021-07-12T03:15:14,620][INFO ][o.e.c.s.ClusterSettings  ] [bigdata] updating [xpack.monitoring.collection.enabled] from [false] to [true]
[2021-07-12T03:15:14,882][INFO ][o.e.l.LicenseService     ] [bigdata] license [1996a97d-c8bc-491c-a001-b0339a62d490] mode [basic] - valid
[2021-07-12T03:15:14,884][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [bigdata] Active license is now [BASIC]; Security is disabled
[2021-07-12T03:15:14,884][WARN ][o.e.x.s.s.SecurityStatusChangeListener] [bigdata] Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
[2021-07-12T03:15:14,889][INFO ][o.e.g.GatewayService     ] [bigdata] recovered [40] indices into cluster_state
[2021-07-12T03:15:14,903][INFO ][o.e.n.Node               ] [bigdata] stopped
[2021-07-12T03:15:14,904][INFO ][o.e.n.Node               ] [bigdata] closing ...
[2021-07-12T03:15:14,915][INFO ][o.e.n.Node               ] [bigdata] closed

how to solve the error above?

can you share the full log instead of the above snippet? Nothing jumps out to me.

For testing purposes, can you disable machine learning via xpack.ml.enabled in the configuration and try again?

this is full error log

Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at java.base/java.nio.file.Files.newOutputStream(Files.java:224)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:484)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:253)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:228)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:343)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.cli.Command.main(Command.java:79)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
Jul 12 09:38:14 bigdata systemd-entrypoint[22633]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
Jul 12 09:38:14 bigdata systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 12 09:38:14 bigdata systemd[1]: Failed to start Elasticsearch.
-- Subject: Unit elasticsearch.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit elasticsearch.service has failed.
-- 
-- The result is failed.
Jul 12 09:38:14 bigdata systemd[1]: Unit elasticsearch.service entered failed state.
Jul 12 09:38:14 bigdata systemd[1]: elasticsearch.service failed.
Jul 12 09:38:14 bigdata polkitd[622]: Unregistered Authentication Agent for unix-process:22627:39434988 (system bus name :1.357, object path /org/freedesktop/PolicyKit1/
Jul 12 09:46:02 bigdata polkitd[622]: Registered Authentication Agent for unix-process:22835:39482037 (system bus name :1.358 [/usr/bin/pkttyagent --notify-fd 5 --fallba
Jul 12 09:46:02 bigdata systemd[1]: Started Kibana.
-- Subject: Unit kibana.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit kibana.service has finished starting up.
-- 
-- The start-up result is done.
Jul 12 09:46:02 bigdata polkitd[622]: Unregistered Authentication Agent for unix-process:22835:39482037 (system bus name :1.358, object path /org/freedesktop/PolicyKit1/
lines 1529-1566/1566 (END)

There must be more of that stack trace earlier in the log, that is needed as well..

this is /var/log/elasticsearch/elasticsearch.log

	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:217) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-7.13.3.jar:7.13.3]
		at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.13.3.jar:7.13.3]
		at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.13.3.jar:7.13.3]

and this error from journalctl -u elasticsearch

Jul 13 08:53:53 bigdata systemd[1]: Starting Elasticsearch...
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.elasticsearch.cli.UserException: unable 
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at java.base/java.nio.file.Files.newOutputStream(Files.java:224)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:484)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:253)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:228)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:343)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.cli.Command.main(Command.java:79)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
Jul 13 08:54:02 bigdata systemd-entrypoint[1403]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
Jul 13 08:54:03 bigdata systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 13 08:54:03 bigdata systemd[1]: Failed to start Elasticsearch.
Jul 13 08:54:03 bigdata systemd[1]: Unit elasticsearch.service entered failed state.
Jul 13 08:54:03 bigdata systemd[1]: elasticsearch.service failed.
Jul 13 09:57:43 bigdata systemd[1]: Starting Elasticsearch...
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.elasticsearch.cli.UserException: unable
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at java.base/java.nio.file.Files.newOutputStream(Files.java:224)
Jul 13 09:57:47 bigdata systemd-entrypoint[10474]: at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)

Can you share the permissions of /etc/elasticsearch/elasticsearch.keystore - if that is owned by root and not by the elasticsearch group that might be the issue (and also needs to be readable by the elasticsearch group).

What Elasticsearch version is this?

[root@bigdata elasticsearch]# ll
total 164
-rw-rw---- 1 root elasticsearch   199 Jul 13 14:07 elasticsearch.keystore
-rw-rw---- 1 root elasticsearch  2801 Jul 13 14:43 elasticsearch.yml
-rw-r--r-- 1 root           989 23111 Jun 11 14:35 hs_err_pid24963.log
-rw-r--r-- 1 root           989 23126 Jun 11 14:37 hs_err_pid25558.log
-rw-r--r-- 1 root           989 23077 Jun 11 14:39 hs_err_pid25688.log
-rw-r--r-- 1 root           989 23077 Jun 11 14:40 hs_err_pid26090.log
-rw-r--r-- 1 root           989 23105 Jun  9 13:58 hs_err_pid8475.log
-rw-rw---- 1 root elasticsearch  3182 Jul  2 12:13 jvm.options
drwxr-s--- 2 root elasticsearch     6 Jul  2 12:17 jvm.options.d
-rw-rw---- 1 root           989  3180 Jul 12 04:28 jvm.options.rpmsave
-rw-rw---- 1 root elasticsearch 18626 Jul  2 12:13 log4j2.properties
-rw-rw---- 1 root elasticsearch   473 Jul  2 12:13 role_mapping.yml
-rw-rw---- 1 root elasticsearch   197 Jul  2 12:13 roles.yml
-rw-rw---- 1 root elasticsearch     0 Jul  2 12:13 users
-rw-rw---- 1 root elasticsearch     0 Jul  2 12:13 users_roles

i use version 7.13.2

can you also share the permissions of the /etc/elasticsearch directory itself? Thank you!

you mean something like this

[root@bigdata elasticsearch]# pwd
/etc/elasticsearch
[root@bigdata elasticsearch]# ll -h
total 164K
-rw-rw---- 1 root elasticsearch  199 Jul 13 14:07 elasticsearch.keystore
-rw-rw---- 1 root elasticsearch 2,8K Jul 13 14:43 elasticsearch.yml
-rw-r--r-- 1 root           989  23K Jun 11 14:35 hs_err_pid24963.log
-rw-r--r-- 1 root           989  23K Jun 11 14:37 hs_err_pid25558.log
-rw-r--r-- 1 root           989  23K Jun 11 14:39 hs_err_pid25688.log
-rw-r--r-- 1 root           989  23K Jun 11 14:40 hs_err_pid26090.log
-rw-r--r-- 1 root           989  23K Jun  9 13:58 hs_err_pid8475.log
-rw-rw---- 1 root elasticsearch 3,2K Jul  2 12:13 jvm.options
drwxr-s--- 2 root elasticsearch    6 Jul  2 12:17 jvm.options.d
-rw-rw---- 1 root           989 3,2K Jul 12 04:28 jvm.options.rpmsave
-rw-rw---- 1 root elasticsearch  19K Jul  2 12:13 log4j2.properties
-rw-rw---- 1 root elasticsearch  473 Jul  2 12:13 role_mapping.yml
-rw-rw---- 1 root elasticsearch  197 Jul  2 12:13 roles.yml
-rw-rw---- 1 root elasticsearch    0 Jul  2 12:13 users
-rw-rw---- 1 root elasticsearch    0 Jul  2 12:13 users_roles

No more like ls -ld /etc/elasticsearch

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.