HELP ! Upgrade ES 6.7.0 to 7.1.0

Elastic Stack Elasticsearch
21 
[2019-06-18T14:45:27,595][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [eta10] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: WriteStateException[failed to write state to the first location tmp file /var/lib/elasticsearch-7/nodes/0/node-63.st.tmp]; nested: AccessDeniedException[/var/lib/elasticsearch-7/nodes/0/_state/node-63.st.tmp];
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]
Caused by: org.elasticsearch.ElasticsearchException: failed to bind service
        at org.elasticsearch.node.Node.<init>(Node.java:582) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]
        ... 6 more
Caused by: org.elasticsearch.gateway.WriteStateException: failed to write state to the first location tmp file /var/lib/elasticsearch-7/nodes/0/node-63.st.tmp
        at org.elasticsearch.gateway.MetaDataStateFormat.writeStateToFirstLocation(MetaDataStateFormat.java:127) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.gateway.MetaDataStateFormat.write(MetaDataStateFormat.java:243) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.gateway.MetaDataStateFormat.writeAndCleanup(MetaDataStateFormat.java:185) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.env.NodeEnvironment.loadOrCreateNodeMetaData(NodeEnvironment.java:411) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:302) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch-7/nodes/0/_state/node-63.st.tmp
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215) ~[?:?]
        at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478) ~[?:?]
        at java.nio.file.Files.newOutputStream(Files.java:219) ~[?:?]
        at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:411) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]

Some part of my error code. No idea how should I do after I have changed "sudo chmod a+rwx" to the files and dir.

22

Check the file permissions / file owners again. Command you issued "sudo chmod a+rwx" is not secure and is not recursive.

From your log
Elasticsearch cannot write data to

/var/lib/elasticsearch-7/nodes/0/node-63.st.tmp

You know problem.. More about permissions on Linux OS you can read there.

23

Hi Vasek, is there a way i can make changes to all required files and directory that are required by ES? Can share the command ? thanks

24

I have already posted it: HELP ! Upgrade ES 6.7.0 to 7.1.0

But be aware of name of directories. It may vary.
e.g.:

/var/lib/elasticsearch != /var/lib/elasticsearch-7
25

I have tried this. but is not working. Are you referring to this command ?

26

Please, read your logs.

Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch-7/nodes/0/_state/node-63.st.tmp

I don't see /var/lib/elasticsearch but there is /var/lib/elasticsearch-7.
See HELP ! Upgrade ES 6.7.0 to 7.1.0

27

Hi yes, this is what I try to do so that I wont confused with the old and new folders to refers to. Or are you trying to tell me I cannot use folders with other name than elasticsearch ?

Regarding the logs, I have granting full access to everyone to access that, but this is still throwing me error.

28

-rw-rw-r-- 1 elasticsearch elasticsearch 0 Jun 14 15:05 wazuh-es_index_search_slowlog.json

I realized changing the ownership does not really help. with this chown -R.

29

Hi all, I have the following users in my server now. These are the users which will be involved in this running of Elasticsearch.

  • Root (creation of system files)
  • Luke account ( running / executing commands to run ES) ID: 1000
  • Elasticsearch account (Owner of the elasticsearch folders and files, but this account does not have a physical /home space. ) ID: 121

May I know in this scenario, what rights or access should I give to the above?

30

Dear all, when I try to run : systemctl start elasticsearch, the following msg triggers me to think. Where is this file and I cant find it.

ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet

I do have this valid file and directory : /usr/share/elasticsearch/bin/elasticsearch
but there is no "elasticsearch.pid" on the server. May I know how do I resolve this? Do I need to create this .pid file ?

31 
[2019-06-19T16:49:28,710][INFO ][o.e.e.NodeEnvironment    ] [eta10] using [1] data paths, mounts [[/ (/dev/mapper/vg--root-lv--root)]], net usable_space [1.9tb], net total_space [2tb], types [ext4]

I notice there is a line in the log stating the above, may I know what does it refers? Does the mapping to the root directory? Can I change it since I am not using the root account to run the ES.

32

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.