Elasticsearch installation and configuration issue

Hello Elasticsearch forum,

My name is Bogdan Oproescu and I am senior data architect for EasyDO Digital Technologies in Bucharest. I am writing you today, to report an issue with our initial Elasticsearch engine configuration failure this morning, on our Linux CentOS server version 8.1. Here is the error message below:

Step 1: install Elasticsearch:
a. For CentOS we use rpm:
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.8.1-x86_64.rpm
b. sudo rpm -i elasticsearch-7.8.1-x86_64.rpm
2. There were dependency problems on our Linux server, with a previous version of Elastic installed on this server (now deprecated!):
a. view previous versions of Elasticsearch installed:
[root@ElasticEDT bin]# rpm -qa | grep elastic
elasticsearch-7.6.1-1.x86_64
b. remove previous versions of Elasticsearch:
[root@ElasticEDT bin]# rpm -e elasticsearch-7.6.1-1.x86_64
Stopping elasticsearch service... OK
warning: file /etc/elasticsearch/log4j2.properties: remove failed: No such file or directory
warning: /etc/elasticsearch/jvm.options saved as /etc/elasticsearch/jvm.options.rpmsave
warning: file /etc/elasticsearch/elasticsearch.yml: remove failed: No such file or directory
Deleting log directory... OK
[root@ElasticEDT bin]#
c. install the new version of Elasticsearch:
sudo rpm -i elasticsearch-7.8.1-x86_64.rpm
--> and the output is this:
[root@ElasticEDT bin]# sudo rpm -i elasticsearch-7.8.1-x86_64.rpm
Creating elasticsearch group... OK
Creating elasticsearch user... OK

NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service

You can start elasticsearch service by executing

sudo systemctl start elasticsearch.service
Created elasticsearch keystore in /etc/elasticsearch/elasticsearch.keystore
[/usr/lib/tmpfiles.d/elasticsearch.conf:1] Line references path below legacy directory /var/run/, updating /var/run/elasticsearch → /run/elasticsearch; please update the tmpfiles.d/ drop-in file accordingly.
[/usr/lib/tmpfiles.d/libgpod.conf:1] Line references path below legacy directory /var/run/, updating /var/run/libgpod → /run/libgpod; please update the tmpfiles.d/ drop-in file accordingly.
[/usr/lib/tmpfiles.d/mdadm.conf:1] Line references path below legacy directory /var/run/, updating /var/run/mdadm → /run/mdadm; please update the tmpfiles.d/ drop-in file accordingly.
[/usr/lib/tmpfiles.d/radvd.conf:1] Line references path below legacy directory /var/run/, updating /var/run/radvd → /run/radvd; please update the tmpfiles.d/ drop-in file accordingly.
[/usr/lib/tmpfiles.d/subscription-manager.conf:1] Line references path below legacy directory /var/run/, updating /var/run/rhsm → /run/rhsm; please update the tmpfiles.d/ drop-in file accordingly.
[root@ElasticEDT bin]#
3. Therefore our new version of Elasticsearch analytics engine was correctly installed on our ElasticEDT server!
a. Now we try to start this new version of Elasticsearch:
[root@ElasticEDT bin]# sudo service elasticsearch start
Starting elasticsearch (via systemctl): Job for elasticsearch.service failed because the control process exited with error code.
See "systemctl status elasticsearch.service" and "journalctl -xe" for details.
[FAILED]
b. To find out more, we can execute this command below:
[root@ElasticEDT elasticsearch]# systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2020-09-01 05:49:18 EEST; 3min 38s ago
Docs: https://www.elastic.co
Process: 2709 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 2709 (code=exited, status=1/FAILURE)

Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.cli.Command.main(Command.java:90)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
Sep 01 05:49:18 ElasticEDT systemd-entrypoint[2709]: For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
Sep 01 05:49:18 ElasticEDT systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 01 05:49:18 ElasticEDT systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Sep 01 05:49:18 ElasticEDT systemd[1]: Failed to start Elasticsearch.
[root@ElasticEDT elasticsearch]#

Can you share the content of the file /var/log/elasticsearch/elasticsearch.log

Hi Yassine,
Thanks for your prompt reply: I have checked earlier our CentOS 8.1 Linux Elastic server, and inside this folder /var/log/elasticsearch we only have these files below:
[root@ElasticEDT elasticsearch]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@ElasticEDT elasticsearch]# pwd
/var/log/elasticsearch
[root@ElasticEDT elasticsearch]# ll
total 48
-rw-r--r--. 1 elasticsearch elasticsearch 1356 Sep 1 22:04 gc.log
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:14 gc.log.00
-rw-r--r--. 1 elasticsearch elasticsearch 1331 Sep 1 05:14 gc.log.01
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:25 gc.log.02
-rw-r--r--. 1 elasticsearch elasticsearch 1344 Sep 1 05:25 gc.log.03
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:49 gc.log.04
-rw-r--r--. 1 elasticsearch elasticsearch 1343 Sep 1 05:49 gc.log.05
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:56 gc.log.06
-rw-r--r--. 1 elasticsearch elasticsearch 1344 Sep 1 05:56 gc.log.07
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:57 gc.log.08
-rw-r--r--. 1 elasticsearch elasticsearch 1343 Sep 1 05:57 gc.log.09
-rw-r--r--. 1 elasticsearch elasticsearch 907 Sep 1 22:04 gc.log.10
[root@ElasticEDT elasticsearch]#

so which one of these would you like me to send you?
regards, Bogdan

if we sort by modification time, newest first, we obtain this listing of files below:
[root@ElasticEDT elasticsearch]# pwd
/var/log/elasticsearch
[root@ElasticEDT elasticsearch]# ls -lt
total 48
-rw-r--r--. 1 elasticsearch elasticsearch 1356 Sep 1 22:04 gc.log
-rw-r--r--. 1 elasticsearch elasticsearch 907 Sep 1 22:04 gc.log.10
-rw-r--r--. 1 elasticsearch elasticsearch 1343 Sep 1 05:57 gc.log.09
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:57 gc.log.08
-rw-r--r--. 1 elasticsearch elasticsearch 1344 Sep 1 05:56 gc.log.07
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:56 gc.log.06
-rw-r--r--. 1 elasticsearch elasticsearch 1343 Sep 1 05:49 gc.log.05
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:49 gc.log.04
-rw-r--r--. 1 elasticsearch elasticsearch 1344 Sep 1 05:25 gc.log.03
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:25 gc.log.02
-rw-r--r--. 1 elasticsearch elasticsearch 1331 Sep 1 05:14 gc.log.01
-rw-r--r--. 1 elasticsearch elasticsearch 898 Sep 1 05:14 gc.log.00
[root@ElasticEDT elasticsearch]#

so it makes sense to send you the first 2 files above, as these seem to be periodically updated. Here you go:

  1. [root@ElasticEDT elasticsearch]# more gc.log
    [2020-09-01T19:04:47.333+0000][12495][gc,heap] Heap region size: 1M
    [2020-09-01T19:04:47.334+0000][12495][gc,heap,coops] Heap address: 0x00000000c0000000, size: 1024 MB, Compressed O
    ops mode: 32-bit
    [2020-09-01T19:04:47.583+0000][12495][gc ] Using G1
    [2020-09-01T19:04:47.584+0000][12495][gc,cds ] Mark closed archive regions in map: [0x00000000fff00000, 0x00
    000000fff7eff8]
    [2020-09-01T19:04:47.584+0000][12495][gc,cds ] Mark open archive regions in map: [0x00000000ffe00000, 0x0000
    0000ffe51ff8]
    [2020-09-01T19:04:47.595+0000][12495][gc ] Periodic GC disabled
    [2020-09-01T19:04:48.032+0000][12495][safepoint ] Safepoint "ICBufferFull", Time since last: 441088403 ns, Reac
    hing safepoint: 169098 ns, At safepoint: 15859 ns, Total: 184957 ns
    [2020-09-01T19:04:48.603+0000][12495][gc,heap,exit ] Heap
    [2020-09-01T19:04:48.603+0000][12495][gc,heap,exit ] garbage-first heap total 1048576K, used 47940K [0x00000000
    c0000000, 0x0000000100000000)
    [2020-09-01T19:04:48.603+0000][12495][gc,heap,exit ] region size 1024K, 46 young (47104K), 0 survivors (0K)
    [2020-09-01T19:04:48.603+0000][12495][gc,heap,exit ] Metaspace used 10816K, capacity 11304K, committed 1152
    0K, reserved 1058816K
    [2020-09-01T19:04:48.603+0000][12495][gc,heap,exit ] class space used 1349K, capacity 1500K, committed 1536K,
    reserved 1048576K
    [root@ElasticEDT elasticsearch]#
  2. and for the second file:
    [root@ElasticEDT elasticsearch]# more gc.log.10
    [2020-09-01T19:04:47.183+0000][12649][gc,heap] Heap region size: 1M
    [2020-09-01T19:04:47.183+0000][12649][gc,heap,coops] Heap address: 0x00000000c0000000, size: 1024 MB, Compressed O
    ops mode: 32-bit
    [2020-09-01T19:04:47.192+0000][12649][gc ] Using G1
    [2020-09-01T19:04:47.236+0000][12649][gc ] Periodic GC disabled
    [2020-09-01T19:04:47.263+0000][12649][gc,heap,exit ] Heap
    [2020-09-01T19:04:47.263+0000][12649][gc,heap,exit ] garbage-first heap total 1048576K, used 1024K [0x00000000c
    0000000, 0x0000000100000000)
    [2020-09-01T19:04:47.263+0000][12649][gc,heap,exit ] region size 1024K, 2 young (2048K), 0 survivors (0K)
    [2020-09-01T19:04:47.263+0000][12649][gc,heap,exit ] Metaspace used 3123K, capacity 4480K, committed 4480K,
    reserved 1056768K
    [2020-09-01T19:04:47.263+0000][12649][gc,heap,exit ] class space used 263K, capacity 384K, committed 384K, re
    served 1048576K
    [root@ElasticEDT elasticsearch]#

I hope this helps, to clarify what the problem is, and why Elasticsearch analytics engine does not start for us!
Greetings from Bucharest, Bogdan

Can you try runnung elasticsearch manually to see what happens ?
Use this : /usr/share/elasticsearch/bin/elasticsearch

When your install RPM package, ES is installed in :

  • /etc/elasticsearch (config file),
  • /usr/share/elasticsearch (binaries)

hi Yassine,
thanks for your reply earlier, and your explanation that ES is installed in:

  • /etc/elasticsearch (config file),
  • /usr/share/elasticsearch (binaries)
    I checked for myself and I'm happy to confirm this is so, the installation makes more sense to me now!

here is the output when trying to run Elasticsearch manually, as you suggested:
[root@ElasticEDT bin]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@ElasticEDT bin]# pwd
/usr/share/elasticsearch/bin
[root@ElasticEDT bin]# ./elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2020-09-02T18:34:28,284][WARN ][o.e.c.l.LogConfigurator ] [ElasticEDT] Some logging configurations have %marker but don't have %node_name. We will automatically add %node_name to the pattern to ease the migration for users who customize log4j2.properties but will stop this behavior in 7.0. You should manually replace %node_name with [%node_name]%marker in these locations:
/etc/elasticsearch/test/log4j2.properties
[2020-09-02T18:34:28,367][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [ElasticEDT] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.8.1.jar:7.8.1]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
uncaught exception in thread [main]
java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
2020-09-02 15:34:28,717860 UTC [25395] ERROR CLogger.cc@311 Cannot log to named pipe /tmp/elasticsearch-10919022139451207318/controller_log_25222 as it could not be opened for writing
2020-09-02 15:34:28,717972 UTC [25395] INFO Main.cc@104 Parent process died - ML controller exiting
[root@ElasticEDT bin]#

here is the error

java.lang.RuntimeException: can not run elasticsearch as root

so it looks like I cannot run Elasticsearch as root:
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
my question is: what user should I run it under, and why is this not stated in your online documentation below:
https://www.elastic.co/guide/en/elastic-stack-get-started/7.8/get-started-elastic-stack.html

hi Yassine,
thanks for your update, we both replied at the same time: please see my previous post below: which user should it run under?
thanks, Bogdan

logout from root and use any normal user with sudo

hi Yassine,
thanks for this, I did this and set the new password for the elasticsearch user.
Now I am getting this error below:
bash-4.4$ ./elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
2020-09-02 19:37:53,812 main ERROR Unable to access file:///etc/elasticsearch/test/log4j2.properties java.io.FileNotFoundException: /etc/elasticsearch/test/log4j2.properties (Permission denied)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
at java.base/java.io.FileInputStream.(FileInputStream.java:157)
at java.base/java.io.FileInputStream.(FileInputStream.java:112)
at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
at java.base/java.net.URL.openStream(URL.java:1140)
at org.apache.logging.log4j.core.config.ConfigurationSource.fromUri(ConfigurationSource.java:247)
at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:237)
at org.elasticsearch.common.logging.LogConfigurator$3.visitFile(LogConfigurator.java:224)
at org.elasticsearch.common.logging.LogConfigurator$3.visitFile(LogConfigurator.java:220)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2724)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:220)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:129)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:354)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)

Exception in thread "main" java.lang.NullPointerException
at org.apache.logging.log4j.core.config.composite.CompositeConfiguration.(CompositeConfiguration.java:65)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:236)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:129)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:354)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
bash-4.4$

hi Yassine,
I have fixed the obvious file and directory permissions errors above, but now I am getting another error, see below for more info:
bash-4.4$ ./elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2020-09-02T19:45:14,029][WARN ][o.e.c.l.LogConfigurator ] [ElasticEDT] Some logging configurations have %marker but don't have %node_name. We will automatically add %node_name to the pattern to ease the migration for users who customize log4j2.properties but will stop this behavior in 7.0. You should manually replace %node_name with [%node_name]%marker in these locations:
/etc/elasticsearch/test/log4j2.properties
[2020-09-02T19:45:14,270][INFO ][o.e.n.Node ] [ElasticEDT] version[7.8.1], pid[27288], build[default/rpm/b5ca9c58fb664ca8bf9e4057fc229b3396bf3a89/2020-07-21T16:40:44.668009Z], OS[Linux/4.18.0-147.5.1.el8_1.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/11.0.8/11.0.8+10-LTS]
[2020-09-02T19:45:14,271][INFO ][o.e.n.Node ] [ElasticEDT] JVM home [/usr/lib/jvm/java-11-openjdk-11.0.8.10-0.el8_2.x86_64]
[2020-09-02T19:45:14,272][INFO ][o.e.n.Node ] [ElasticEDT] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-17844278366693308801, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
[2020-09-02T19:45:16,175][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [aggs-matrix-stats]
[2020-09-02T19:45:16,176][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [analysis-common]
[2020-09-02T19:45:16,176][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [constant-keyword]
[2020-09-02T19:45:16,177][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [flattened]
... and so on ...
[transform]
[2020-09-02T19:45:16,182][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [transport-netty4]
[2020-09-02T19:45:16,182][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [vectors]
[2020-09-02T19:45:16,183][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-analytics]
[2020-09-02T19:45:16,183][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-async-search]
[2020-09-02T19:45:16,183][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-autoscaling]
[2020-09-02T19:45:16,183][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-ccr]
[2020-09-02T19:45:16,184][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-core]
[2020-09-02T19:45:16,184][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-deprecation]
[2020-09-02T19:45:16,184][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-enrich]
[2020-09-02T19:45:16,184][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-eql]
[2020-09-02T19:45:16,184][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-graph]
[2020-09-02T19:45:16,185][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-identity-provider]
[2020-09-02T19:45:16,185][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-ilm]
[2020-09-02T19:45:16,185][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-logstash]
[2020-09-02T19:45:16,185][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-ml]
[2020-09-02T19:45:16,186][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-monitoring]
[2020-09-02T19:45:16,186][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-ql]
[2020-09-02T19:45:16,186][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-rollup]
[2020-09-02T19:45:16,186][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-security]
[2020-09-02T19:45:16,187][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-sql]
[2020-09-02T19:45:16,187][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-voting-only-node]
[2020-09-02T19:45:16,187][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-watcher]
[2020-09-02T19:45:16,187][INFO ][o.e.p.PluginsService ] [ElasticEDT] no plugins loaded
[2020-09-02T19:45:16,234][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [ElasticEDT] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.8.1.jar:7.8.1]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:301) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
Caused by: java.io.IOException: failed to obtain lock on /var/lib/elasticsearch/nodes/0
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:223) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/nodes/0/node.lock
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:292) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:345) ~[?:?]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:125) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:216) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
Likely root cause: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/nodes/0/node.lock
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:125)
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:216)
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271)
at org.elasticsearch.node.Node.(Node.java:335)
at org.elasticsearch.node.Node.(Node.java:266)
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
bash-4.4$

You still have permissions issue to /var/lib/elasticsearch/nodes/

hi Yassine,
I have opened this nodes folder in the maximal possible way, 777:
[root@ElasticEDT elasticsearch]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@ElasticEDT elasticsearch]# pwd
/var/lib/elasticsearch
[root@ElasticEDT elasticsearch]# ll
total 0
drwxrwsrwx. 3 elasticsearch elasticsearch 15 Mar 30 22:28 nodes
[root@ElasticEDT elasticsearch]#
and I also changed the owner to elasticsearch, as you can see, and yet the problem is still happening, see below:
,,, and so on ...
[2020-09-02T20:02:21,218][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-monitoring]
[2020-09-02T20:02:21,219][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-ql]
[2020-09-02T20:02:21,219][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-rollup]
[2020-09-02T20:02:21,219][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-security]
[2020-09-02T20:02:21,219][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-sql]
[2020-09-02T20:02:21,220][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-voting-only-node]
[2020-09-02T20:02:21,220][INFO ][o.e.p.PluginsService ] [ElasticEDT] loaded module [x-pack-watcher]
[2020-09-02T20:02:21,220][INFO ][o.e.p.PluginsService ] [ElasticEDT] no plugins loaded
[2020-09-02T20:02:21,266][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [ElasticEDT] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.8.1.jar:7.8.1]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:301) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
Caused by: java.io.IOException: failed to obtain lock on /var/lib/elasticsearch/nodes/0
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:223) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/nodes/0/node.lock
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:292) ~[?:?]
at java.nio.channels.FileChannel.open(FileChannel.java:345) ~[?:?]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:125) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:216) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.1.jar:7.8.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.1.jar:7.8.1]
... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
Likely root cause: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/nodes/0/node.lock
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:125)
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:216)
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:271)
at org.elasticsearch.node.Node.(Node.java:335)
at org.elasticsearch.node.Node.(Node.java:266)
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
bash-4.4$

The /var/lib/elasticsearch/nodes/0/node.lock is from the old installation
is there any data in your node from the old installation in /var/lib/elasticsearch/nodes/0 ?

No, there is no data below from the old installation:
[root@ElasticEDT 0]# pwd
/var/lib/elasticsearch/nodes/0
[root@ElasticEDT 0]# ll
total 0
-rwxr-xr-x. 1 root root 0 Mar 30 22:28 node.lock
drwxr-sr-x. 2 root root 121 Mar 31 13:47 _state
[root@ElasticEDT 0]#

should I delete this folder?

remove /var/lib/elasticsearch/nodes/0/node.lock

I have removed this node.lock, see below:
[root@ElasticEDT 0]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@ElasticEDT 0]# pwd
/var/lib/elasticsearch/nodes/0
[root@ElasticEDT 0]# ll
total 0
drwxr-sr-x. 2 root root 121 Mar 31 13:47 _state
[root@ElasticEDT 0]#