Server disk broken and recover, but failed to start

jinjinbari · July 22, 2020, 6:51am

Hello.

My server disk was broken and recover now.

But the elasticsearch service isn`t started.

Here is my logs.

[2020-07-22T15:54:22,153][INFO ][o.e.e.NodeEnvironment    ] [apm-data-node4] using [1] data paths, mounts [[/data/1 (/dev/mapper/vg_data-lv_data1)]], net usable_space [934.6gb], net total_space [1.7tb], types [xfs]
[2020-07-22T15:54:22,155][INFO ][o.e.e.NodeEnvironment    ] [apm-data-node4] heap size [3.8gb], compressed ordinary object pointers [true]
[2020-07-22T15:54:22,699][INFO ][o.e.n.Node               ] [apm-data-node4] node name [apm-data-node4], node ID [B9s7SzjMRE65YF4wrULl7Q]
[2020-07-22T15:54:22,700][INFO ][o.e.n.Node               ] [apm-data-node4] version[6.5.4], pid[25116], build[default/rpm/d2ef93d/2018-12-17T21:17:40.758843Z], OS[Linux/3.10.0-693.21.1.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09]
[2020-07-22T15:54:22,700][INFO ][o.e.n.Node               ] [apm-data-node4] JVM arguments [-ea, -Xms20g, -Xmx20g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Des.cgroups.hierarchy.override=/, -Xms4g, -Xmx4g, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.YJ1pgI3u, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm]
[2020-07-22T15:54:23,860][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [aggs-matrix-stats]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [analysis-common]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [ingest-common]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [lang-expression]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [lang-mustache]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [lang-painless]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [mapper-extras]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [parent-join]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [percolator]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [rank-eval]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [reindex]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [repository-url]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [transport-netty4]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [tribe]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-ccr]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-core]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-deprecation]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-graph]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-logstash]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-ml]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-monitoring]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-rollup]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-security]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-sql]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-upgrade]
[2020-07-22T15:54:23,861][INFO ][o.e.p.PluginsService     ] [apm-data-node4] loaded module [x-pack-watcher]
[2020-07-22T15:54:23,862][INFO ][o.e.p.PluginsService     ] [apm-data-node4] no plugins loaded
[2020-07-22T15:54:23,880][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [apm-data-node4] fatal error in thread [main], exiting
java.lang.AssertionError: null
        at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:487) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:531) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.xpack.ml.MachineLearning.additionalSettings(MachineLearning.java:327) ~[?:?]
        at org.elasticsearch.plugins.PluginsService.updatedSettings(PluginsService.java:203) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.node.Node.<init>(Node.java:339) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.5.4.jar:6.5.4]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.5.4.jar:6.5.4]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.5.4.jar:6.5.4]

How can I fix this?

Thank you

warkolm · July 22, 2020, 6:52am

Is that all that is in the logs?

jinjinbari · July 22, 2020, 6:56am

Hi~ I edit the logs. Thank you

warkolm · July 22, 2020, 6:56am

Thanks!
Is there anything after that last entry?

jinjinbari · July 22, 2020, 6:59am

Yes.. There are no more logs after last entry..

Steve_Mushero · July 23, 2020, 4:44am

Why is -Xms, -Xmx on the command line twice? Both for 20G and 4G?

This in docker, as I see OsProbe.getCgroup as last area the exception happened, which is odd - any other strange settings as it's still probing for OS and related settings.

Looks like a CentOS VM but maybe some weird cgroup settings, like cgroup is returning null, which I suspect it should not do (I think Linux has a default cgroup).

Seems this happens sometimes, see these and fixed in V7.4

github.com/elastic/elasticsearch

ES not able to start

opened 08:14AM - 17 Oct 17 UTC

closed 02:17AM - 14 Mar 18 UTC

hello4usharath

:Core/Infra/Core

``` ][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [********] uncaught e…xception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.NullPointerException at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.3.jar:5.6.3] Caused by: java.lang.NullPointerException at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:273) ~[?:?] at org.elasticsearch.common.io.PathUtils.get(PathUtils.java:60) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.os.OsProbe.readSysFsCgroupCpuAcctCpuAcctUsage(OsProbe.java:274) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.os.OsProbe.getCgroupCpuAcctUsageNanos(OsProbe.java:261) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:419) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:464) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:45) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:45) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.node.Node.<init>(Node.java:362) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:233) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:233) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.3.jar:5.6.3] ```

github.com/elastic/elasticsearch

ES 6.8.1 (also 7.2.1) fails to start (probably due to cgroups)

opened 04:20PM - 09 Aug 19 UTC

closed 08:26AM - 16 Aug 19 UTC

kakra

:Core/Infra/Core

**Elasticsearch version** (`bin/elasticsearch --version`): 6.8.1 (Gentoo ebuild…) running in a systemd nspawn container **Plugins installed**: [] **JVM version** (`java -version`): openjdk version "1.8.0_212" OpenJDK Runtime Environment (IcedTea 3.12.0) (Gentoo icedtea-3.12.0) OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode) **OS version** (`uname -a` if on a Unix-like system): Linux es6 5.2.7-gentoo #1 SMP Fri Aug 9 11:40:38 CEST 2019 x86_64 Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz GenuineIntel GNU/Linux **Description of the problem including expected versus actual behavior**: Since upgrading ES in my development container, it no longer starts. It crashes with a stack trace pointing to problems determining cgroup controllers. Researching other reports showed that the problem should be fixed, but it still shows for me. Logs below. Last known good version: `Sat Sep 15 02:21:22 2018 >>> app-misc/elasticsearch-6.4.0` **Steps to reproduce**: ``` es6 ~ # ls -l /sys/fs/cgroup insgesamt 0 drwxr-xr-x 2 nobody nobody 0 9. Aug 17:42 blkio drwxr-xr-x 2 nobody nobody 0 9. Aug 17:42 cpu lrwxrwxrwx 1 root root 3 9. Aug 17:42 cpuacct -> cpu drwxr-xr-x 2 nobody nobody 0 9. Aug 17:42 devices drwxr-xr-x 2 nobody nobody 0 9. Aug 17:42 memory dr-xr-xr-x 2 nobody nobody 0 9. Aug 11:57 net_cls lrwxrwxrwx 1 root root 7 9. Aug 17:42 net_prio -> net_cls drwxr-xr-x 2 nobody nobody 0 9. Aug 17:42 pids drwxr-xr-x 5 root root 0 9. Aug 17:42 systemd drwxr-xr-x 5 root root 0 9. Aug 17:42 unified es6 ~ # cat /proc/cgroups #subsys_name hierarchy num_cgroups enabled cpu 2 88 1 blkio 4 108 1 memory 5 128 1 devices 3 86 1 net_cls 6 1 1 pids 7 92 1 es6 ~ # cat /proc/self/cgroup 7:pids:/ 6:net_cls:/ 5:memory:/ 4:blkio:/ 3:devices:/ 2:cpu:/ 1:name=systemd:/user.slice/user-0.slice/session-c1.scope 0::/user.slice/user-0.slice/session-c1.scope es6 ~ # mount -t cgroup cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/cpu type cgroup (rw,nosuid,nodev,noexec,relatime,cpu) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd) es6 ~ # grep cgroup /proc/self/mountinfo 1067 1141 0:106 / /sys/fs/cgroup ro,nosuid,nodev,noexec - tmpfs tmpfs ro,mode=755,uid=65536,gid=65536 1068 1067 0:29 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,memory 1069 1067 0:30 / /sys/fs/cgroup/net_cls rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,net_cls 1070 1067 0:27 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,devices 1071 1067 0:28 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,blkio 1073 1067 0:26 / /sys/fs/cgroup/cpu rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpu 1074 1067 0:31 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,pids 1075 1067 0:21 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup rw,nsdelegate 1076 1067 0:22 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,xattr,name=systemd # FULL LOG AT THE END es6 ~ # systemctl start elasticsearch.service es6 ~ # systemctl status elasticsearch.service ● elasticsearch.service - Elasticsearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/elasticsearch.service.d └─00gentoo.conf, override.conf Active: failed (Result: exit-code) since Fri 2019-08-09 18:18:28 CEST; 640ms ago Docs: https://www.elastic.co Process: 437 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS) Process: 438 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid -Epath.logs=${LOG_DIR} -Epath.data=${DATA_DIR} (code=exited, status=1/FAILURE) Main PID: 438 (code=exited, status=1/FAILURE) Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:46) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.node.Node.<init>(Node.java:399) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.1.jar:6.8.1] Aug 09 18:18:28 es6 elasticsearch[438]: ... 6 more Aug 09 18:18:28 es6 systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE Aug 09 18:18:28 es6 systemd[1]: elasticsearch.service: Failed with result 'exit-code'. ``` **Provide logs (if relevant)**: ``` java.lang.NullPointerException: null at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:273) ~[?:?] at org.elasticsearch.common.io.PathUtils.get(PathUtils.java:60) ~[elasticsearch-core-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.readSysFsCgroupCpuAcctCpuAcctUsage(OsProbe.java:277) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.getCgroupCpuAcctUsageNanos(OsProbe.java:264) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:483) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:603) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:49) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:46) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.node.Node.<init>(Node.java:399) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.1.jar:6.8.1] at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.1.jar:6.8.1] [2019-08-09T17:46:33,105][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [QoY4z88] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.NullPointerException at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.8.1.jar:6.8.1] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.8.1.jar:6.8.1] Caused by: java.lang.NullPointerException at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:273) ~[?:?] at org.elasticsearch.common.io.PathUtils.get(PathUtils.java:60) ~[elasticsearch-core-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.readSysFsCgroupCpuAcctCpuAcctUsage(OsProbe.java:277) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.getCgroupCpuAcctUsageNanos(OsProbe.java:264) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:483) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:603) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:49) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:46) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.node.Node.<init>(Node.java:399) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.1.jar:6.8.1] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.1.jar:6.8.1] ... 6 more ```

Also some workarounds here:

jinjinbari · July 23, 2020, 6:52am

Thank you

It had problems for mount.

I mounted cgroups, then the elasticsearch was started.

Thank you for your help

system · August 20, 2020, 6:52am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Elasticsearch does not start Elasticsearch	18	2610	December 8, 2017
Cannot start services after restart Elasticsearch	1	316	August 7, 2018
Elasticsearch not starting Elasticsearch	3	452	July 21, 2022
Ancient Broken ElasticSearch question Elasticsearch	3	341	August 6, 2019
Elastic Server is stopped and not starting Elasticsearch	10	4015	February 21, 2018

Server disk broken and recover, but failed to start

Related topics