# Elasticsearch no longer works under systemd (7.4.0 on CentOS 7.7.1908)?

**URL:** <https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846>\
**Category:** Elasticsearch\
**Created:** [October 1, 2019, 5:03pm UTC](https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846 "2019-10-01T17:03:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![IanC](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ianc/32/55209_2.png) [@IanC](https://discuss.elastic.co/u/IanC)\
**Post date:** [October 1, 2019, 5:03pm UTC](https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846/1 "2019-10-01T17:03:30Z")

</div>

I'm at a loss here. I have a WHM-managed CentOS 7.7 that's regularly applying patches in the background. I did a reboot today to have a kernel update take effect and when the system came back up ElasticSearch failed to start.

I'm running the latest ES package that's available according to yum on my machine:

```auto
# yum info elasticsearch
Loaded plugins: langpacks, universal-hooks
Installed Packages
Name : elasticsearch
Arch : x86_64
Version : 7.4.0
Release : 1
Size : 469 M
Repo : installed
From repo : elasticsearch-7.x
Summary : Distributed RESTful search engine built for the cloud
URL : https://www.elastic.co/
License : Elastic License
Description : Reference documentation can be found at
            : https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
            : and the 'Elasticsearch: The Definitive Guide' book can be found at
            : https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html

```

And my OS is:

```auto
# cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)

```

The problem seems to be releated to `ES_SD_NOTIFY=true` -- if this is set to `true` I get:

```auto
[2019-10-01T12:59:52,782][INFO][o.e.n.Node] [server] started
[2019-10-01T12:59:52,784][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [sever] fatal error in thread [main], exiting
java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
        at org.elasticsearch.systemd.Libsystemd.lambda$static$0(Libsystemd.java:34) ~[?:?]
        at java.security.AccessController.doPrivileged(AccessController.java:312) ~[?:?]
        at org.elasticsearch.systemd.Libsystemd.<clinit>(Libsystemd.java:33) ~[?:?]
        at org.elasticsearch.systemd.SystemdPlugin.sd_notify(SystemdPlugin.java:66) ~[?:?]
        at org.elasticsearch.systemd.SystemdPlugin.onNodeStarted(SystemdPlugin.java:74) ~[?:?]
        at java.util.ArrayList.forEach(ArrayList.java:1507) ~[?:?]
        at org.elasticsearch.node.Node.start(Node.java:768) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:273) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.4.0.jar:7.4.0]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.4.0.jar:7.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.4.0.jar:7.4.0]

```

When the ES process is starting up. If I set it to false ES starts, but then systemd is never notified that ES has started correctly and, as a result, times out the startup thinking ES failed to start and shuts down ES.

I'm sort of at a loss for how to resolve this one. Anyone have any pointers? I'm using the rpm-installed ES so it has its own JDK.

---

<div class="post-metadata">

**Author:** ![IanC](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ianc/32/55209_2.png) [@IanC](https://discuss.elastic.co/u/IanC)\
**Post date:** [October 1, 2019, 5:15pm UTC](https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846/2 "2019-10-01T17:15:19Z")

</div>

Figured it out. `/tmp` is set to mount with `noexec` on the system and WHM seems to want to enforce that pretty hard. I just set `ES_TMPDIR` to point to another location and now everything seems to work correctly.

Leaving this here as the answer for anyone else that runs into the problem.

---

<div class="post-metadata">

**Author:** ![IanC](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ianc/32/55209_2.png) [@IanC](https://discuss.elastic.co/u/IanC)\
**Post date:** [October 1, 2019, 5:22pm UTC](https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846/3 "2019-10-01T17:22:58Z")

</div>

I'll add that I set ES\_TMPDIR in two places. First I set it in `/etc/sysconfig/elasticsearch` and then I ran `systemctl edit --full elasticsearch.service` and added:

```auto
Environment=ES_TMPDIR=/usr/share/elasticsearch/tmp

```

In the `[service]` section. I'm not 100% certain the second step is required but systemd is such an utter mess I can't be certain it was or was not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2019, 5:22pm UTC](https://discuss.elastic.co/t/elasticsearch-no-longer-works-under-systemd-7-4-0-on-centos-7-7-1908/201846/4 "2019-10-29T17:22:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
