Elasticsearch.service file issue during upgrade to 6.1

[pk.241011]

I tried to install 6.1 on my cluster which was running originally on 5.6.3. I got the same error as here.
What was new to me was this statement:

@jasontedor

This is the key log line. We do not have an ExecStartPre entry in 6.0.0 yet clearly your unit file still refers to one after the upgrade. This means you still have an old 5.x unit file and the upgrade did not replace the unit file with the 6.0.0 unit file. Maybe you edited the unit file (don’t do that!) and when you upgraded the package installer gave you a .rpmnew. You need to use the 6.0.0 unit file.

Yet in the elasticsearch.service file itself this was mentioned.
Specifies the maximum number of bytes of memory that may be locked into RAM Set to "infinity" if you use the 'bootstrap.memory_lock: true' option in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch

LimitMEMLOCK=infinity

So I did change this file. And as pointed out this file was not changed when I upgraded from 5.2.2 to 5.6.3 as this line at the end of current file shows
Built for distribution-5.2.2 (distribution)
After upgrade I think it should have been 5.6.3.

The error I see in /var/log/messages after install of 6.1
is
Failed at step EXEC spawning /usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec: No such file or directory

What are the options I have now?
Is there any place I can copy the Unit file for 6.1 or is there a better way to deal with this?

[jasontedor]

@pk.241011

What are the options I have now?

Are you on an RPM-based system or a Debian-based system?

[pk.241011]

RPM. Redhat if that matters.

[jasontedor]

Download the RPM and extract the service file:

$ mkdir /tmp/rpm
$ rpm --badreloc --nodeps --noscripts --notriggers --dbpath /tmp/rpm/rpm-database --relocate /=/tmp/rpm/rpm-extracted -I /path/to/rpm/elasticsearch-6.1.0.rpm

Then, the service file will be in /tmp/rpm/rpm-extracted/usr/lib/systemd/system/elasticsearch.service and you can copy it in place.

Does this help?

[tatdat (Dat Pham Tat)]

Hi @pk.241011,

I think you have same problem with me. You modified usr/lib/systemd/system/elasticsearch.service before, so when you run upgrade with rpm package, it can't replace new file startup for ES 6.1.

Now, u can open usr/lib/systemd/system/elasticsearch.service and replace content with:

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
RuntimeDirectory=elasticsearch
Environment=ES_HOME=/usr/share/elasticsearch
Environment=ES_PATH_CONF=/etc/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch

WorkingDirectory=/usr/share/elasticsearch

User=elasticsearch
Group=elasticsearch

ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet

StandardOutput=journal
StandardError=inherit
LimitNOFILE=65536
LimitNPROC=4096
LimitAS=infinity
LimitFSIZE=infinity
TimeoutStopSec=0
KillSignal=SIGTERM
KillMode=process
SendSIGKILL=no
SuccessExitStatus=143
WantedBy=multi-user.target

Remember, add LimitMEMLOCK=infinity setting to this file (for bootstrap setting).

[pk.241011]

@tatdat I plan to go thorough all the *.rmpnew files created by upgrade/install process. I will ideally like these files to be in sync content wise corresponding to the release. *.rpmsave is ok as it is copy of old settings but *.rpmnew is something I have to account for.

/usr/lib/systemd/system/elasticsearch.service.rpmnew
/etc/elasticsearch/elasticsearch.yml.rpmnew
/etc/logstash/logstash.yml.rpmnew
/etc/kibana/kibana.yml.rpmnew

[pk.241011]

In case someone is stuck with this error, the best way is to hunt for the *.rpmnew file which the installer leaves behind (It will not replace the the file which has user modifications).

Migrate your changes from the old
/usr/lib/systemd/system/elasticsearch.service
to the
/usr/lib/systemd/system/elasticsearch.service.rpmnew
In my case it was just adding this variable
LimitMEMLOCK=infinity

Then take a backup of the old elasticsearch.service file and copy the contents of
/usr/lib/systemd/system/elasticsearch.service.rpmnew
to
/usr/lib/systemd/system/elasticsearch.service