Can't start elasticsearch with Ubuntu 16.04

No logs are produced by the service

root@rubix2:/usr/share/elasticsearch# ls -l logs
lrwxrwxrwx 1 root root 22 Dec 24 00:15 logs -> /var/log/elasticsearch

root@rubix2:/usr/share/elasticsearch# cd /var/log/elasticsearch/
root@rubix2:/var/log/elasticsearch# rm *
root@rubix2:/var/log/elasticsearch# ls
root@rubix2:/var/log/elasticsearch# service elasticsearch start
root@rubix2:/var/log/elasticsearch# ls
root@rubix2:/var/log/elasticsearch# service elasticsearch status
● elasticsearch.service - LSB: Starts elasticsearch
   Loaded: loaded (/etc/init.d/elasticsearch; bad; vendor preset: enabled)
   Active: active (exited) since Thu 2016-04-28 16:03:54 EDT; 32min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1192 ExecStart=/etc/init.d/elasticsearch start (code=exited, status=0
    Tasks: 0 (limit: 512)

Apr 28 16:03:54 rubix2 systemd[1]: Starting LSB: Starts elasticsearch...
Apr 28 16:03:54 rubix2 systemd[1]: Started LSB: Starts elasticsearch.
Apr 28 16:36:19 rubix2 systemd[1]: Started LSB: Starts elasticsearch.
root@rubix2:/var/log/elasticsearch# ls
root@rubix2:/var/log/elasticsearch#

Did you setup the repository like in the instructions here?

What version do you have installed?

I tried with https://launchpad.net/ubuntu/xenial/+source/elasticsearch, so 1.7.3.

I would think that others are likely to try this, so if that version shouldn't be used, perhaps a blog post or some such is in order? I'll try the latest from packages.elastic.co next (probably tomorrow morning).

Now I followed the instructions, again from a fresh install of Ubuntu 16.04. Still no success:

root@rubix2:~# dpkg -l elasticsearch
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  elasticsearch  2.3.2        all          Elasticsearch is a distributed RE
root@rubix2:~# /bin/systemctl daemon-reload
root@rubix2:~# /bin/systemctl enable elasticsearch
Synchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...
Executing /lib/systemd/systemd-sysv-install enable elasticsearch
root@rubix2:~# curl http://localhost:9200/
curl: (7) Failed to connect to localhost port 9200: Connection refused
root@rubix2:~# sleep 10
root@rubix2:~# curl http://localhost:9200/
curl: (7) Failed to connect to localhost port 9200: Connection refused

Information when run directly:

root@rubix2:/usr/share/elasticsearch# ls /etc/elasticsearch/
elasticsearch.yml  logging.yml  scripts
root@rubix2:/usr/share/elasticsearch# ls
bin  lib  modules  NOTICE.txt  plugins  README.textile
root@rubix2:/usr/share/elasticsearch# bin/elasticsearch
Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at org.elasticsearch.common.logging.log4j.LogConfigurator.resolveConfig(LogConfigurator.java:142)
    at org.elasticsearch.common.logging.log4j.LogConfigurator.configure(LogConfigurator.java:103)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:243)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.

The int script does work, but only after you enable it in systemd (in disabled/bad state it doesn't use the sysv wrapper).

root@xenial-sandbox:~# systemctl enable elasticsearch
elasticsearch.service is not a native service, redirecting to systemd-sysv-install
Executing /lib/systemd/systemd-sysv-install enable elasticsearch
insserv: warning: current start runlevel(s) (empty) of script elasticsearch' overrides LSB defaults (2 3 4 5). insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script elasticsearch' overrides LSB defaults (0 1 6).
root@xenial-sandbox:~# systemctl start elasticsearch
root@xenial-sandbox:~# systemctl status elasticsearch
● elasticsearch.service - LSB: Starts elasticsearch
Loaded: loaded (/etc/init.d/elasticsearch; bad; vendor preset: enabled)
Active: active (running) since Thu 2016-05-26 10:26:36 CEST; 5s ago
Docs: man:systemd-sysv-generator(8)
Process: 1528 ExecStart=/etc/init.d/elasticsearch start (code=exited, status=0/SUCCESS)
Tasks: 29 (limit: 512)
Memory: 213.0M
CPU: 4.960s
CGroup: /system.slice/elasticsearch.service
└─1560 /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMe

Edit: the 1.7.3 init script refuses to work, i copied the one from the 2.3.1 package, which does.
Edit2: just upgraded systemd to 229-4ubuntu6 and now both init scripts are "broken".

2 Likes

It seems that to get Elasticsearch to run on 16.04 you have to set START_DAEMON to true on /etc/default/elasticsearch. It comes commented out by default, and uncommenting it makes Elasticsearch start again just fine.

Be sure to use systemctl restart instead of just start because the service is started right after installation, and apparently there's some socket/pidfile/something that systemd keeps that must be released before being able to start the service again --- I didn't bother to track that down, feel free to look for it :slight_smile:

Here's a sample Vagrantfile for it:

Vagrant.configure(2) do |config|
  config.vm.box = 'ubuntu/xenial64'

  config.vm.provision 'shell', inline: <<-SHELL
    sudo apt-get update
    sudo apt-get install elasticsearch -y

    sudo sed -i 's/#START_DAEMON/START_DAEMON/' /etc/default/elasticsearch

    sudo systemctl restart elasticsearch
    systemctl status elasticsearch
  SHELL
end
18 Likes

Thank-you this works for me.

And the Oscar goes to... @fcoelho

Thank you, there are two days I try to make it work.

:slight_smile: YEAH! IT WORKS!

@fcoelho Thanks for the solution.It really works!.

Thanks for the script. It really works.
But the guide in online doc is still wrong :
https://www.elastic.co/start

1 Like

This is still not fixed. On a clean install elasticsearch is not able to start.

2 Likes

This was a pain to trouble shoot. Thanks fcoelho. All my other installs and versions were fine was specific to this apt-get package that it just, no logs, no nothing, said was active, etc, so seemed normal.

:tada: IT WORKS!

Works for me too! Thanks =D

I am lucky. I was spent 3 hours.

I'd like my hour back. How can this possibly not be fixed yet? Your default install on probably your most popular platform fails with no error message and no logs. It hardly inspires faith in the platform.

I just tried the documented method of install and things started without a problem - I only needed to adjust the default heap use cause my VM was small.

If you can provide details on what is happening we can work to fix it, simply saying it's broken makes that very difficult.

1 Like

Hi. What is happening is what is described by the other posters above.

Specifically, on Ububtu (in my case 16), the elasticsearch init script (ie typing 'systemctl start elasticsearch' from a shell) does not start elasticsearch. It returns '[OK]', but elasticsearch does not start. It gives no error message and leaves no logs. It is therefore very difficult to debug, know where to start, or give any useful feedback.

As others have posted, running this command from a shell will successfully launch elasticsearch for me:

sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch/elasticsearch.pid --default.config=/etc/elasticsearch/elasticsearch.yml --default.path.home=/usr/share/elasticsearch --default.path.logs=/var/log/elasticsearch --default.path.data=/var/lib/elasticsearch --default.path.work=/tmp/elasticsearch --default.path.conf=/etc/elasticsearch

So it would seem that the init script is broken. Since it gives no output, it is difficult to debug, but I will attempt to do so today.

My suspicion is that it makes wrong assumptions about its environment, since I'm guessing it works on some distros, for example it worked for you.

1 Like

I just tried again using https://app.vagrantup.com/v0rtex/boxes/xenial64 and it worked fine, just needed to set heap to suit the VM.