Elasticsearch Service suddenly not working in Ubuntu

[TheCthulhuKid (Marc Anthony Taylor)]

When I left the office yesterday I was using ES (5.5.2) with no problem on my Ubuntu 16.10 VM as I have been for the last couple of months. This morning I start up the VM and try to access the cluster and get an exception that the socket connection was refused.

When checking the status I get the following:
Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

I know ubuntu uses systemd but it has been working fine up until now.

There was an update from the Elasticsearch repo yesterday and I am not certain if I restarted the machine at any point after that. There was no update installed this morning.

I have tried stopping and restarting the service, as well as reloading the daemon.

Before I do a fresh install I thought I'd ask here.

Cheers!

[xavierfacq]

Hi,

Can you give the content of the log file when you try to start the service ?

more /var/log/elasticsearch/[nodename].log

[TheCthulhuKid (Marc Anthony Taylor)]

And that, of course, found the issue. The update yesterday made the ingest plugin incompatible with the ES version. Strange that the exception mentions upstart. Below is the start of the log:

[2017-08-22T07:42:35,895][ERROR][o.e.b.Bootstrap ] Exception
java.lang.IllegalArgumentException: plugin [ingest-attachment] is incompatible with version [5.5.2]; was designed for version [5.5.1]
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:146) ~[elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Spawner.spawnNativePluginControllers(Spawner.java:76) ~[elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:167) ~[elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.5.2.jar:5.5.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.5.2.jar:5.5.2]
[2017-08-22T07:42:35,901][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: plugin [ingest-attachment] is incompatible with version [5.5.2]; was designed for version [5.5.1]