Elasticsearch service fail afte upgrade


(Xabi) #1

Hi,

I upgrade elasticsearch from 6.3 to 6.4 and now i can't start service. I'm on Debian 9.

Here systemctl status elasticsearch:

   elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)`
   Active: failed (Result: exit-code) since Wed 2018-09-26 17:06:11 CEST; 16h ago
   Docs: http://www.elastic.co
   Process: 14084 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
   Main PID: 14084 (code=exited, status=1/FAILURE)

   Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

And here elasticsearch log /var/log/elasticsearch/elasticsearch.log

at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:524) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:464) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:156) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.node.Node.<init>(Node.java:315) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.node.Node.<init>(Node.java:256) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.4.1.jar:6.4.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.4.1.jar:6.4.1]
    ... 6 more

This is the /etc/elasticsearch/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: localhost
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

plugin.mandatory: ingest-attachment

Any advice?

Thanks,

Xabi


(David Pilato) #2

Can you share the full logs?


(Xabi) #3

Here the full logs

   [2018-09-27T13:28:22,425][INFO ][o.e.n.Node               ] [] initializing ...
[2018-09-27T13:28:22,593][INFO ][o.e.e.NodeEnvironment    ] [DU7Y9_T] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [13.9gb], net total_space [17.5gb], types [ext4]
[2018-09-27T13:28:22,594][INFO ][o.e.e.NodeEnvironment    ] [DU7Y9_T] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-09-27T13:28:22,623][INFO ][o.e.n.Node               ] [DU7Y9_T] node name derived from node ID [DU7Y9_TjTJ-TE883WSrQ_Q]; set [node.name] to override
[2018-09-27T13:28:22,624][INFO ][o.e.n.Node               ] [DU7Y9_T] version[6.4.1], pid[17122], build[default/deb/e36acdb/2018-09-13T22:18:07.696808Z], OS[Linux/4.9.0-7-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_181/25.181-b13]
[2018-09-27T13:28:22,624][INFO ][o.e.n.Node               ] [DU7Y9_T] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.M16yWwxI, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=deb]
[2018-09-27T13:28:26,680][ERROR][o.e.b.Bootstrap          ] Exception
java.lang.IllegalArgumentException: Plugin [ingest-attachment] was built for Elasticsearch version 6.3.2 but version 6.4.1 is running
  at org.elasticsearch.plugins.PluginsService.verifyCompatibility(PluginsService.java:339) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:524) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:464) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:156) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.node.Node.<init>(Node.java:315) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.node.Node.<init>(Node.java:256) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) [elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) [elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) [elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) [elasticsearch-6.4.1.jar:6.4.1]
[2018-09-27T13:28:26,691][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [ingest-attachment] was built for Elasticsearch version 6.3.2 but version 6.4.1 is running
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.4.1.jar:6.4.1]
  at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.4.1.jar:6.4.1]
Caused by: java.lang.IllegalArgumentException: Plugin [ingest-attachment] was built for Elasticsearch version 6.3.2 but version 6.4.1 is running
  at org.elasticsearch.plugins.PluginsService.verifyCompatibility(PluginsService.java:339) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:524) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:464) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:156) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.node.Node.<init>(Node.java:315) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.node.Node.<init>(Node.java:256) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.4.1.jar:6.4.1]
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.4.1.jar:6.4.1]
  ... 6 more

I think this is the question

Plugin [ingest-attachment] was built for Elasticsearch version 6.3.2 but version 6.4.1 is running

ingest-attachment it's not compatible with Elasticserach 6.4.1?


(David Pilato) #4

You need to upgrade the plugin as well


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.