Unable to upgrade elasticsearch from 6.8.1 to 7.16.3

Hi,
Unable to upgrade Elasticsearch to 7.16.3 from 6.81 in our elastic cluster with 4 nodes. getting below messages in log file with this. please help.

[2022-03-12T17:04:36,236][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.194.121] waiting for elected master node [{uklvapapp102p}{LQtfZsRfT0KjDJHxayQliw}{xxiooMrCSG6B--ywZ3ZmEA}{uklvapapp102p.gdc.standardchartered.com}{10.4.195.178:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)

4 node cluster:
uklvapapp103p
uklvasapp633
uklvapapp102p (master)
uklvapapp633

Thanks.

Welcome.

Did you do a full cluster restart or did you try a rolling upgrade?

Rolling upgrade.

first tried in 103p server, same issue occurs and later proceeded to 633s, there also same error.

can you pls advise further on this issue..

You should try a full cluster restart may be?

If it does not work, please provide the full logs of all the nodes.

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.

Yes, later tried to upgrade in all 4 elastic nodes, still no luck. Logs of all nodes in quite big, shall i forward to your mail id, kindly share it. thanks.

If some outputs are too big, please share them on gist.github.com and link them here.

Unable to upload in other link, have added the below messages from logs, which repeatedly occurs:

[2022-03-12T10:13:48,409][INFO ][o.e.n.Node               ] [10.4.195.179] started
[2022-03-12T10:14:20,052][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:23,238][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:25,305][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:27,629][WARN ][r.suppressed             ] [10.4.195.179] path: /_cat/templates/metricbeat-7.14.0, params: {name=metricbeat-7.14.0}
org.elasticsearch.transport.RemoteTransportException: [indices:admin/index_template/get]
Caused by: org.elasticsearch.transport.ActionNotFoundTransportException: No handler for action [indices:admin/index_template/get]
	at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1033) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:932) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:763) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:53) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1436) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) ~[?:?]
	at java.lang.Thread.run(Thread.java:748) [?:?]

kindly help to advise here.

Please share the full logs of all nodes.

Hi,
We are unable to load the logs in the gist.github location due to security reasons.. can you pls provide you elastic mailid and we shall forward all the details to you.
Thanks.

Kindly provide your elastic mail id. Thanks.

Please be patient in waiting for responses to your question and refrain from pinging multiple times asking for a response or opening multiple topics for the same question. This is a community forum, it may take time for someone to reply to your question. For more information please refer to the Community Code of Conduct specifically the section "Be patient". Also, please refrain from pinging folks directly, this is a forum and anyone that participates might be able to assist you.

If you are in need of a service with an SLA that covers response times for questions then you may want to consider talking to us about a subscription.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

Could you find another service like pastebin or google drive or whatever sharing service?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.