Unable to upgrade elasticsearch from 6.8.1 to 7.16.3

karthik_s4 · March 15, 2022, 7:27am

Hi,
Unable to upgrade Elasticsearch to 7.16.3 from 6.81 in our elastic cluster with 4 nodes. getting below messages in log file with this. please help.

[2022-03-12T17:04:36,236][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.194.121] waiting for elected master node [{uklvapapp102p}{LQtfZsRfT0KjDJHxayQliw}{xxiooMrCSG6B--ywZ3ZmEA}{uklvapapp102p.gdc.standardchartered.com}{10.4.195.178:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)

4 node cluster:
uklvapapp103p
uklvasapp633
uklvapapp102p (master)
uklvapapp633

Thanks.

dadoonet · March 15, 2022, 8:18am

Welcome.

Did you do a full cluster restart or did you try a rolling upgrade?

karthik_s4 · March 15, 2022, 8:21am

Rolling upgrade.

first tried in 103p server, same issue occurs and later proceeded to 633s, there also same error.

karthik_s4 · March 15, 2022, 9:45am

can you pls advise further on this issue..

dadoonet · March 15, 2022, 9:58am

You should try a full cluster restart may be?

If it does not work, please provide the full logs of all the nodes.

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.

karthik_s4 · March 15, 2022, 10:52am

Yes, later tried to upgrade in all 4 elastic nodes, still no luck. Logs of all nodes in quite big, shall i forward to your mail id, kindly share it. thanks.

dadoonet · March 15, 2022, 11:10am

If some outputs are too big, please share them on gist.github.com and link them here.

karthik_s4 · March 15, 2022, 12:06pm

Unable to upload in other link, have added the below messages from logs, which repeatedly occurs:

[2022-03-12T10:13:48,409][INFO ][o.e.n.Node               ] [10.4.195.179] started
[2022-03-12T10:14:20,052][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:23,238][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:25,305][INFO ][o.e.x.m.e.l.LocalExporter] [10.4.195.179] waiting for elected master node [{uklvapapp633}{5dBCyWwhSneYs51e7eswXA}{z6fxVZikT8mEuYIBMIQrrg}{uklvapapp633.gdc.standardchartered.com}{10.4.194.121:9300}{dim}{ml.machine_memory=67387404288, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}] to setup local exporter [default_local] (does it have x-pack installed?)
[2022-03-12T10:14:27,629][WARN ][r.suppressed             ] [10.4.195.179] path: /_cat/templates/metricbeat-7.14.0, params: {name=metricbeat-7.14.0}
org.elasticsearch.transport.RemoteTransportException: [indices:admin/index_template/get]
Caused by: org.elasticsearch.transport.ActionNotFoundTransportException: No handler for action [indices:admin/index_template/get]
	at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1033) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:932) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:763) ~[elasticsearch-7.16.3.jar:7.16.3]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:53) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1436) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[?:?]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[?:?]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) ~[?:?]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) ~[?:?]
	at java.lang.Thread.run(Thread.java:748) [?:?]

karthik_s4 · March 15, 2022, 12:52pm

kindly help to advise here.

dadoonet · March 15, 2022, 1:35pm

Please share the full logs of all nodes.

karthik_s4 · March 16, 2022, 7:31am

Hi,
We are unable to load the logs in the gist.github location due to security reasons.. can you pls provide you elastic mailid and we shall forward all the details to you.
Thanks.

karthik_s4 · March 16, 2022, 8:55am

Kindly provide your elastic mail id. Thanks.

dadoonet · March 16, 2022, 9:53am

Please be patient in waiting for responses to your question and refrain from pinging multiple times asking for a response or opening multiple topics for the same question. This is a community forum, it may take time for someone to reply to your question. For more information please refer to the Community Code of Conduct specifically the section "Be patient". Also, please refrain from pinging folks directly, this is a forum and anyone that participates might be able to assist you.

If you are in need of a service with an SLA that covers response times for questions then you may want to consider talking to us about a subscription.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

dadoonet · March 16, 2022, 9:54am

Could you find another service like pastebin or google drive or whatever sharing service?

system · April 13, 2022, 9:54am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Rolling upgrade problem from 6.8 to 7.1.1 Elasticsearch	18	2857	July 18, 2019
Waiting for elected master node in Rolling upgrade Elasticsearch docker	5	2306	January 19, 2022
Upgrade from 7.17.1 to 8.5.3 failed Elasticsearch	9	800	January 21, 2023
Upgrade failed to 8.x Elasticsearch	7	693	July 24, 2023
Uprading Elastic search upgrade from 6.6.1 to 7.16.3 Elasticsearch	4	192	January 8, 2024

Unable to upgrade elasticsearch from 6.8.1 to 7.16.3

Related topics