Not able to start Elasticsearch cluster after disabling security

[hallaoui (Hamid Allaoui)]

Hi All,

I disabled the security due to expiration of platinum subscription but when trying to start the cluster it is not starting.

Any help will be high appreciated.

Thanks

[TimV (Tim Vernum)]

You'll need to provide more information than that.

What error do you get?
What version of Elasticsearch are you running?
What change did you make to disable security?

[hallaoui (Hamid Allaoui)]

Thank you Tim for your replay.

The ELK version is 7.16 and to disable the security I just comment below lines in Elasticsearch config file and set xpack.security.enabled: false

xpack.security.http.ssl.enabled: false
#xpack.security.transport.ssl.enabled: true
#xpack.security.http.ssl.key: certs/Elasticsearch-01.key
#xpack.security.http.ssl.certificate: certs/Elasticsearch-01.crt
#xpack.security.http.ssl.certificate_authorities: certs/ca.crt
#xpack.security.transport.ssl.key: certs/Elasticsearch-01.key
#xpack.security.transport.ssl.certificate: certs/Elasticsearch-01.crt
#xpack.security.transport.ssl.certificate_authorities: certs/ca.crt

For your info I reverted back the changes enabling the security and managed to start 2 cluster nodes out of 3 but when I run below curl I got error.

curl --cacert /etc/Elasticsearch/certs/ca.crt -u 'elastic:XXXXXX' 'https://elasticsearch-02:9200/_cat/nodes?v'

{"error":{"root_cause":[{"type":"security_exception","reason":"current license is non-compliant for [security]","license.expired.feature":"security"}],"type":"security_exception","reason":"current license is non-compliant for [security]","license.expired.feature":"security"},"status":403}

Thanks for your supprt.

[hallaoui (Hamid Allaoui)]

I can see below logs when trying to start the 3rd node.

[2022-01-18T17:13:21,095][TRACE][o.e.d.PeerFinder         ] [elasticsearch-03] startProbe(10.73.65.78:9300) not probing local node
[2022-01-18T17:13:21,095][TRACE][o.e.d.PeerFinder         ] [elasticsearch-03] address [10.73.65.77:9300], node [null], requesting [false] attempting connection
[2022-01-18T17:13:21,096][TRACE][o.e.d.PeerFinder         ] [elasticsearch-03] address [10.73.65.76:9300], node [null], requesting [false] attempting connection
[2022-01-18T17:13:21,096][TRACE][o.e.d.HandshakingTransportAddressConnector] [elasticsearch-03] [connectToRemoteMasterNode[10.73.65.77:9300]] opening probe connection
[2022-01-18T17:13:21,096][DEBUG][o.e.d.SeedHostsResolver  ] [elasticsearch-03] resolveConfiguredHosts: lifecycle is STOPPED, not proceeding
[2022-01-18T17:13:21,097][TRACE][o.e.d.HandshakingTransportAddressConnector] [elasticsearch-03] [connectToRemoteMasterNode[10.73.65.76:9300]] opening probe connection
[2022-01-18T17:13:21,092][ERROR][i.n.u.c.D.rejectedExecution] [elasticsearch-03] Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated
        at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:923) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:350) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:343) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:825) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:815) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:842) [netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:499) [netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) [netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) [netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109) [netty-common-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:89) [netty-transport-4.1.66.Final.jar:4.1.66.Final]
        at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:998) [netty-transport-4.1.66.Final.jar:4.1.66.Final]