Getting xpack missing authentication token error after uninstalling xpack

Elastic Stack Elasticsearch
1

Hello,

I understand that with xpack installed you need to pass credentials along with a request. In this case, I have uninstalled xpack on all nodes in the cluster and restarted the cluster.

On cluster restart, the status is red and inspection of the logs shows the following error message.

'''
[2019-11-12T09:51:51,876][DEBUG][o.e.a.a.c.n.i.TransportNodesInfoAction] [017507000505] failed to execute on node
org.elasticsearch.transport.RemoteTransportException: [127.0.0.1:9450][cluster:monitor/nodes/info[n]]
Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [cluster:monitor/nodes/info[n]]
at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
'''

2

Hi @lschmid

It's a little tricky to know exactly what's wrong from that error message but it could be that you have xpack monitoring enabled but xpack is not available. There are a number of monitoring-related settings related to this which may be of use. I would try with setting xpack.monitoring.enabled to false and see if any improvement is observed. Here is a list of other possible settings which may be of use: https://www.elastic.co/guide/en/elasticsearch/reference/current/monitoring-settings.html#general-monitoring-settings

I can't say for certain whether or not this is the right path forward but it should give you a place to start, at least.

3

Hi @Mike_Place

Thanks for replying. I should have said in my previous post, I am running elasticsearch 5.6.7.

All xpack related settings have been deleted from the .yml file and a query to the _cluster/settings API shows no xpack settings.

4

All xpack related settings have been deleted from the .yml file and a query to the _cluster/settings API shows no xpack settings.

In my previous post I suggested xpack.monitoring.enabled to false. Did you try that?

5

I did, the cluster does not come up and the log shows the following error message:

[WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node] uncaught exception in thread [main]

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [xpack.monitoring.enabled] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

6

@jakelandis Do you have any suggestions here from the Elasticsearch side?

7

If xpack is fully removed then any of the xpack settings will not be available. I am not sure this is related to monitoring (despite the action name). That particular action could be used outside of monitoring collection (not sure on that code base).

It is suspect that you are getting an xpack exception in the stack trace
org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) since that is only possible if there is xpack code running. Are you sure you got xpack removed from all the nodes in the cluster ?

8

Are you sure you got xpack removed from all the nodes in the cluster ?

I am sure, no plugins are listed when I use the cat plugins API on either of the two nodes in the cluster

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.