Kibana not starting after making service account changes for upgrade to 8


Running into an issue upgrading to Elasticsearch/kibana 8. The upgrade assistant had me switch from username/password authentication to service accounts via: Service accounts | Elasticsearch Guide [8.0] | Elastic

I've gone through the process of setting up my service token for elastic/kibana on the node (server_1) that also has kibana and added to my kibana.yml. I'm running into this error:

[server_2] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

As well as this one which suggests a conflict with the default authentication and this new service account:

FATAL Error: [config validation of [Elasticsearch].serviceAccountToken]: serviceAccountToken cannot be specified when "username" is also set.

do I need to create a service token on Elasticsearch node then add that entry to the kibana.yml file. Something like this:

elasticsearch.serviceAccountToken: token_for_server_1
elasticsearch.serviceAccountToken: token_for_server_2
elasticsearch.serviceAccountToken: token_for_server_3

Any help is appreciated.


I looked at the Kibana 8.0 configuration doc page. It seems like Kibana has only one Elasticsearch.serviceAccountToken setting. Also, it seems like that setting can't be mixed with Elasticsearch.username and Elasticsearch.password. Is it possible your configuration has both types of settings?

In general, I think your Kibana node expects to use one service token to connect to all Elasticsearch nodes in a cluster. Is the same service token configured in both Elasticsearch nodes?

Here is the link where I found that information, in case it helps.

Here is the specific excerpt I read.

If your Elasticsearch is protected with basic authentication, this token provides the credentials that the Kibana server uses to perform maintenance on the Kibana index at startup. This setting is an alternative to elasticsearch.username and elasticsearch.password .

That would probably the reason for the error. The other node is not setup with the service account. But how do you set the same token key across the cluster? Its a randomly generated key when you run the elasticsearch-service-tokens cli utility. I don't see a way to copy that token across other nodes: Service accounts | Elasticsearch Guide [8.0] | Elastic


The CLI creates a service token in a file, which is not replicated.
The API creates a service token in an index, which is replicated.

The URL method and path are of the form:
POST /_security/service/<namespace>/<service>/credential/token/<token_name>

Perhaps one of these curl examples would work:
$ curl -s -X POST -u elastic:PASSWORD "ADDRESS:9200/_security/service/elastic/kibana/credential/token/token1"

$ curl -s -X POST -H 'Authorization: Bearer TOKEN' "ADDRESS:9200/_security/service/elastic/kibana/credential/token/token1"

Here is a link to the doc page if that is helpful.

Unfortunately, the Kibana UI incorrectly recommends using the CLI to setup service tokens.
That has been fixed in an upcoming release

1 Like

We're also working on improving the Elasticsearch docs around this

Thanks that solved the issue!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.