Kibana keeps logging users out with message "An unexpected authentication error occurred. Please log in again."

Elastic Stack Kibana
1

I am running an Elastic Service v7.16.2, when I open a single tab or multiply tabs of Kibana, it unexpectedly logs the user out, sometimes after a minute or less. Other times it seems to be stable and keeps user logged in then bam! logged out again with error "An unexpected authentication error occurred. Please log in again."

This occurs with either using only the Kibana that comes with the Elastic Service or only the Kibana instance I have set up from scratch or both. I found a similar issue in the forum: Kibana keeps logging users out "An unexpected authentication error..." and followed its recommendation of Load balancing across multiple Kibana instances Use Kibana in a production environment | Kibana Guide [8.1] | Elastic.

I was able to configure the following setting successfully on the Kibana I have set up from scratch, but not on the provided Kibana from the Elastic Service.

I could not save the following setting in the Kibana from the Elastic service:

server.uuid
server.name
xpack.security.encryptionKey 
xpack.encryptedSavedObjects.encryptionKey 
xpack.encryptedSavedObjects.keyRotation.decryptionOnlyKeys

the error it gave when trying to save the Kibana user setting was the following:

Save configuration settings?

Your changes cannot be applied
Kibana - 'xpack.security.encryptionKey': is not allowed
Kibana - 'xpack.encryptedSavedObjects.encryptionKey': is not allowed
Kibana - 'server.uuid': is not allowed
Kibana - 'server.name': is not allowed

Set server.name to UNIQUE_NAME
Set server.uuid to 7b14df5e-4392-42e2-8aa1-bebc1c33a077
Set xpack.encryptedSavedObjects.encryptionKey to AVeryLongSecret12345678910
Set xpack.security.encryptionKey to AVeryLongSecret12345678910

Unsure where to go from this point, looking for some guidance.

Mike

2

We don't recommend connecting an on-prem Kibana instance to an ESS deployment, but it can be done.

Each Kibana instance doesn't need to have fully identical configurations -- in particular don't need to change the server.uuid or server.name in ESS.
However, you do need to have the same encryption keys and authentication providers configured across all your Kibana instances.

You can't change encryption keys in ESS (those are auto-generated and applied in a special way) but you can open a support ticket to ask for your deployment's key.
Once you have your ESS encryption key, you can use them in your local Kibana instance.

If you have saved objects that were encrypted with a different key (from your local Kibana instance), you'll need to do the following:

  1. Add your old encryption key to your local Kibana instance's xpack.encryptedSavedObjects.keyRotation.decryptionOnlyKeys setting
  2. Change your local Kibana instance's xpack.encryptedSavedObjects.encryptionKey setting to use your ESS encryption key
  3. Call the API on your local Kibana instance to manually rotate the encryption key (this will attempt to decrypt any existing saved objects and re-encrypt them with the new key)

Note, that's only necessary if you have set up Alerting rules / connectors, those are the only saved objects in 7.16 that use encryption.

Also, don't forget to set xpack.reporting.encryptionKey too.

1 Like
3

Thanks for the tip to open a support ticket to get those secrets.

Just to clarify, this issue came about by just using the supplied Kibana in my Elastic service. Before standing up my own Kibana instance, I've occur this issue when I've open several tabs in Chrome.

4

Hmm, if you can reproduce this issue even without your own on-prem Kibana instance would you mind enabling verbose logs (logging.verbose: true) and check if you see anything suspicious in the logs. If not then we'd need the logs + a HAR file from the browser for that same time frame to see what's going on.

5

Tried to enable verbose logging on provided the Kibana service and got this error:

Your changes cannot be applied

Kibana - Enhanced logging can only be used if the cluster has a target deployment set for log shipping. Either enable log shipping to use enhanced logging, or remove these logging settings from [kibana] configurations: [logging.verbose]

Set logging.verbose to true
6

Yeah, it has a few prerequisites explained here (you can use the same deployment as monitoring deployment just to simplify debugging): Enable logging and monitoring | Elasticsearch Service Documentation | Elastic

7

I've have asked support for the those settings and they said they do not give them out. Maybe I'm not asking or wording the question right.

Do you know in fact they are suppose to give those settings out upon request?

Mike

8

Mike:
Sorry for the trouble, I think there is some confusion on our end here.
I'm tracking this down internally and I will get back to you soon.

9

Hey Mike, just wanted to let you know this hasn't fallen off of my radar, waiting for some clarification, will get back to you soon.

10

I am in discussion with the support team and they are adamant that these key/values are something they do not share.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

12

We've had several discussions on this topic internally. Just to close the loop on this thread:

In the past we technically allowed connecting a local Kibana instance to an Elastic Cloud deployment in a variety of ways -- there were different ways to make this work, but this was not officially supported/documented and it was a pretty rare use case.

Moving forward, we are not in a position to support or allow connecting a local Kibana instance to an Elastic Cloud deployment. I apologize again for the confusion here.