after upgrade from 8.9.1 to 8.11.1 , I have error on fleet page "unable to initialize fleet, uninstall token is missing the token"
Hi @mohd_sa im also facing same issue, any luck?
i did not find any solution
so whtas the next move? just asking. We got this error because kibana instance got replaced as a part of autoscaling group health check and it resulted with the error uninstall token missing when clicked on fleet tab in kibana interface. sad part is elk documentation also doesnt have any clue of this.
Hi folks, We've seen this reported in a few places so far on the 8.11.1 release and we're taking a look. Once we have a public issue or know a little bit more about the root cause I'll post an update here.
One thing that'd be helpful to know: does anyone encountering this issue make use of Kibana's saved objects encryption feature? The errors message we're seeing here is related to saved object encryption, and we're trying to track down the root cause.
Many thanks, and I hope to have an update for you all soon.
All my rules (under Alerts and Insights) were generating api key errors
I disabled them and did Update Api Key for all of them.
After that I did not have any api key errors anymore.
But fleet is still is still giving
My environment is on Openshift
The kibana.yml is stored in a secret and has
Thanks @HansPeterSloot - this is very helpful as it appears to confirm this is a broader issue with encrypted saved objects that isn't limited to Fleet. We've got a few teams looking at this internally today.
@kpollich is there any public issue on github about this?
We were planning to upgrade to 8.11, but after seeing that this can cause issues with Alert Rules, which is our main use case we can not upgrade anymore and risk any impacts to our operations.
Would like to follow some issue to know when it is safe to Upgrade.
@leandrojmp - I've just created [Encrypted Saved Objects] Multiple plugins experiencing errors with encrypted saved objects in 8.11.1 · Issue #171630 · elastic/kibana · GitHub to track this publicly. I'll keep this updated with our findings as we work, so please feel free to follow along there
Question for folks here: is anyone using Elastic Cloud on Kubernetes (Overview | Elastic Cloud on Kubernetes [2.10] | Elastic), or are these all self-hosted clusters?
We've seen this reported in support cases internally on ECK, and I see Hans's example above is on OpenShift (so similar K8's concerns), and I'm trying to determine whether these are related.
Many thanks. We have 3-4 engineering teams all looking at this urgently today and I will continue to provide updates here and on the public issue.
If we want to proceed to normal operations isnt reverting back (from a restore) to 8.10 a possible way forward until this is resolved. As it does not look like this is an easy problem (wrong encryption key in a new software release?)
It's not clear to me whether rolling back to a previous version would fix the issue, but it's certainly worth a try. I'd be curious to know if you're running on ECK, and if you could provide any Kibana logs from before and after the upgrade.
I've posted a lot of findings and details over on [Encrypted Saved Objects] Multiple plugins experiencing errors with encrypted saved objects in 8.11.1 · Issue #171630 · elastic/kibana · GitHub today, for anyone following along.
@HansPeterSloot could you provide these ?
We are running on Openshift.
I do not have any logs from before.
I am now restoring a snapshot from before the upgrade on a separate cluster.
Hi all, I've posted a workaround to unblock Fleet in the instance of this error here: [Encrypted Saved Objects] Multiple plugins experiencing errors with encrypted saved objects in 8.11.1 · Issue #171630 · elastic/kibana · GitHub.
We've validated this workaround internally and in production for a few users, but please let me know if you run into any issues with it.
As far as a root cause goes, I'm afraid I don't have much news there. We're still working with a few teams at trying to track down the root cause here and why this happened specifically during the 8.11 release cycle. I will post an update as soon as I have one.
Thanks for your patience