Kibana pod won't start with fleet enabled in hyphenated namespace

I have an existing ES cluster for logging/SIEM and wish to enable fleet/agent. The cluster is deployed via ECK on kubernetes in namespace ‘elasticsearch-enterprise’ to differentiate it from our basic licensed clusters. The problem I’m running into when trying to enable Fleet/Agent is Kibana won’t start, complaining with the below error. Up to now the cluster has been running for around a year with no problems in this namespace.

FATAL][root] Reason: [config validation of [xpack.fleet].agentPolicies.0.namespace]: Namespace contains invalid characters

I see some older issues similarly throwing errors for invalid chars in namespaces but that was in UI and not on starting the Kibana service itself, and included specific references to the invalid characters and hyphen was not one of them: Error "Namespace contains invalid characters" is displayed while provide name in "Default namespace" with special characters #(hash) or *(asterisk) under policies page. · Issue #79818 · elastic/kibana · GitHub

Hello @smashley

As per my understanding the namespace “elasticsearch-enterprise” is at kuberentes level no where related to ELK. The error you have shared “FATAL][root] Reason: [config validation of [xpack.fleet].agentPolicies.0.namespace]: Namespace contains invalid characters” seems to be related to the namespace defined at Agentpolicy level.

Could you please check the kibana.yml if it has default preconfigured policy like below which can be causing the issue :

xpack.fleet.agentPolicies:

name: My Policy
id: my-policy
namespace: default-policy

Thanks!!

Yes, I had populated that value with the k8s namespace my cluster runs in. Is this not necessary? The documentation doesn’t elaborate much on what this value is used for, but if it ultimately doesn’t have anything to do with the k8s namespace, that is fine. I was able to work-around the issue by setting the xpack.fleet.agentpolicies.namespace to ‘elasticsearch’.