Hi everyone! we would like to use ES operator in all our cluster to monitor kubernetes logs.
We have tried to get fleet and the agents working based on our scenario that looks the following:
We are trying to send the logs and metrics from all of our clusters (>10) to one central infrastructure cluster that saves all the data from all our clusters.
However it seems that Fleet and the Agents are not really suited to work over different Kubernetes clusters and use a central ES instance that is located in another cluster.
We have tried different setups already. Everything with ECK is not working as we cannot use fleetServerRef and kibanaRef across clusters. Then we tried to set up the fleet server (in the central cluster) with ECK and the Agents in the other clusters with a deployment and manually set the fleet enrollment token to connect it to the public exposed endpoints (through ingresses) of the fleet server and elasticsearch. We have also tried to manually setup both the fleet server and the elastic agent (in fleet mode) but we couldn’t find documentation on how to do this properly. We also tried to use standalone mode where we managed to send some data to the elastic instance but it seems that setup is missing a lot of the security features and the way to configure the agent and the integrations isn’t documented at all.
As you can see we have tried to find a way to get this to work, but at this point all our efforts so far to use a central ES instance, while agents are running on different Kuberenetes clusters are failing. Unfortunately there is also no documentation in the ES manuals that would describe such a centralized setup.
Is their any possibility to get such a setup to work? If yes how it can be possible? Thanks in advance.