My problem is similar to below.
Failed to deploy fleet-server via eck in kubernetes.
My K8S Environment:
kubectl version command output is here.
clientVersion:
buildDate: "2022-08-23T17:44:59Z"
compiler: gc
gitCommit: a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2
gitTreeState: clean
gitVersion: v1.25.0
goVersion: go1.19
major: "1"
minor: "25"
platform: darwin/arm64
kustomizeVersion: v4.5.7
serverVersion:
buildDate: "2023-04-12T12:08:36Z"
compiler: gc
gitCommit: 49433308be5b958856b6949df02b716e0a7cf0a3
gitTreeState: clean
gitVersion: v1.24.13
goVersion: go1.19.8
major: "1"
minor: "24"
platform: linux/amd64
What works?
When I use the recipes : Configuration Examples | Elastic Cloud on Kubernetes [2.8] | Elastic. Everything beautifully works as expected. (this creates ServiceAccount/ClusterRole/ClusterRoleBinding/Kibana/Elasticsearch/Elastic-agent in Fleet Mode (deployment) and elastic-agent (in demon set)
What does not work?
We have a very well working dev/production environment setup in our K8S environment
Elasticsearch/kibana/logstash/filebeat/metricbeat/ElasticAPM and it works very well.
Target:
We want to move to Fleet. And we are not able to move to Fleet (existing kibana/elasticsearch clusters)
So as a part of this: I created ClusterRole/ServiceAccount/ClusterRoleBinding.
- wanted to start elastic-agent in Fleet mode (for fleet server)
- made changes in kibana (xpack.fleet) as per the recipies (including policies).
Further changes/additions (running elastic agent in daemonset)- withheld.
Expectation:
Elastic-Agent in Fleet mode is supposed to establish connection to kibana (kibanaRef) and elasticsearch (elasticsearchRef). And then start the fleet-server.
What happens:
Elastic_agent in fleet mode
- establishes connection to elasticsearch (same cluster/ same namespace)
- Not able to establish connection to kibana (same clsuter/ same namespace).
What has been tried?
Increase the log level/ verbosity in ECK to manage only my namespace for some hints.
Not able to make out much.
When I go to kibana, and start fleetserver .. it does not go beyond step 3.
Status of Agent/ Fleet is like below:
metadata:
annotations:
association.k8s.elastic.co/es-conf-1095366406: >-
{"authSecretName":"fleet-server-elksdev-elk-dev-agent-user","authSecretKey":"token","isServiceAccount":true,"caCertProvided":true,"caSecretName":"fleet-server-agent-es-elksdev-elk-dev-ca","url":"https://elk-dev-es-http.elksdev.svc:9200","version":"8.5.2"}
### removed some ###
status:
elasticsearchAssociationsStatus:
elksdev/elk-dev: Established
kibanaAssociationStatus: Pending
observedGeneration: 15
spec:
deployment:
podTemplate:
metadata:
creationTimestamp: null
spec:
automountServiceAccountToken: true
containers: null
securityContext:
runAsUser: 0
serviceAccountName: elastic-agent
replicas: 1
strategy: {}
elasticsearchRefs:
- name: elk-dev
fleetServerEnabled: true
fleetServerRef: {}
http:
service:
metadata: {}
spec: {}
tls:
certificate: {}
kibanaRef:
name: kibana-dev
mode: fleet
policyID: eck-fleet-server
version: 8.5.2
In successful 'installation / recipe provided by elasticsearch', in metadata of fleet-server I get associations for both the elasticsearch and kibana. And in my installation I get only metadata of fleet-server
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
annotations:
association.k8s.elastic.co/es-conf-1095366406: >-
{"authSecretName":"fleet-server-elksdev-elk-dev-agent-user","authSecretKey":"token","isServiceAccount":true,"caCertProvided":true,"caSecretName":"fleet-server-agent-es-elksdev-elk-dev-ca","url":"https://elk-dev-es-http.elksdev.svc:9200","version":"8.5.2"}
What Help is needed?
In the recipie installation of kibana/elastic/fleet-server, things work
When we want to introduce fleet server, things do not work (association not found to kibana)
What should we do OR how should we proceed.
