Elastic Agents, ECK, and Kubernetes namespaces

I'm setting up elastic stack on our kubernetes cluster using ECK. One of the ideas is to make sure teams have access to the logs and metrics of workloads only from their kubernetes namespace.

This seem to map to the namespace concept from elastic agents.

I tried following this example and successfully deployed agents and fleet server: cloud-on-k8s/fleet-kubernetes-integration.yaml at master · elastic/cloud-on-k8s · GitHub

However, now I'm not sure how to further customize kubernetes agent package. Specifically, I was trying to add options like this (note that this is just me stumbling in the dark because I can't find documentation of different agent/fleet package options):

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
  name: kibana
  namespace: elastic
  version: 7.15.2
  count: 1
    name: elasticsearch
    xpack.fleet.agents.elasticsearch.host: "https://elasticsearch-es-http.elastic.svc:9200"
      - name: kubernetes
        version: latest
      - name: Default Fleet Server on ECK policy
        is_default_fleet_server: true
          - package:
              name: fleet_server
            name: fleet_server-1
      - name: Default Elastic Agent on ECK policy
        is_default: true
        unenroll_timeout: 900
          - package:
              name: system
            name: system-1
          - package:
              name: kubernetes
            name: kubernetes-1
              - type: logfile
                data_stream.namespace: '${kubernetes.namespace}'
                  - data_stream:
                      dataset: kubernetes.container_logs
                      type: logs
                      - name: paths
                        value: '/var/log/containers/*${kubernetes.container.id}.log'
                      - name: custom
                        value: |
                          symlinks: true

But this just gives the following error:

FATAL  Error: [config validation of [xpack.fleet].agentPolicies.1.package_policies.1.inputs.0.data_stream]: definition for this key is missing
  1. Is this something that's even possible to do, or am I misunderstanding something completely?
  2. Where can I find documentation of the available configuration options under xpack.fleet.agentPolicies.0.package_policies?
  3. It seems that the config I specify under xpack.fleet.agentPolicies applies only once and any changes I make in kibana.yml are ignored after this initial setup. Can this be fixed somehow?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.