Elastic Agents, ECK, and Kubernetes namespaces

Elastic Stack Beats
1

I'm setting up elastic stack on our kubernetes cluster using ECK. One of the ideas is to make sure teams have access to the logs and metrics of workloads only from their kubernetes namespace.

This seem to map to the namespace concept from elastic agents.

I tried following this example and successfully deployed agents and fleet server: cloud-on-k8s/fleet-kubernetes-integration.yaml at master · elastic/cloud-on-k8s · GitHub

However, now I'm not sure how to further customize kubernetes agent package. Specifically, I was trying to add options like this (note that this is just me stumbling in the dark because I can't find documentation of different agent/fleet package options):

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
  namespace: elastic
spec:
  version: 7.15.2
  count: 1
  elasticsearchRef:
    name: elasticsearch
  config:
    xpack.fleet.agents.elasticsearch.host: "https://elasticsearch-es-http.elastic.svc:9200"
    xpack.fleet.agents.fleet_server.hosts:
      ["https://fleet-server-agent-http.elastic.svc:8220"]
    xpack.fleet.packages:
      - name: kubernetes
        version: latest
    xpack.fleet.agentPolicies:
      - name: Default Fleet Server on ECK policy
        is_default_fleet_server: true
        package_policies:
          - package:
              name: fleet_server
            name: fleet_server-1
      - name: Default Elastic Agent on ECK policy
        is_default: true
        unenroll_timeout: 900
        package_policies:
          - package:
              name: system
            name: system-1
          - package:
              name: kubernetes
            name: kubernetes-1
            inputs:
              - type: logfile
                data_stream.namespace: '${kubernetes.namespace}'
                streams:
                  - data_stream:
                      dataset: kubernetes.container_logs
                      type: logs
                    vars:
                      - name: paths
                        value: '/var/log/containers/*${kubernetes.container.id}.log'
                      - name: custom
                        value: |
                          symlinks: true

But this just gives the following error:

FATAL  Error: [config validation of [xpack.fleet].agentPolicies.1.package_policies.1.inputs.0.data_stream]: definition for this key is missing
  1. Is this something that's even possible to do, or am I misunderstanding something completely?
  2. Where can I find documentation of the available configuration options under xpack.fleet.agentPolicies.0.package_policies?
  3. It seems that the config I specify under xpack.fleet.agentPolicies applies only once and any changes I make in kibana.yml are ignored after this initial setup. Can this be fixed somehow?
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.