JSON ingestion for logs picked up elastic fleet agent ECK operator

Sagar_Gulabani
Wed, 20 Jul 2022 13:54:22 +0000
Elastic Cloud on Kubernetes (ECK)

Hi, We are using the elastic operator for Kubernetes.
We are using the fleet managed elastic agent
We are giving the fleet agent configuration using the kibana CRD as show below.
Our logs from our applications are JSON logs following the ECS format.
But they end up in elastic as escaped strings.
How do we add json processor to the configuration below such that they end up as JSON in elasticsearch ?
Thank you

    xpack.fleet.packages:
      - name: system
        version: latest
      - name: elastic_agent
        version: latest
      - name: fleet_server
        version: latest
      - name: apm
        version: latest
      - name: log
        version: latest
    xpack.fleet.agentPolicies:
      - name: Fleet Server on ECK policy
        id: eck-fleet-server
        is_default_fleet_server: true
        namespace: default
        monitoring_enabled:
          - logs
          - metrics
        package_policies:
        - name: fleet_server-1
          id: fleet_server-1
          package:
            name: fleet_server
      - name: Elastic Agent on ECK policy
        id: eck-agent
        namespace: default
        monitoring_enabled:
          - logs
          - metrics
        unenroll_timeout: 900
        is_default: true
        package_policies:
          - name: system-1
            id: system-1
            package:
              name: system
          - package:
              name: log
            name: log-1
            inputs:
            - type: logfile
              enabled: true
              streams:
              - data_stream:
                  dataset: log.log
                enabled: true
                vars:
                - name: paths
                  value:
                  - '/var/log/containers/*${kubernetes.container.id}.log'
                - name: custom
                  value: |
                    symlinks: true
                    condition: ${kubernetes.namespace} == 'default'

2 posts - 2 participants