Docker integration - condition not working

Elastic Stack Elastic Agent
1

Hi all, thanks for amazing product. I'm trying to set condition for docker logs ingestion and I'm not having success. I always get some kind of error, see below.

Please help me enable conditional logs based on label.

Using condition arrayContains(${docker.labels}, "filebeat_enable")
I get the following error:

$ elastic-agent inspect
Error: error printing config: error parsing config as hashmap: error unpacking subarray in config for skip key inputs: expected ',' when parsing 'arrayContains(${docker.labels}, "filebeat_enable")' accessing 'inputs.5'

For help, please see our troubleshooting guide at https://www.elastic.co/guide/en/fleet/8.17/fleet-troubleshooting.html

Trying condition arrayContains(${docker.container.labels}, "filebeat_enable") I get error

applying new policy: generating component model: rendering inputs failed: condition "arrayContains(${docker.container.labels}, "filebeat_enable")" evaluation failed: arrayContains: first argument must be an array; received map[string]interface {}
2

Hi @dot-mike

Welcome to the community.

It is unclear where you are trying to set that condition...

Elastic Agent Integration? ... Filebeat?

Auto discovery?

What version?

Show us exactly where you are trying to set that condition?

Show us what your current configuration is.

Then perhaps someone can help

3

Hi and thank you for the reply @stephenb
I'm trying to set this condition within Agent Policy for Docker integration.

image
image1319×417 38.1 KB

YAML output of this is (got this from Kibana web-ui running elastic-agent inspect fails as noted in OP)

  - id: filestream-docker-4eaf178e-9601-4e1d-9a76-1b5265f5ebad
    name: docker-1
    revision: 19
    type: filestream
    use_output: default
    meta:
      package:
        name: docker
        version: 2.14.0
    data_stream:
      namespace: default
    package_policy_id: 4eaf178e-9601-4e1d-9a76-1b5265f5ebad
    streams:
      - id: 'docker-container-logs-${docker.container.name}-${docker.container.id}'
        data_stream:
          dataset: docker.container_logs
          type: logs
          elasticsearch:
            dynamic_dataset: true
            dynamic_namespace: true
        paths:
          - '/var/lib/docker/containers/${docker.container.id}/*-json.log'
        condition: 'arrayContains(${docker.labels}, "filebeat_enable")'
        parsers:
          - container:
              stream: all
              format: docker
4

Perhaps try per the example

arrayContains(${docker.labels}, 'filebeat_enable')

Single quotes into the UI

Also looking at the docker provider

It looks like the provider variable is

docker.container.labels

Not

docker.labels

So perhaps

arrayContains(${docker.container.labels}, 'filebeat_enable')

That is probably the issue.. always need to look at the provider... But the example should be fixed

5

Hello,

I tried arrayContains(${docker.labels}, 'filebeat_enable') with single quotes and when I execute elastic-agent inspect I see

Error: error printing config: error parsing config as hashmap: error unpacking subarray in config for skip key inputs: expected ',' when parsing 'arrayContains(${docker.labels}, 'filebeat_enable')' accessing 'inputs.5'
For help, please see our troubleshooting guide at https://www.elastic.co/guide/en/fleet/8.17/fleet-troubleshooting.html

Also tried arrayContains(${docker.container.labels}, 'filebeat_enable') and gives this error in logs:

[elastic_agent][error] applying new policy: generating component model: rendering inputs failed: condition "arrayContains(${docker.container.labels}, 'filebeat_enable')" evaluation failed: arrayContains: first argument must be an array; received map[string]interface {}

So I'm not having any luck with the examples.

Environment info:

  • Elastic Agent version: 8.17.5
  • Docker integration: 2.14.0
6

Hmm I don't know...

I would take the condition out... collect some logs and look at the labels etc... and see what fields are actually there then comeback and try the condition, that is what I would do.

wonder if you only have 1 label of you should just try a simple string compare.