How to Filter Desired Container Logs in Docker Integration in Fleet？

[XYYYYY (X541443901)]

Hello everyone, I am a newcomer using elastic agent. I have tried to collect container logs using Docker integration, but I have a wide variety of container logs. I only want to obtain a few of them. How can I achieve this?

I am currently trying to use the Condition option in Collect Docker Container Logs to set the container filter I want, but I am not sure how to input to filter out the container I want, or use additional Processors. I hope you can answer. Thank you.

[XYYYYY (X541443901)]

The current situation is that I am trying to use wildcards or regular expressions to implement this function, because I can obtain the container name I want through container.image.name. The names of the container logs I want to collect start with a fixed format (ed) , so I wonder if this can be done, and how to achieve it?

[stephenb (Stephen Brown)]

Hi @XYYYYY Welcome to the community!

Please take a look at this...

EDIT Fixed Image

See how conditions work here with specific examples here.

and what variables the k8s provider populates here

As far as I can tell the condition does not support Regex but it supports a lot, if you can give some samples / examples perhaps we can help

Condition syntax

The conditions supported by Elastic Agent are based on EQL's boolean syntax, but add support for variables from providers and functions to manipulate the values.

Supported operators:

• Full PEMDAS math support for + - * / %.
• Relational operators < <= >= > == !=
• Logical operators and and or

Functions:

• Array functions arrayContains
• Dict functions hasKey (not in EQL)
• Length functions length
• Math functions add, subtract, multiply, divide, modulo
• String functions concat, endsWith, indexOf, match, number, startsWith, string, stringContains

[XYYYYY (X541443901)]

Thank you very much for your reply! which made me feel the good atmosphere and friendliness of this community. I will read it carefully and hope it can solve my problem

[XYYYYY (X541443901)]

Please take a look at my configuration here. If I only want to obtain the container log with the image name beginning with "ed" in the docker, and whether this writing is correct, I observe that it does not seem to be effective

Perhaps I shouldn't use stringContains (${docker. name}, 'ed')

Should use arrayContains (${docker. name}, 'ed *')？

[XYYYYY (X541443901)]

I have noticed that there is a time difference between us. I am currently working in the afternoon. Due to work restrictions, I can only log in to the community during working hours. I hope you can understand. Thank you very much

This is my current docker filestream configuration

    "docker-filestream": {
      "enabled": true,
      "streams": {
        "docker.container_logs": {
          "enabled": true,
          "vars": {
            "condition": "stringContains(${docker.name}, 'ed')",
            "paths": [
              "/var/lib/docker/containers/${docker.container.id}/*-json.log"
            ],
            "additionalParsersConfig": "# - ndjson:\n#     target: json\n#     ignore_decoding_error: true\n# - multiline:\n#     type: pattern\n#     pattern: '^\\['\n#     negate: true\n#     match: after\n",
            "containerParserStream": "all",
            "processors": "- rename:\r\n    fields:\r\n      - from: \"container.image\"\r\n        to: \"container.image.name\"\r\n    ignore_missing: true\r\n    fail_on_error: false"
          }
        }
      }
    }