Nomad filebeat autodiscover seems to ignore template's condition

Elastic Stack Beats
1

Hi,
We have this setup

filebeat.autodiscover:
  providers:
    - type: nomad
      node: {{ env "node.unique.name" }}
      scope: node
      address: <nomad-cluster-address>
      templates:
        - condition:
           not:
              contains:
                nomad.task.name: filebeat
          config:
            - type: log
              paths:
                - /node-logs/$${data.nomad.allocation.id}/alloc/logs/$${data.meta.nomad.task.name}.stdout.*
                - /node-logs/$${data.nomad.allocation.id}/alloc/logs/$${data.meta.nomad.task.name}.stderr.*

As I understand, this should prevent filebeat from harvesting the logs from the tasks containing 'filebeat' in their names.

However, when I take a look at the filebeat registry, I see a lot of log files from those tasks. Like

{"_key":"filebeat::logs::native::4386819-66305","prev_id":"","source":"/node-logs/08599431-b9c8-21c0-5fca-33a82fbaf234/alloc/logs/filebeat.stdout.0","timestamp":[754348773,1703599823],"ttl":-1,"id":"native::4386819-66305","offset":9,"type":"log","FileStateOS":{"inode":4386819,"device":66305},"identifier_name":"native"}

Am I excluding these logs in a wrong way?
Thanks!

2

Hi @Yuri_Nakshin Welcome to the community...

Not sure if just cut-n-paste or if this is the issue...

      templates:
        - condition:
           not: << Here indented incorrectly
              contains:
                nomad.task.name: filebeat

Should be

      templates:
        - condition:
            not: 
              contains:
                nomad.task.name: filebeat

Unfortunately I do not have a Nomad cluster to test on.
Nomad is also tech preview so be carefull support / functionality could change.

3

Thanks for your reply, @stephenb
This must be a cut and paste issue because the config is a valid yaml digested and used by our filebeat without any complains. I just checked our source: "not"'s n is directly under "condition"'s "n" :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.