Filebeat: Autodiscover with Nomad + Docker Metadata

Elastic Stack Beats
,
3

I found a solution going the other "direction":

  1. instruct the nomad docker plugin to write to journald
plugin "docker" {
    config {
        logging {
          type = "journald"
          config { }
        }
  1. Collect the container logs from the journal
   type: journald
   id: docker.service
   include_matches:
     - _SYSTEMD_UNIT=docker.service
   processors:
     - drop_event:
         when:
           not:
             has_fields: ['container.id']
     - add_docker_metadata:
          host: "unix:///var/run/docker.sock"
          match_fields: ["container.id"]  # this is the container.id that systemd provided
          match_source: false
  1. Enrich the event with the nomad meta data
  - add_nomad_metadata:
      address: http://localhost:4646
      default_indexers.enabled: false
      default_matchers.enabled: false
      node: mynode
      indexers:
        - allocation_uid: # this typo is on purpose, see https://github.com/elastic/beats/issues/29743
       matchers:
        - fields:
            lookup_fields:
              - "container.labels.com_hashicorp_nomad_alloc_id"

et voila we have an event that contains both the docker and nomad metadata.