Configuration sharing

[speechkey]

Hi folks,

we have one installation of Elasticsearch & Kibana and multiple teams which logs to it. We deploy our services as Docker containers and each of the Docker hosts has a Filebeat container which ships the logs from other containers to Elasticsearch.

team 1:
  service a
  service b

team 2:
  service a
  service c

We are using filebeat.autodiscover to provide the configuration for each container:

filebeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition.and:
          - contains:
              docker.container.image: service.a
          config:
            - type: container
              paths:
                 ...
        - condition.and:
          - contains:
              docker.container.image: service.b
          config:
            - type: container
              paths:
                ...

Each team uses own filebeat config file. There are services of the same type (same log structure) which are used by different teams, so if team 1 improves the filebeat configuration of the service a, it has to notify all other teams, which use services a, so they copy the improved configuration in their own.

The process we have, has a lot of overhead. Is there any better solution?

Best regards
-s

[Marius_Iversen]

I think this is one of the "problems" that can have many different solutions and could try to provide one or more alternatives at least, all depending on what you want to do.

If you want to share the filebeat configuration between teams, which seems to be a good option, as long as none actually make changes that would break another teams setup, then I would focus on using conditions based on labels, and ask each team to pick a label, like team1, team2 etc.

That way you can have this list of conditions, and at the top there is one condition per team label, and any condition under it could for example be specified by each team individually, so that they don't impact eachother.

An example on label based conditions:

Filebeat Autodiscovery: Filtering by container labels

Hi, I would like to set up Filebeat configuration with docker autodiscovery provider to create prospectors only for docker containers with certain label, e.g., filebeat.enable: "true". However I find it difficult to find the correct condition format to achieve this. I have tried the following config, but it does not seem to match any docker events: filebeat.autodiscover: providers: - type: docker templates: - condition: equals: docker.container.la…

Is this pure docker or is it running on for example k8?

[speechkey]

Hi Marius,

thank you for a reply. We are using both pure docker and k8s. If I understood correctly your idea, you would have one large filebeat.yml configuration file, where all team collaborate. We had this setup for a while, but it becomes large and confusing.

[legoguy1000 (Alex)]

I would use some sort of git server to track the different configs and use a ci/cd service to handle to changes. You could do a single file that is used by everyone or separate files per service that gets combined on ci/cd pipelines...