One index/index template per module


I'd like to enable just a few filebeat modules (let's say nginx, postgresql). And I'd like to have on Elasticsearch one index per module (per month). So... filebeat-nginx-2022-01 and filebeat-postgresql-2022-01. Also, I'd like the index template for these indexes to only include the fields the index will contain -- so only the nginx fields for filebeat-nginx for instance.
I don't want ILM.

I would have actually expected filebeat to include in the index template it generates only the fields from the modules I've enabled, but instead of that, I currently get fields for all the modules that filebeat supports.
I did all my changes in /etc/filebeat/filebeat.yml; I have tried disabling ILM, using %{[event.module]:other} and %{[fields.type]:logs} patterns in my index name, also listing multiple indices in filebeat.yml, with conditions like:

        event.module: "nginx"

But no matter what I do, I always end up with one index with all the filebeat-supported fields in the index template.

This is my current filebeat.yml file:

  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

  enabled: true
  path: inputs.d/*.yml

  hosts: ["xxxx:9200"]
  # index: "filebeat-%{[fields.type]:logs}-"
  # index: "filebeat-%{[agent.version]}-%{[event.module]:other}-%{+yyyy.MM.dd}"
    - index: "filebeat-nginx-%{+yyyy.MM.dd}"
        event.module: "nginx"
    - index: "filebeat-system-%{+yyyy.MM.dd}"
        event.module: "system"

  protocol: "https" "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.template.pattern: "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.ilm.enabled: false
ilm.enabled: false

I tried all the things I could find from the docs and various forums and google results, and now I'm out of ideas. What am I missing?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.