Hello,
I'd like to enable just a few filebeat modules (let's say nginx, postgresql). And I'd like to have on Elasticsearch one index per module (per month). So... filebeat-nginx-2022-01 and filebeat-postgresql-2022-01. Also, I'd like the index template for these indexes to only include the fields the index will contain -- so only the nginx fields for filebeat-nginx for instance.
I don't want ILM.
I would have actually expected filebeat to include in the index template it generates only the fields from the modules I've enabled, but instead of that, I currently get fields for all the modules that filebeat supports.
I did all my changes in /etc/filebeat/filebeat.yml; I have tried disabling ILM, using %{[event.module]:other} and %{[fields.type]:logs} patterns in my index name, also listing multiple indices in filebeat.yml, with conditions like:
when.equals:
event.module: "nginx"
But no matter what I do, I always end up with one index with all the filebeat-supported fields in the index template.
This is my current filebeat.yml file:
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
filebeat.config.inputs:
enabled: true
path: inputs.d/*.yml
output.elasticsearch:
hosts: ["xxxx:9200"]
# index: "filebeat-%{[fields.type]:logs}-"
# index: "filebeat-%{[agent.version]}-%{[event.module]:other}-%{+yyyy.MM.dd}"
indices:
- index: "filebeat-nginx-%{+yyyy.MM.dd}"
when.equals:
event.module: "nginx"
- index: "filebeat-system-%{+yyyy.MM.dd}"
when.equals:
event.module: "system"
protocol: "https"
setup.template.name: "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.template.pattern: "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.ilm.enabled: false
ilm.enabled: false
I tried all the things I could find from the docs and various forums and google results, and now I'm out of ideas. What am I missing?