Filebeat Index Pattern Creation

[Eric_Orcutt]

I'm attempting to create a custom index using ILM policy through Filebeat and everything appears to be fine except that the Index Pattern created in Kibana by Filebeat is not using the custom pattern that I'm providing in my configuration:

filebeat.modules:
- module: traefik
  access:
    enabled: true

output.elasticsearch:
  hosts: ["tasks.elasticsearch:9200"]
  worker: 2
  index: "traefik-%{[agent.version]}"

setup.dashboards.enabled: true
setup.dashboards.index: "traefik-*"

setup.template.enabled: true
setup.template.name: "traefik"
setup.template.pattern: "traefik-*"
setup.template.overwrite: true

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "traefik"
setup.ilm.pattern: "{now/d{MM.dd.yyyy|America/New_York}}-000001"
setup.ilm.policy_name: "TraefikLogPolicy"
setup.ilm.policy_file: "/usr/share/filebeat/traefik.ilm.policy.json"
setup.ilm.overwrite: true

setup.kibana:
  host: "kibana:5601"

logging.level: info

monitoring.enabled: true
monitoring.elasticsearch:
  hosts: ["tasks.elasticsearch:9200"]

The index creates without issue (using rollover pattern and attached to ILM policy), the dashboards even update to the correct pattern ("traefik-*"). However, Kibana doesn't show the "traefik-*" index pattern but instead always creates a "filebeat*" pattern instead. What am I missing in my config to properly set the index pattern automatically in Kibana?

I'm currently using Docker 19.03.2 (Swarm) and Elastic 7.3.2

[Eric_Orcutt]

Anyone see an issue in my config on this? Is this a bug?

[Kaiyan_Sheng]

When you create index pattern in Kibana,

Could you try add the custom index pattern ID under the advanced options? I remember that solved the problem at one point for me.

[Eric_Orcutt]

Thanks for the response!

Just curious why there isn't a way to set this via the configuration though. Having to manually add/modify something that's already being created seems silly when 90% of the index configuration and ILM policy is being created at run-time.

Also, I'm not even certain this will work unless I delete the automatically created "filebeat-*" index pattern and re-create it. The "filebeat-*" index pattern that is created at run-time throws an error when I attempt to drill-down into it.

[Eric_Orcutt]

This does fix my dashboards but I'm really confused why this isn't something I can set in the filebeat.yml at run-time, or am I missing something?