Confused about ilm, index templates and new indices

Elastic Stack Beats
,
1

I have a serve where I install some beats (e.g. packetbeat) but I don't enable the systemd service. I only use packetbeat setup --index-management on this server so that the packetbeat index template and the ILM policy get inserted into elasticsearch. This works fine, I can see the index template and ILM policy in Kibana. These are the settings I use:

setup:
  ilm:
    check_exists: false
    enabled: true
    overwrite: true
    pattern: '{now/d}-000001'
    rollover_alias: packetbeat-%{[agent.version]}
  template:
    enabled: true
    overwrite: true
    pattern: '%{[agent.name]}-%{[agent.version]}-*'
    type: legacy

On a second server, I install packetbeat with the following settings:

setup:
  ilm:
    check_exists: false
    enabled: true
    overwrite: false
    pattern: '{now/d}-000001'
    rollover_alias: packetbeat-%{[agent.version]}
  template:
    enabled: false

On the second server I enable the packetbeat service. I can see logs being sent to elasticsearch. But there are some problems and some questions that I have:

  • The index packetbeat creates is packetbeat-7.12.1. This does not match the template pattern that has a dash at the end: packetbeat-7.12.1- and ILM is not enforced neither are the template settings used.
  • If I enable the ILM policy, then I can't change the setup.template.pattern in the first server.
  • If I set setup.template.enabled: true on the second server, then the index template gets used and ILM is enabled.

However, I cannot understand why I need to set setup.template.enabled: true to the second packetbeat for the whole chain to workk. This setting seems like a setting that is used on beat startup and instructs it whether to push the tempate to elasticsearch or not. At least, this is the case with the setup.ilm.enabled and setup.dashboards.enabled settings. My approach was to disable templating loading on the second server, because the elastic user that packetbeat uses in that installation does not have permissions to manage index templates.

Still, setup.template.enabled seems to affect the name of the index that packetbeat will push to. If this is correct and intended, it should be documented in the packetbeat.reference.yml at least.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.