Index Templates: One or more for an intergration?

Elastic Stack Beats
1

I have deployed Elastic agents to few windows hosts and are managed by fleet.

I need to create a custom index template so I can change the ILP based on my custom namespace. So for example for the System integration there are several modules within it and its has it's own index template:

  • system.auth
  • system.application
  • system.security
  • etc

So my question is, should I create a custom index template for each module or should I just create one? (one for each integration)

The problem with the multiple ones is that for many integrations and many namespaces the templates are a lot.

2

This is a long standing issue with customization with Elastic Agent integrations, you need to create a custom template for each dataset in each integration.

So for the system integration, you will need to use the @custom component template for the auth dataset, another for the application dataset etc.

There is an issue to solve this and add more levels of customization, like using a custom template for integration, but it is still open: [Fleet] Add support for customizing integration data streams at more levels of granularity · Issue #149484 · elastic/kibana · GitHub

But your case, if you want to customize the retention per namespace, then it is a lot more of work.

The @custom component templates applies to the dataset in all namespaces, if you want to haver different policies for the same dataset but on different namespaces, then you need to follow this documentation: Tutorial: Customize data retention policies | Fleet and Elastic Agent Guide [8.17] | Elastic

1 Like
3

Thanks, I see the issue open that would greatly help us in the long run.

One question though. I created my component template(that defines the ILP) and I add that to each Index template that I want to apply it. Is this much different that using the @custom template?

Thanks

4

I'm not sure.

I use the built-in @custom template the applies at the dataset level, not per namespace.

For customization per namespace you need to follow the steps on the linked documentation.