Filebeat 8.17.10 disable template data_stream

Elastic Observability Logs
1

Hi

I have filebae version 8.17.10 running on k8s

And send logs to elasticsearch with the same version

This is my configuration

filebeat:
inputs:

  • type: filestream
    id: vouchers-logs-stream
    paths:
    • /path/to/logs/*.log
      json:
      keys_under_root: true
      add_error_key: true
      overwrite_keys: true
      message_key: message
      parsers:
    • ndjson:
      target: ""
      add_error_key: true
      output:
      elasticsearch:
      hosts: [ "..." ]
      username: "..."
      password: "..."
      index: voucher-app-logs-%{[agent.version]}-%{+yyyy.MM.dd}
      setup:
      template:
      name: "voucher-app-logs"
      pattern: "voucher-app-logs*"
      overwrite: false
      ilm:
      enabled: true
      policy_name: "voucher-app-logs-lifecycle-policy"

When I start it I see in kibana index management the created template enabled data stream i want to disable it can i do it from filebeat configuration not from kibana ?

2

Hi @almteref

I don't think you can do exactly what you want to do through filebeat, if I understand you wanted filebeat to generate the template, but you want to change the data stream setting. I don't think you can do that..

You can load your own template if you like see here

3

What i want to create index by the filebeat with this name : events-listners-k8s-pp-2025.11.18-00001

And apply the ilm and template

This works in the elasticsearch and filebeat version 7.x

But with version 8.x not working now , in ilm there is no pattern setting %{now/d}-00001

4

Yes I pretty sure I know it you want...

No pretty sure you're not going to be able to do it. How you want... Metricbeat defaults to data streams now

You can also just set up a data stream and specify the ILM it to roll over everyday AND the backing index will have the same format.

Or just load your own template with your own automaton.

Load your own index template

To load your own index template, set the following options:

setup.template.name: "your_template_name"
setup.template.fields: "path/to/fields.yml"

If the template already exists, it’s not overwritten unless you configure Filebeat to do so.

You can load templates for both data streams and indices

You could also just load a template via the REST API with your own automation.

That is my suggestion at this point.