# Change data stream name in filebeat 8.\*

**URL:** <https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531>\
**Category:** Beats\
**Tags:** ilm-index-lifecycle-management, filebeat, datastreams\
**Created:** [April 4, 2022, 3:09pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531 "2022-04-04T15:09:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andres\_Altamirano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andres_altamirano/32/94143_2.png) [@Andres\_Altamirano](https://discuss.elastic.co/u/Andres_Altamirano)\
**Post date:** [April 4, 2022, 3:09pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/1 "2022-04-04T15:09:49Z")

</div>

We used to store filebeat data from different sources in a different index due to storage size and document category using different ILM policies.

Now we can't do that because if we use ILM, we can't change the index name, so now all documents store on the same index and with the same ILM policy. That's a problem because there are policies with 1 week and others with a year that mage huge difference when it comes to storage.

Is there a way to change de data stream name for filebeat?

---

<div class="post-metadata">

**Author:** ![Shaoranlaos](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@Shaoranlaos](https://discuss.elastic.co/u/Shaoranlaos)\
**Post date:** [April 5, 2022, 7:24am UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/2 "2022-04-05T07:24:09Z")

</div>

Hello Andrés,

You can change the ilm policy the filebeat is targeting:

> **[Configure index lifecycle management | Filebeat Reference \[8.1\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/ilm.html)**

With this the underling index and retention is also changed.

Hope this helps

---

<div class="post-metadata">

**Author:** ![Andres\_Altamirano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andres_altamirano/32/94143_2.png) [@Andres\_Altamirano](https://discuss.elastic.co/u/Andres_Altamirano)\
**Post date:** [April 5, 2022, 1:20pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/3 "2022-04-05T13:20:04Z")

</div>

Thanks @Shaoranlaos

May be I wasn't clear. We need to change index name.

The thing is that rollover\_alias has been removed, so I see no way to change the index name from ILM policy.

---

<div class="post-metadata">

**Author:** ![Andres\_Altamirano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andres_altamirano/32/94143_2.png) [@Andres\_Altamirano](https://discuss.elastic.co/u/Andres_Altamirano)\
**Post date:** [April 13, 2022, 3:33pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/4 "2022-04-13T15:33:31Z")

</div>

In case somebody reads this,

Could be an option to set filebeat output to send events to logstash?  
It seems possible to handle index name and ilm in a more flexible way form logstash than filebeat.

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [April 13, 2022, 3:39pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/5 "2022-04-13T15:39:19Z")

</div>

@Andres_Altamirano ,

A solution might be to add a source or app id into the filebeat configuration for the input and the use logstash to split the sources into separate indices or streams, using IF / ELSE statements in Logstash.Output.

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [April 13, 2022, 3:54pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/6 "2022-04-13T15:54:23Z")

</div>

Here is a filebeat example that creates and writes to datastream filebeat-8.x.x-elastic-internal, index names are like .ds-filebeat-8.1.2-elastic-internal-2022.04.04-000001

```auto
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0

setup.template.name: "filebeat-%{[beat.version]}-elastic-internal"
setup.template.pattern: "filebeat-%{[beat.version]}-elastic-internal*"

setup.ilm.enabled: true
setup.ilm.rollover_alias: "filebeat-%{[agent.version]}-elastic-internal"
setup.ilm.policy_name: "roll_daily_delete_30"

setup.kibana:
  host: "https://...:5601"
 
output.elasticsearch:
  hosts: ["https://...:9200"]
  index: "filebeat-%{[agent.version]}-elastic-internal"
  username: "..."
  password: "..."
  ssl.certificate_authorities: ["/etc/filebeat/..."]
  ssl.certificate: "/etc/filebeat/..."
  ssl.key: "/etc/filebeat/..."

processors:
  - add_host_metadata: ~
  - add_locale: ~

logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

monitoring.enabled: false
monitoring.cluster_uuid: ...

http.enabled: true

monitoring.elasticsearch:

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/filebeat/..."]
setup.kibana.ssl.certificate: "/etc/filebeat/..."
setup.kibana.ssl.key: "/etc/filebeat/..."

```

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [April 13, 2022, 3:58pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/7 "2022-04-13T15:58:35Z")

</div>

Edited, my ILM policy was incorrect, I fixed it and rollover is happening.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 11, 2022, 5:58pm UTC](https://discuss.elastic.co/t/change-data-stream-name-in-filebeat-8/301531/8 "2022-05-11T17:58:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
