# Questions about Filebeat 7.x to 8.x upgrade regarding data streams

**URL:** <https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 4, 2025, 5:49pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736 "2025-08-04T17:49:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![NominaSumpta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nominasumpta/32/52412_2.png) [@NominaSumpta](https://discuss.elastic.co/u/NominaSumpta)\
**Post date:** [August 4, 2025, 5:49pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736/1 "2025-08-04T17:49:13Z")

</div>

Hi,

I have a couple questions regarding the 7.x to 8.x upgrade for Beats/Filebeat.

1. 

In Beats/Filebeat 8.x, data streams are used instead of indexes when ingesting data to Elasticsearch.

Question: does this apply only to the Elasticsearch output? ([Configure the Elasticsearch output | Beats](https://www.elastic.co/docs/reference/beats/filebeat/elasticsearch-output))

My Filebeat outputs to Logstash (which, in turn, outputs to Elasticsearch). It's unclear to me whether the 'data stream change' affects that case.

1. 

[Upgrade | Beats Platform Reference [8.19] | Elastic](https://www.elastic.co/guide/en/beats/libbeat/8.19/upgrading.html#upgrade-index-template) says:

> Starting in version 8.0, the default Elasticsearch index templates configure data streams instead of traditional Elasticsearch indices. [...] To use data streams, load the default index templates

... followed by the command `beatname setup --index-management`.

Questions:

- Should `beatname` be replaced by `filebeat` (in my case), or is `beatname` an existent binary?
- Does this command need to be run on a single Filebeat instance? (I would assume so, if all it does is change an index template on the cluster.)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 4, 2025, 8:23pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736/2 "2025-08-04T20:23:48Z")

</div>

Hi @NominaSumpta

This is a pretty good guide .... This Should work for later versions of 8.x

> **[Ingest data from Beats with Logstash as a proxy | Elastic Docs](https://www.elastic.co/docs/manage-data/ingest/ingesting-data-from-applications/ingest-data-from-beats-to-elasticsearch-service-with-logstash-as-proxy)**
>
> This guide explains how to ingest data from Filebeat and Metricbeat to Logstash as an intermediary, and then send that data to your Elastic Cloud Hosted...

To answer your specific questions above

> [@NominaSumpta](#):
>
> Should `beatname` be replaced by `filebeat` (in my case), or is `beatname` an existent binary?

`filebeat`

> [@NominaSumpta](#):
>
> Does this command need to be run on a single Filebeat instance? (I would assume so, if all it does is change an index template on the cluster.)

The setup commands should be run once each time you have a new beat configuration, version , module etc

Pro.tip..Also you should just run

`filebeat setup -e` which sets up all assets instead of trying to setup separate assets like `index-management` you will need to have that setup filebrat pointed at Kibana as well

---

<div class="post-metadata">

**Author:** ![NominaSumpta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nominasumpta/32/52412_2.png) [@NominaSumpta](https://discuss.elastic.co/u/NominaSumpta)\
**Post date:** [August 18, 2025, 6:00pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736/3 "2025-08-18T18:00:00Z")

</div>

Hi Stephen,

That actually answers none of my concrete questions…

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 18, 2025, 6:44pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736/4 "2025-08-18T18:44:30Z")

</div>

> [@NominaSumpta](#):
>
> That actually answers none of my concrete questions…

Hmmmm I specifically answered the following...

> To answer your specific questions above
> 
> > [@NominaSumpta](#):
> >
> > Should `beatname` be replaced by `filebeat` (in my case), or is `beatname` an existent binary?
> 
> `filebeat`
> 
> > [@NominaSumpta](#):
> >
> > Does this command need to be run on a single Filebeat instance? (I would assume so, if all it does is change an index template on the cluster.)
> 
> The setup commands should be run once each time you have a new beat configuration, version , module etc
> 
> Pro.tip..Also you should just run
> 
> `filebeat setup -e` which sets up all assets instead of trying to setup separate assets like `index-management` you will need to have that setup filebrat pointed at Kibana as well

With respect to data streams....

## When you use beats -\> logstash -\> elasticsearch in 8.x

Whether data streams are used is defined by the [logstash elasticsearch output configuration](https://www.elastic.co/docs/reference/logstash/plugins/plugins-outputs-elasticsearch) not the beats configuration.

**Do you WANT to used datastreams (which I would recommend) or not?**

> **[Configure the Logstash output | Beats](https://www.elastic.co/docs/reference/beats/filebeat/logstash-output)**
>
> The Logstash output sends events directly to Logstash by using the lumberjack protocol, which runs over TCP. Logstash allows for additional processing...

This [Winlogbeat example](https://www.elastic.co/docs/reference/logstash/working-with-winlogbeat-modules) shows a good configurations example

This should create data stream with beats output \> logstash input \> logstash elasticsearch output -\> elasticsearch

```auto
input {
  beats {
    port => 5044
  }
}

output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "https://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create" <<< IMPORTANT!!
      pipeline => "%{[@metadata][pipeline]}"
      user => "elastic"
      password => "secret"
    }
  } else {
    elasticsearch {
      hosts => "https://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create" <<< IMPORTANT!!
      user => "elastic"
      password => "secret"
    }
  }
}

```

I would recommend setting up and testing

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 18, 2025, 10:25pm UTC](https://discuss.elastic.co/t/questions-about-filebeat-7-x-to-8-x-upgrade-regarding-data-streams/380736/5 "2025-08-18T22:25:35Z")

</div>

BTW I tested 8.19.2

I ran this config

I ran `filebeat setup -e` first pointing to Kibana and Elasticsearch then changed to Logstash output

```auto

filebeat.inputs:

- type: filestream

  # Unique ID among all inputs, an ID is required.
  id: my-filestream-id

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

# setup.kibana:

# ---------------------------- Elasticsearch Output ----------------------------
# output.elasticsearch:
# # Array of hosts to connect to.
# hosts: ["localhost:9200"]

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  #The Logstash hosts
  hosts: ["localhost:5044"]

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

```

I ran the following Logtash.conf

```auto
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create" 
      pipeline => "%{[@metadata][pipeline]}"
      user => "elastic"
      password => "secret"
    }
  } else {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create" 
      user => "elastic"
      password => "secret"
    }
  }
}

```

Created a data stream

 ![Screenshot 2025-08-18 at 3.21.47 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/1/31099f837ed1a0d6bbe80c210a64a0d45fec5c31.jpeg)

 ![Screenshot 2025-08-18 at 3.24.55 PM](https://us1.discourse-cdn.com/elastic/original/3X/e/b/ebbc75e6b30d686031af3b328016ef6fab41b4a9.jpeg)