# Manage several data stream(s) in the elasitcsearch output with interpolation

**URL:** <https://discuss.elastic.co/t/manage-several-data-stream-s-in-the-elasitcsearch-output-with-interpolation/332981>\
**Category:** Logstash\
**Tags:** datastreams\
**Created:** [May 9, 2023, 5:39pm UTC](https://discuss.elastic.co/t/manage-several-data-stream-s-in-the-elasitcsearch-output-with-interpolation/332981 "2023-05-09T17:39:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pascal\_Nuccio](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pascal_nuccio/32/120817_2.png) [@Pascal\_Nuccio](https://discuss.elastic.co/u/Pascal_Nuccio)\
**Post date:** [May 9, 2023, 5:39pm UTC](https://discuss.elastic.co/t/manage-several-data-stream-s-in-the-elasitcsearch-output-with-interpolation/332981/1 "2023-05-09T17:39:58Z")

</div>

If you need to manage several data streams for one LOGSTASH instance, you can configure the elasticsearch output like this in your logstash configuration:

We must use a filter to configure the data\_stream parameters (type, dataset and namespace)

```auto
filter: |-
  mutate {
    add_field => {
      "[data_stream][type]" => "logs"
      "[data_stream][dataset]" => "filebeat-lexi-%{[env_name]}"
      "[data_stream][namespace]" => "%{[@metadata][version]}"
    }
  }

output: |-
      elasticsearch {
            data_stream => true

        }

```

---

<div class="post-metadata">

**Author:** ![Ugo\_Sangiorgi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ugo_sangiorgi/32/146361_2.png) [@Ugo\_Sangiorgi](https://discuss.elastic.co/u/Ugo_Sangiorgi)\
**Post date:** [May 9, 2023, 6:23pm UTC](https://discuss.elastic.co/t/manage-several-data-stream-s-in-the-elasitcsearch-output-with-interpolation/332981/2 "2023-05-09T18:23:51Z")

</div>

> Badly formatted index, after interpolation still contains placeholder

Just to give context, in case you are facing the error above, its likely because Elasticsearch output does not accept field names to compose the data\_stream name like it does for a normal "index" name.

This will not work, for instance:

```auto
data_stream => true
data_stream_type => "logs"
data_stream_dataset => "%{[my_field]}"
data_stream_namespace => "%{[@metadata][version]}"

```

You should instead add the fields in a mutate filter, in the `filter` section, like @Pascal_Nuccio mentioned:

```auto
mutate {
    add_field => {
      "[data_stream][type]" => "logs"
      "[data_stream][dataset]" => "%{[my_field]}"
      "[data_stream][namespace]" => "%{[@metadata][version]}"
    }
  }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2023, 6:23pm UTC](https://discuss.elastic.co/t/manage-several-data-stream-s-in-the-elasitcsearch-output-with-interpolation/332981/3 "2023-06-06T18:23:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.