# Logstash elasticsearch output - data\_stream\_auto\_routing

**URL:** https://discuss.elastic.co/t/logstash-elasticsearch-output-data-stream-auto-routing/297624
**Category:** Logstash
**Created:** [February 18, 2022, 4:24pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-data-stream-auto-routing/297624 "2022-02-18T16:24:10Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![mybyte](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mybyte/32/99989_2.png) [@mybyte](https://discuss.elastic.co/u/mybyte)
#### Post date: [February 18, 2022, 4:24pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-data-stream-auto-routing/297624/1 "2022-02-18T16:24:10Z")

</div>

I'm a bit confused about the data\_stream\_auto\_routing option when using elasticsearch output to data streams in logstash.

According to documentation:

> Automatically routes events by deriving the data stream name using specific event fields with the `%{[data_stream][type]}-%{[data_stream][dataset]}-%{[data_stream][namespace]}` format.

My output config:

```auto
output {
  stdout {}
  elasticsearch {
    hosts => ["${ES_HOSTS}"]    
    user => "${ES_USER}"
    password => "${ES_PASS}"
    ssl => "true"
    data_stream => "true"   
    data_stream_auto_routing => "true"
  }
}

```

So I was under the assumption that when logging from different sources, they could all send their own name spaces via `data_stream.namespace` and logstash would send the output to different data\_streams, i.e. `logs-generic-app1`, `logs-genetic-app2` and so forth...

Instead, it seems that all events are being sent to `logs-generic-default` while also throwing an error:

```auto
 "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Mapper for [data_stream.namespace] conflicts with existing mapper:\n\tCannot update parameter [value] from [default] to [app1]"}}}}

```

Am I completely misunderstanding how this is supposed to work?

---

<div class="post-metadata">

### Author: ![mybyte](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mybyte/32/99989_2.png) [@mybyte](https://discuss.elastic.co/u/mybyte)
#### Post date: [February 21, 2022, 10:58am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-data-stream-auto-routing/297624/2 "2022-02-21T10:58:42Z")

</div>

Never mind... Figured it out. For future reference:

This only works when the data is a literal object in logstash, meaninig

```auto
  "data_stream": { "dataset": "some_set", "namespace": "namespace" } 

```

The incoming fields were dotted (`data_stream.namespace:something`), so logstash completely ignored them, while Elasticsearch was treating them as nested fields...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 21, 2022, 10:59am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-data-stream-auto-routing/297624/3 "2022-03-21T10:59:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
