# Use field name in datastream namespace or dataset
**URL:** https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188
**Category:** Logstash
**Created:** [April 12, 2022, 8:15am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188 "2022-04-12T08:15:55Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@sirReeall](https://discuss.elastic.co/u/sirReeall)
#### Post date: [April 12, 2022, 8:15am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/1 "2022-04-12T08:15:55Z")
I'm trying to use a field with logstash output to Elasticsearch but I can't seem to get this to work. My configuration is as below:
```auto
output {
elasticsearch {
hosts => "es01"
data_stream => "true"
data_stream_namespace => "%{[ticket_number]}"
}
}
```
The resulting index has the name:
```auto
logs-generic-%{[ticket_number]}
```
I'm not sure that dynamic fields are supported or not? Maybe the syntax is incorrect?
---
### Author:  [@sirReeall](https://discuss.elastic.co/u/sirReeall)
#### Post date: [April 12, 2022, 8:40am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/2 "2022-04-12T08:40:21Z")
I tried a few different approaches too, the following didn't work:
```auto
"%[ticket_number]"
"%{ticket_number}"
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 12, 2022, 2:49pm UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/3 "2022-04-12T14:49:04Z")
> [@sirReeall](#):
>
> I'm not sure that dynamic fields are supported or not?
The output [does not sprintf](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/48ddb890ce350afa3ddbc8602bb282c9c185aebf/lib/logstash/outputs/elasticsearch/data_stream_support.rb#L44) the datastream name, but you can use the [data\_stream\_auto\_routing](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data_stream_auto_routing) option to tell the output to reference fields on the event for parts of the stream name.
---
### Author:  [@sirReeall](https://discuss.elastic.co/u/sirReeall)
#### Post date: [April 13, 2022, 11:02am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/4 "2022-04-13T11:02:14Z")
Hey @Badger - Thanks for the source code link, I was looking for this and couldn't find it.
I've tried your suggestion with `data_stream_auto_routing`, but I receive an error. My configuration looks like this:
```auto
filter {
// set ticket_number based on grok filter
grok {
match => {
"path" => "%{GREEDYDATA}/data/%{GREEDYDATA:[ticket_number]}/%{GREEDYDATA:[node_name]}/logs/%{GREEDYDATA}\.log"
}
}
mutate {
replace => {"data_stream.dataset" => "%{ticket_number}"}
}
}
output {
elasticsearch {
hosts => "es01"
data_stream => "true"
data_stream_auto_routing => "true"
}
}
```
The error I receive is as follows:
```auto
[2022-04-13T10:48:54,447][WARN][logstash.outputs.elasticsearch][debug-logs][b1d4c5c8928af2ea4daa905af544039dc325d7cc8c34d4e82fd26795b0a83810] Could not index event to Elast
icsearch. {
:status=>400,
:action=>[
"create",
{
:_id=>nil,
:_index=>"logs-generic-default",
:routing=>nil
},
{
"javaclass"=>"o.n.k.i.t.l.c.CheckPointerImpl",
"@version"=>"1",
"ticket_number"=
>"123456",
"node_name"=>"node1",
"message"=>"[system/00000000] Checkpoint triggered by \"Store copy\" @ txId: 73 checkpoint started...",
"tags"=>[
"debug_log",
"_grokparsefailure"
], "data_
stream.dataset"=>"123456", "path"=>"/home/logstash/data/123456/node1/logs/mylog.log", "loglevel"=>"INFO", "@timestamp"=>2022-04-04T10:21:16.563Z, "data_stream"=>{"type"=>"logs", "dataset"
=>"generic",
"namespace"=>"default"
}
}
],
:response=>{
"create"=>{
"_index"=>".ds-logs-generic-default-2022.04.13-000001",
"_type"=>"_doc",
"_id"=>"spKKIoABpJlrM6S2W1GG",
"status"=>400, "erro
r"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [data_stream.dataset
] of type [constant_keyword
] in document with id 'spKKIoABpJlrM6S2W1GG'. Preview of field's v
alue: '123456'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"[constant_keyword
] field [data_stream.dataset
] only accepts values that are equal to the value defined in t
he mappings [generic
], but got [
123456
]"}}}}}
```
Looking at the above I see the following ` :_index=>"logs-generic-default"`, which tells me that the index is already created somehome, and this event is already destined for and index called `logs-generic-default` rather then `logs-123456-default`
I reviewed the logs in prior to the errors and see that maybe this could be something to do with ecs\_compatability and ootb templates... but I'm not sure. Here is what I noticed in the logs:
```auto
[2022-04-13T10:48:50,847][INFO][logstash.outputs.elasticsearch][mylog-logs] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2022-04-13T10:48:50,888][WARN][deprecation.logstash.filters.grok][mylog-logs] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major
release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2022-04-13T10:48:50,934][INFO][logstash.outputs.elasticsearch][mylogs-logs] Installing Elasticsearch template {:name=>"logstash"}
```
Do I need to set some additional configuration to make this work? I'm not sure what I am missing....
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 13, 2022, 6:04pm UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/5 "2022-04-13T18:04:12Z")
> [@sirReeall](#):
>
> `replace => {"data_stream.dataset" =>`
logstash does not use the same syntax as kibana and elasticsearch. It can distinguish between a field with a . in its name and a nested field. Use `[data_stream][dataset]`.
---
### Author:  [@sirReeall](https://discuss.elastic.co/u/sirReeall)
#### Post date: [April 14, 2022, 8:17am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/6 "2022-04-14T08:17:11Z")
That worked a treat 👍
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 12, 2022, 8:17am UTC](https://discuss.elastic.co/t/use-field-name-in-datastream-namespace-or-dataset/302188/7 "2022-05-12T08:17:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.