# Logstash output to Elasticsearch name index based on field?

**URL:** <https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584>\
**Category:** Logstash\
**Created:** [February 17, 2021, 1:46pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584 "2021-02-17T13:46:51Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 1:46pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/1 "2021-02-17T13:46:51Z")

</div>

Hello. Im trying to name a index based on a field but it doesnt seem to work:

output  
{

```
elasticsearch {

```

hosts =\> ["localhost"]  
user =\> ["elastic"]  
password =\> ["pass"]

index =\> "winlogbeat-server-winapplication-%{[event][provider]}-%{+yyyy.MM.dd}"

```
    }

```

}

As you see, Im trying even a generic one and it does not produce the index.

How can I do this?

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 2:02pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/2 "2021-02-17T14:02:27Z")

</div>

Tried index =\> "winlogbeat-mssql-winapplication-%{[\_source][event][provider]}-%{+yyyy.MM.dd}"  
but no dice either

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 2:06pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/3 "2021-02-17T14:06:23Z")

</div>

Amazing. I dont understand why but:

filter {

```
mutate {

    lowercase => ["[event][provider]" ]

}

```

}

I tried this because same thing happens to the hostname and I need to lowercase it.

Why?

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [February 17, 2021, 2:12pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/4 "2021-02-17T14:12:13Z")

</div>

Yes elasticsearch allow only lowercase name for indexes

Index name must be lowercase, cannot begin with an underscore, and cannot contain commas

> **[Document Metadata | Elasticsearch: The Definitive Guide \[2.x\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/current/_document_metadata.html)**

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 2:32pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/5 "2021-02-17T14:32:13Z")

</div>

Is there a way to instead of

filter {

```auto
mutate {

    lowercase => ["[event][provider]" ]

}

```

}

Add a additional field called [eventproviderlower], insert the value of [event][provider] and use that? Its mostly not to mutate or touch the logs in any way shape or form (compilance and such)

Thanks

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 17, 2021, 2:41pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/6 "2021-02-17T14:41:31Z")

</div>

You can use the mutate filter to create a new field, lowercase it and use it in your output, you can use a `[@metadata]` field for it, the `[@metadata]` is only present in your pipeline, it will not be in your final document.

You need something like this:

```auto
mutate {
    copy => { "[event][provider]" => "[@metadata][eventlower]" }
}
mutate {
    lowercase => ["[@metadata][eventlower]" ]
}

```

You need two different mutates because of the order which the filters are executed internally by mutate.

And in your output you need to use `%{[@metadata][eventlower]}`

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 3:57pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/7 "2021-02-17T15:57:12Z")

</div>

Thanks a lot 🙂 Great idea

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 17, 2021, 4:11pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/8 "2021-02-17T16:11:21Z")

</div>

This is what you mean correct

mutate {  
copy =\> { "[host][hostname]" =\> "[@metadata][hostlower]" }  
}  
mutate {  
lowercase =\> ["[@metadata][hostlower]" ]  
}

output  
{

```
elasticsearch {

```

hosts =\> ["localhost"]  
user =\> ["elastic"]  
password =\> ["4234"]

index =\> "winlogbeat-winapplication-%{[@metadata][hostlower]}-%{+yyyy.MM.dd}"

```
    }

```

}

Because I just implemented it and it doesnt seem to be working

(I used hostname as another example)

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 18, 2021, 9:47am UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/9 "2021-02-18T09:47:25Z")

</div>

No ideas?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 18, 2021, 6:08pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/10 "2021-02-18T18:08:05Z")

</div>

> [@riahc3](#):
>
> it doesnt seem to be working

What exactly does that mean?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 18, 2021, 6:36pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/11 "2021-02-18T18:36:06Z")

</div>

The config seems right, do you have anything in the logs? Share your full pipeline and a sample log, your problem could be in other parts of your pipeline.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [February 23, 2021, 4:18pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/12 "2021-02-23T16:18:24Z")

</div>

Figured out the issue.

I had to open and close a filter section:

filter  
{  
mutate {  
copy =\> { "[host][hostname]" =\> "[@metadata][hostlower]" }  
}  
mutate {  
lowercase =\> ["[@metadata][hostlower]" ]  
}

}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 23, 2021, 4:18pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-name-index-based-on-field/264584/13 "2021-03-23T16:18:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.