# Dynamically change the index for Elasticsearch in Logstash output

**URL:** <https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788>\
**Category:** Logstash\
**Created:** [August 12, 2020, 11:30pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788 "2020-08-12T23:30:10Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![udesh](https://avatars.discourse-cdn.com/v4/letter/u/7ab992/32.png) [@udesh](https://discuss.elastic.co/u/udesh)\
**Post date:** [August 12, 2020, 11:30pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/1 "2020-08-12T23:30:10Z")

</div>

'Hi All,  
I'm trying to change the logstash.config file according to a log file. The log file consists three different types of log records which generated through my java application.  
logstash.config file is as follows:

# logstash configuration

input {  
beats {  
port =\> 5044

}  
}

filter {

```
if[fields][messagetype] == "customer_request"{
	grok{
		match =>{
			"message" => ["%{WORD:apptime}::%{WORD:messagetype}::%{WORD:correlationId}::%{WORD:user_id}::%{WORD:user_gender}::%{WORD:user_type}::%{WORD:message}"]
		}						   
	}
	mutate{
		add_field => ["index_key" => "customer-request"]
	}
}	

```

}

output {  
stdout {  
codec =\> rubydebug  
}

```
  elasticsearch {
  index => "%{[index_key]}-%{+YYYY.MM.dd}"
      hosts => ["localhost:9200"]
  }

```

}

But the issue is index\_key related value is not assigning which I added in add\_filed in mutate section. It shows as %{[index\_key]}-2020-08-10. I'm new to ELK stack and if anyone can help me, that would be a great.  
Thanks...!

'

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2020, 11:34pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/2 "2020-08-12T23:34:51Z")

</div>

> [@udesh](#):
>
> It shows as %{[index\_key]}-2020-08-10.

In Kibana, look at some of the records in that index. Do they have an index\_key field?

---

<div class="post-metadata">

**Author:** ![aditya.p](https://avatars.discourse-cdn.com/v4/letter/a/848f3c/32.png) [@aditya.p](https://discuss.elastic.co/u/aditya.p)\
**Post date:** [August 13, 2020, 7:35am UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/3 "2020-08-13T07:35:59Z")

</div>

You need to send index\_key without rectangular brace.  
Only in your filter block field names need to be passed in `[]`

Use this:

`index => "%{index_key}-%{+YYYY.MM.dd}"`

---

<div class="post-metadata">

**Author:** ![udesh](https://avatars.discourse-cdn.com/v4/letter/u/7ab992/32.png) [@udesh](https://discuss.elastic.co/u/udesh)\
**Post date:** [August 13, 2020, 8:24am UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/4 "2020-08-13T08:24:48Z")

</div>

I changed it to index =\> "%{index\_key}-%{+YYYY.MM.dd}" . But it does not work.

 ![disss](https://us1.discourse-cdn.com/elastic/original/3X/2/5/255776830cc648cd23d0ad68e115f1c1d2f899db.png)

---

<div class="post-metadata">

**Author:** ![aditya.p](https://avatars.discourse-cdn.com/v4/letter/a/848f3c/32.png) [@aditya.p](https://discuss.elastic.co/u/aditya.p)\
**Post date:** [August 13, 2020, 11:26pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/5 "2020-08-13T23:26:47Z")

</div>

> [@udesh](#):
>
> But it does not work.

What exactly you mean by this. "Docs count" refers to the individual events coming in your index. The health being "yellow" means your cluster is unstable. Verify your input block as the output block seems fine.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 13, 2020, 11:47pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/6 "2020-08-13T23:47:31Z")

</div>

Again, look at the documents in the index called "%{index\_key}-2020.08.13". Do they have a field called index\_key? If they do not, then this is exactly what you should expect.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 10, 2020, 11:47pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788/7 "2020-09-10T23:47:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.